欢迎来到天天文库
浏览记录
ID:39255239
大小:1.78 MB
页数:32页
时间:2019-06-28
《[HiC]The Challenges and Opportunities in Interfacing Hadoop with Condor英文学习材料》由会员上传分享,免费在线阅读,更多相关内容在学术论文-天天文库。
1、TheChallengesandOpportunitiesinInterfacingHadoopwithCondorMironLivnyCenterforHighThroughputComputingMorgridgeInstituteforResearchandUniversityofWisconsin-MadisonTechnologyAdoptionHadoop=MR+HDFSwhereMR«HDFS**6Tier-2sitesofUS-CMSareoperating2PBHDFSfacilitieseachCondor-Users11/11Ihavesomeusersw
2、hoareinterestedinrunningHadoopjobsonourCondorcluster…I'mwonderingifanyonecanpointmetosomemoredetailedinformationonhowtogetthisgoing?…DavidBrodbeck,SystemAdministrator,LinguisticsUniversityofWashingtonIhavestartedtouseit.Ithinkmapreduceisirrelevantwhenyouusecondor.However,HDFSisextemlyusefulf
3、orstreaminglargedata(filerbiggerthan300mb).Ritarmorgan466@gmail.comItsaphase.Nowdayseveryoneasks,DoesithaveHadoop?peoplenotevenknowingwhatitisanddoes.Granted,HDFSisnice.MagGammagawake@gmail.comAteamofcomputerscientistsfromtheUniversityofWisconsin-MadisonandtheUniversityofMarylandrecentlyasse
4、mbledafullhumangenomefrommillionsofpiecesofdata—steppingupfromcommonlyassembledgenomesseveralordersofmagnitudelesscomplex—andtheydiditwithoutabig-ticketsupercomputer.…"It'stwoplustwoequalsfive,ifyouwill,"Tannenbaumsays."CondorintegratedwithHadoopisasoftwaresystempowerfulenoughtotackleproblem
5、sascomplexashumangenomeassembly…HighThroughputComputingWefirstintroducedthedistinctionbetweenHighPerformanceComputing(HPC)andHighThroughputComputing(HTC)inaseminarattheNASAGoddardFlightCenterinJulyof1996andamonthlaterattheEuropeanLaboratoryforParticlePhysics(CERN).InJuneof1997HPCWirepublishe
6、daninterviewonHighThroughputComputing.WhyHTC?Formanyexperimentalscientists,scientificprogressandqualityofresearcharestronglylinkedtocomputingthroughput.Inotherwords,theyarelessconcernedaboutinstantaneouscomputingpower.Instead,whatmatterstothemistheamountofcomputingtheycanharnessoveramonthora
7、year---theymeasurecomputingpowerinunitsofscenariosperday,windpatternsperweek,instructionssetspermonth,orcrystalconfigurationsperyear.HighThroughputComputingisa24-7-365activityFLOPY(60*60*24*7*52)*FLOPSMapReducebroughtback(legitimized)distributedco
此文档下载收益归作者所有