HiC2009-Using_Simulation_for_Large-scale_Distributed_System

《HiC2009-Using_Simulation_for_Large-scale_Distributed_System》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、MumakUsingSimulationforLarge-scaleDistributedSystemVerificationandDebuggingHongTang2009.11-HadoopWorld,BeijingHadoopPerformanceEngineering@Yahoo!ProductionCloudInvestigation-optimizationCycleAnalyticsBenchmarkingImplementationEvaluationToolsMetricsCollectionBenchmarkDevelopm

2、entDevelopmentOutlineMotivationsOverviewandStatusArchitectureLessonsandExperiencesConclusionsandFutureWorkMotivationsLarge-scaledistributedsystemishardtoverifyanddebug-Cannotafforda2000-nodeclusterforeverydeveloper,totesteveryfeatureenhancementorbugfix-Timeconsumingtor

3、unbenchmarks-HardtoreproducecornercaseconditionsMotivations(cont.)JobTrackerisafertileareaforexperimentation-Schedulingpolicies–wehavefourschedulersalready-SynergybetweenMapReduceandHDFS(e.g.blockplacementpolicies)-Improvingspeculativeexecutionpolicies-WEWANTMOREPEOPLETOHEL

4、PUSINNOVATE!But,JobTrackeristoocomplextomodifycorrectly-Manyfactorstoconsider:fairness,capacity/SLAguarantees,datalocality,loadbalance,failurehandlingandrecovery,etc-Manycontrolknobsincurrentimplementationwithsubtleinteractions–cantrickevenseasonedHadoopdevelopers-Generalad

5、vicetopeoplenewtoHadoopMapReducecodebase:DONOTPLAYWITHFIRE!Mumak–TheVisionDiscrete-eventsimulation-Cansimulateaclusterwiththousandsofnodesinoneprocess-DoesnotperformactualIOorcomputation-Virtualclock“spins”fasterthanwallclockPluggingintherealJobTrackerandSchedulercode-Good

6、:Noneedtoseparatelymodeltheschedulingpolicies-Bad/Good:InheritbothfeaturesandbugsinJTandSchedulerSimulateallconditionsofaproductioncluster-WorkloadandclusterconfigurationdrivenbytracegeneratedbyRumen-Jobsubmission,inter-arrival,jobdependencies,high-ramjobs-Allkindsoffailure

7、sandfailurerecoverylogic-ResourcecontentionProjectStatusWork-in-progressFirst-cutversioncommittedtoHadoop0.21andtrunk-Basicframework-Nomodelingofresourcecontention•Taskexecutiontimetakenas-isfromtraces-Onlyindividualtaskfailures,nonodefailuresorfailurecorrelations-Nojobdep

8、endencies-NospeculativeexecutionTheTeam-YahooHadoopPerformanceEngineeringT