欢迎来到天天文库
浏览记录
ID:40659179
大小:138.02 KB
页数:8页
时间:2019-08-05
《Multi-Query Optimization for Parallel Data》由会员上传分享,免费在线阅读,更多相关内容在学术论文-天天文库。
1、Multi-QueryOptimizationforParallelData owSystemsPeterAlvaro,NeilConway,andAndrewKrioukovMay14,2009Abstractandeachworkernodeisassignedoneormoretasks.Whenaworkercompletesatask,HadoopassignstheExistingparalleldata owsystemsarestrictlyreac-workeranothertasktoexecute,usingsomesimpletiveinth
2、eiroptimizations.Atbest,suchapproachesheuristicsthattrytoplacecomputationsclose"toapproximatetheoptimalstrategy,missingopportuni-theirinputdata.However,noglobalanalysisisper-tiestooptimizeacrossmultiplequeriesandresched-formedtoattempttocolocatedataandcomputation,ulequeriestoimprovelo
3、cality.Weproposethreetopredicttheoptimalscheduleforjobsandtasks,ortechniquesthatimprovequeryexecutionperformancetoshareworkbetweensimilarjobs.byutilizinghigh-levelknowledgeoftheworkload.WhenMap-Reduceisusedastheexecutionplat-Thersttechniquepredictivelyreplicatesdatatoim-formforadeclar
4、ativequerylanguagesuchasHiveproveaggregatereadbandwidthandincreaselocal-orPig,therearemoreopportunitiesforoptimiza-ity.Thesecondtechniquereorderssimilarqueriestion[13].However,currentsystemsimplementonlytoimprovecacheperformance.Thethirdtechniquesimple,conservativeoptimizations:forexam
5、ple,Pigschedulesmultiplequeriesinparalleltoimprovere-canpushdistributiveoralgebraicaggregateevalua-sourceutilization.Weevaluatethesetechniquesus-tionbeneathjoins[14],andHiveappliesprojectioningApacheHiveonAmazonEC2andshowperfor-andselectionpushdown[19].Inparticular,noneofmanceimproveme
6、ntsof5%to50%.thesesystemscurrentlyperformsmultiple-queryop-timization.1IntroductionTheprimitivestateofqueryoptimizationforthesesystemsisonlypartlyduetotheirrelativeimma-Paralleldata owsystemssuchasMap-Reduce[6]andturity.Thereisalsoaphilosophicaldierencebe-Hadoop[2]haverecentlyexperien
7、cedasurgeinpop-tweenthesesystemsandparallelrelationaldatabases.ularity.ThesesystemsareincreasinglyusedfordataTheoptimizationsappliedbytraditionaldatabaseswarehousingandanalytics,eitherdirectlyorthroughrequireanaccuratemodelthatpredictstheruntimetheuseofahigh-levelquerylanguagethatisc
此文档下载收益归作者所有