欢迎来到天天文库
浏览记录
ID:46776902
大小:614.03 KB
页数:37页
时间:2019-11-27
《HiC2010-Adaptive_and_Scalable_Metadata_Manage...》由会员上传分享,免费在线阅读,更多相关内容在学术论文-天天文库。
1、AdaptiveandScalableMetadataManagementtoSupportATrillionFileJingXingNationalResearchCenterForIntelligentComputingSystemsInstituteofComputingTechnologyChineseAcademyofSciencesOutline1.Introduction2.Ourapproach3.Evaluation4.Conclusion5.FutureWorksINTRODUCTIONIntroductionOurapproac
2、hEvaluationConclusionFutureworkTrendofDataProcessingAsthedevelopmentofglobalinformatization,moreandmoredataneedtobestored.(InternetService,BusinessTransaction,PersonalLife)Ontheotherhand,thepopularizationofInternetapplications,theincrementofscientificcomputingrequirement,andthe
3、enhancementofhardware,weneedmuchhigherdataprocessingperformance.FromIDC&EMCFromPDSIHIC2010BeijingXINGJINGNCIC@ICT@CASIntroductionOurapproachEvaluationConclusionFutureworkTrendofMetadataProcessingMetadataprocessingrequirementincreaseswithdataprocessingrequirement.Onthefieldsofsc
4、alabilityandperformance.•WithIncreaseddatastorage,metadataserversneedtoefficientlyorganizeandmaintainlargenumberoffiles--Facebook550kops/sec--Flickr38kops/sec--LSST9*10^18files•DifferentmetadataserversneedtobebalancedFromIDC&EMCdynamically--dynamicworkload--dynamicserverconfigu
5、rationHIC2010BeijingXINGJINGNCIC@ICT@CASIntroductionOurapproachEvaluationConclusionFutureworkWhatmatermostindistributedmetadatamanagement?1.Howtodistributenamespaceonmultiplemetadataservers•Thedefinitionofmetadataprocessingunit•Thelocatingmechanismofmetadataprocessingunit•None-
6、partition/Sub-treepartition/sub-dirpartition/filepartition2.Howtoimprovetheconcurrencyofmetadataprocessing3.Howtobetterutilizemetadatacache4.HowtobalanceworkloadamongmultiplemetadataserversHIC2010BeijingXINGJINGNCIC@ICT@CASIntroductionOurapproachEvaluationConclusionFutureworkNo
7、nepartition•Example(GoogleFS,HDFS)•Thereisnometadatapartitioninthisarchitecture.•Severalcopyofnamespacemetadataismaintainedinmultiplemetadataserverstoensurereliability.•Metadataiskeptinmemorytoimprovetheperformance.•WeakscalabilityatbothscaleandperformanceHIC2010BeijingXINGJING
8、NCIC@ICT@CASIntroductionOurapproachEvaluationConclusio
此文档下载收益归作者所有
点击更多查看相关文章~~