The Google File System (GFS)谷歌文件系统(GFS).pdf

《The Google File System (GFS)谷歌文件系统(GFS).pdf》由会员上传分享，免费在线阅读，更多相关内容在应用文档-天天文库。

1、TheGoogleFileSystem(GFS)SanjayGhemawat,HowardGobioff,andShun-TakLeung詹剑锋讲解AcknowledgementPartsofcontentsarefromCSE490h–IntroductiontoDistributedComputing,Winter2008,WashingtonUniversityDistributedFileSystemsTradeoffsinDistributedFileSystemsPerformanceScalabilityReliabili

2、tyAvailabilityTwoCoreApproachesSuperComputer?manycheapcomputers?MotivationGooglewentthecheapcommodityroute…Lotsofdataoncheapmachines!Whynotuseanexistingfilesystem?UniqueproblemsGFSisdesignedforGoogleworkloadsGoogleappsaredesignedforGFSDesignconstraints(1/2)Componen

3、tfailuresarethenormLarge-scalecheapsystemsBugs,humanerrors,failuresofmemory,disk,connectors,networking,andpowersuppliesMonitoring,errordetection,faulttolerance,automaticrecoveryFilesarehugebytraditionalstandardsMulti-GBfilesarecommonButtherearen’tTHATmanyfilesDesigncons

4、traints(2/2)MutationsaretypicallyappendingnewdataRandomwritesarerareOncewritten,filesareonlyread,andtypicallysequentiallyOptimizeforthis!Largeconsecutivereads,smallrandomreadsWanthighsustainedbandwidthlowlatencyisnotthatimportantGoogleisco-designingappsANDfilesystemGF

5、SInterfaceSupportsusualcommandsCreate,delete,open,close,read,writeSnapshotCopiesafileoradirectorytreeRecordAppendAllowsmultipleconcurrentappendstosamefileGFSArchitectureSinglemasterMultiplechunkserversArchitecturalDesign(1/4)AGFSclusterAsinglemasterMultiplechunkser

6、verspermasterAccessedbymultipleclientsRunningoncommodityLinuxmachinesAfiledividedintofixed-sizechunks.Labeledwith64-bituniqueglobalIDsStoredatchunkservers3-wayMirroredacrosschunkserversArchitecturalDesign(2/4)MasterserverMaintainsallmetadataNamespace,accesscontrol,fi

7、le-to-chunkmappings,garbagecollection,chunkmigrationcontrolssystem-wideactivitieschunkleasemanagement,garbagecollectionoforphanedchunks,andchunkmigrationbetweenchunkservers.periodicallycommunicateswitheachchunkserverinHeartBeatmessagestogiveitinstructionsandcollectitsstate

8、.ArchitecturalDesign(3/4)GFSclientsConsultmasterformetadataAccess