hadoop-数据挖掘研究组

hadoop-数据挖掘研究组

ID:40236510

大小:670.77 KB

页数:38页

时间:2019-07-27

hadoop-数据挖掘研究组_第1页
hadoop-数据挖掘研究组_第2页
hadoop-数据挖掘研究组_第3页
hadoop-数据挖掘研究组_第4页
hadoop-数据挖掘研究组_第5页
资源描述:

《hadoop-数据挖掘研究组》由会员上传分享,免费在线阅读,更多相关内容在行业资料-天天文库

1、HadoopIntroducing InstallationandConfiguration数据挖掘研究组DataMiningGroup@XiamenUniversityADistributeddata-intensiveProgrammingFrameworkHDFSMapReduceHadoopDistributedstorageParallelcomputing数据挖掘研究组DataMiningGroup@XiamenUniversityIntroducingtoHDFSHadoopDistributedFileSystem(HDFS)Anopen-sourceimp

2、lementationofGFShasmanysimilaritieswithdistributedfilesystems.However,comesdifferenceswithit.HDFSishighlyfault-tolerantandisdesignedtobedeployedonlow-costhardware.HDFSprovideshighthroughputaccesstoapplicationdataandissuitableforapplicationsthathavelargedatasets.数据挖掘研究组DataMiningGroup@Xiame

3、nUniversityHowitworks?FeaturesofitAnimportantfeatureofthedesign:dataisnevermovedthroughthenamenode.Instead,alldatatransferoccursdirectlybetweenclientsanddatanodes数据挖掘研究组DataMiningGroup@XiamenUniversityMapReduce?Let’stalkitnexttime………数据挖掘研究组DataMiningGroup@XiamenUniversity“RunningHadoop?” W

4、hatmeansforit?“RunningHadoop”meansrunningasetofdaemons.NameNodeDataNode SecondaryNameNodeJobTrackerTaskTracker数据挖掘研究组DataMiningGroup@XiamenUniversityWhoWorksforwho?HDFSMapReduceHadoopNameNodeSecNDTaskTrackerJobTrackerDataNodeNameNodeHadoopemploysamaster/slavearchitectureforbothdistributeds

5、torageanddistributedcomputation.NameNodeisthemasterofHDFSthatdirectstheslaveDataNodedaemonstoperformthelow-levelI/OtasksNameNodeisthebookkeeperofHDFSkeepstrackofhowyourfilesarebrokendownintofileblockskeepstrackoftheoverallhealthofthedistributedfilesystemDataNodereadingandwritingHDFSblocksforc

6、lientscommunicatewithotherDataNodestoreplicateitsdatablocksforredundancy数据挖掘研究组DataMiningGroup@XiamenUniversityNameNodeandDataNodeSecondaryNameNodeSNNisanassistantdaemonformonitoringthestateoftheclusterHDFSdiffersfromtheNameNodeinthatthisprocessdoesn’treceiveorrecordanyreal-timechangestoHD

7、FScommunicateswiththeNameNodetotakesnapshotsoftheHDFSmetadataRecovery:NameNodefailure????WereconfiguretheclustertousetheSNNastheprimaryNameNodeJobTrackertheliaisonbetweenyourapplicationandHadoopsubmityourcodetoyourcluster,theJobTrackerdeterminestheexecutionplan

当前文档最多预览五页,下载文档查看全文

此文档下载收益归作者所有

当前文档最多预览五页,下载文档查看全文
温馨提示:
1. 部分包含数学公式或PPT动画的文件,查看预览时可能会显示错乱或异常,文件下载后无此问题,请放心下载。
2. 本文档由用户上传,版权归属用户,天天文库负责整理代发布。如果您对本文档版权有争议请及时联系客服。
3. 下载前请仔细阅读文档内容,确认文档内容符合您的需求后进行下载,若出现内容与标题不符可向本站投诉处理。
4. 下载文档时可能由于网络波动等原因无法下载或下载错误,付费完成后未能成功下载的用户请联系客服处理。