资源描述：
《关于使用企业食品生产许可证标志有》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。
1、NetFlowAnalysiswithMapReduceChungnamNationalUniversity{teshi85,yhlee06,lee}@cnu.ac.kr2010.04.24(Sat)WonchulKang,YeonheeLee,YoungseokLee1basedon"AnInternetTrafficAnalysisMethodwithMapReduce",Cloudmanworkshop,April2010IntroductionFlow-basedtrafficmonitoringVolumeofprocesseddataisreduc
2、edPopularflowstatisticstools:CiscoNetFlow[1]Traditionalflow-basedtrafficmonitoringRunonahighperformancecentralserverFlowDataRoutersHighPerformanceServerStorage2MotivationAhugeamountofflowdataLong-termcollectionofflowdataShort-termperiodofflowdataMassiveflowdatafromanomalytrafficdata
3、ofInternetwormandDDoSClusterfilesystemandcloudcomputingplatformGoogle’sprogrammingmodel,MapReduce,bigtable[8]Open-sourcesystem,Hadoop[9]Flowdatainourcampusnetwork(/16prefix)#ofRouters1Day1Month1Year11.2GB13GB156GB56GB65GB780GB1012GB130GB1.5TB200240GB2.6TB30TB3MapReduceMapReduceisapr
4、ogrammingmodelforlargedatasetFirstsuggestedbyGoogleJ.DeanandS.Ghemawat,“MapReduce:SimplifiedDataProcessingonLargeCluster,”OSDI,2004[8]UseronlyspecifyamapandareducefunctionAutomaticallyparallelizedandexecutedonalargecluster4Shuffle&SortMapReduce5Split4Split3Split2Split1MapMapReduceRe
5、duceResult(K1,V)List(K2,V2)(k2,list(v2))List(v3)Map:returnalistcontainingzeroormore(k,v)pairOutputcanbeadifferentkeyfromtheinputOutputcanhavesamekeyReduce:returnanewlistofreducedoutputfrominputHadoopOpen-sourceframeworkforrunningapplicationsonlargeclustersbuiltofcommodityhardwareImp
6、lementationofMapReduceandHDFSMapReduce:computationalparadigmHDFS:distributedfilesystemNodefailuresareautomaticallyhandledbyframeworkHadoopAmazon:EC2,S3serviceFacebook:analyzetheweblogdata6RelatedWorkWidelyusedtoolsforflowstatisticsFlow-tools,flowscanorCoralReef[5]P2P-baseddistribute
7、danalysisofflowdataDIPStorage:eachstoragetankassociatedwitharule[11]MapReducesoftwareSnortloganalysis:NCHCcloudcomputingresearchgroup[16]7ContributionAflowanalysismethodwithMapReduceProcessflowdatainacloudcomputingplatform,hadoopImplementationofflowanalysisprogramswithHadoopDecrease
8、flowcomputationtime
