资源描述：
《mapreduce学习日志之我的mapreduce程序学习》由会员上传分享，免费在线阅读，更多相关内容在工程资料-天天文库。
1、MapReduce学习日志之我的MapReduce程序学习将一批电话通信清单，记录了用户A拨打用户B的记录，需要做一个倒排索引，记录拨打给用户B的所有用户A。如原有的txt为:13999999991389999999129999999913722222221880000666137222222218944444441608612613800138000138001380601261638616086首先，我们应该把源文件传到Hdfs上，然后将原始数据进行分割，将被叫作为KEY,主叫作为Value,W拨打相同被叫的主叫
2、号码汇总起来输出到HDFS。程序如下：packagecom・xxs;importjava.io.IOExccption;importorg・apache・hadoop・conf・Configuration;importorg・apache・hadoop・conf.Configured;importorg・apache・hadoop.fs.Path;importorg.apache,hadoop.io.*;importorg.apache,hadoop・mapreduce・*;importorg・apacho.hado
3、op・mapreduce・lib・input・FilcInputFormat;importorg・apache・hadoop・mapreduce.Lib・output・FileOutputFormat;importorg.apache,hadoop・mapreduce-lib・output・TextOutputFormat;importorg・apache・hadoop・uti1・Tool;importorg.apache,hadoop.util.ToolRunncr;//此处为导入的包，一般为固定的。publicc
4、lassMapTcst_2extendsConfiguredimp1ementsTool{enumCounterLINESKIP,}〃出错的行，出错计数器publicstaticclassMapextendsMappcrpublicvoidmap(LongWritablekey,Textvalue,Contextcontext)throwsIOException,InterruptedException{Stringline=valuc・toString();
5、try{String[]lineSplit=line.splitC”)；Stringanum=lineSplit[0];Stringbnum=lincSplit[1];context,write(newText(bnum),newText(anum));}catch(java・lang・ArraylndexOutOfBoundsExceptione){context.getCounter(Counter.LINESKIP).increment(1);return;}publicstaticclassReduceext
6、endsReducer{publicvoidreduce(Textkey,Iterablevalues,Contextcontext)throwslOException,InterruptedException{StringvalueString;Stringout=，，，/;for(Textvalue:values){va1ueString=va1ue・toStringO;out+=valueString+z，";}context.write(key,newTe
7、xt(out));publicintrun(String[]args)throwsExceptionConfigurationconf=getConf();Jobjob=newJob(conf,，/MapTest_2，/);//任务名job・setJarByClass(MapTest_2・class);//指定classFilelnputFormat.addInputPa?/?(job,newPath(args[0]));//输入路径FileOutputFormat.setOutputPa?/?(job,newPat
8、h(args[1]));//输出路径job.setMapperClass(Map・class);job・setReducerClass(Reduce・class);job.setOutputFormatClass(TextOutputFormat.class);job・setOutputKeyClass(Text,class);job・setO
