在vm下hadoop集群搭建

《在vm下hadoop集群搭建》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、.（原创）VM下搭建hadoop集群一．前期准备1.安装VMware_Workstation_wmb2.安装三台CentOS-6.3-i386-bin-DVD1Master；192.168.66.174Slave1：192.168.66.171Slave2：9:1.168.66.173二．安装步骤：（在安装centos时就把pc的名字改好，免得后面改hostname）1.在每台pc上的/etc/hosts中加入：127.0.0.1localhost192.168.66.174master192.168.66.171slave1192.168.66.173slave22.在每台pc安装j

2、ava：在/etc/profile中加入：exportJAVA_HOME=/usr/local/java/jdk1.6.0_45exportPATH=:$JAVA_HOME/bin:/sbin:/usr/bin:/usr/sbin:/binexportCLASSPATH=.:$JAVA_HOME/lib3.配置SSH无密码登录：...在每台pc上面：ssh-keygen-trsacat~/.ssh/id_rsa.pub>>~/.ssh/authorized_keyschmod700.sshchmod644authorized_keys编辑sshd配置文件/etc/ssh/sshd_co

3、nfig，把#AuthorizedKeysFile.ssh/authorized_keys前面的注释取消掉。在master上面：ssh-copy-id-i~/.ssh/id_rsa.pubroot@slave1ssh-copy-id-i~/.ssh/id_rsa.pubroot@slave2重启servicesshdrestart在slave1上面：ssh-copy-id-i~/.ssh/id_rsa.pubroot@masterssh-copy-id-i~/.ssh/id_rsa.pubroot@slave2重启servicesshdrestart在slave2上面：ssh-copy

4、-id-i~/.ssh/id_rsa.pubroot@slave1ssh-copy-id-i~/.ssh/id_rsa.pubroot@master重启servicesshdrestart再sshslave1;sshslave2;sshmaster验证...**遇到Agentadmittedfailuretosignusingthekey解決方式使用ssh-add指令将私钥加进来（根据个人的密匙命名不同更改id_rsa）#ssh-add~/.ssh/id_rsa**遇到ssh:connecttohostmasterport22:Noroutetohost是由于ip地址问题，检查/etc

5、/hosts4.配置hadoop下载hadoop后scphadoop-1.2.0.tar.gzroot@slave1:/usr/localtarxzvfhadoop-1.2.0.tar.gzmvhadoop-1.2.0/usr/local/hadoop在profile和./bashrc中加入：exportHADOOP_HOME=/usr/local/hadoopexportPATH=$PATH:$HADOOP_HOME/bin配置hadoop-env.sh设置JAVA_HOME配置core-site.xmlfs.default.name

6、alue>hdfs://master:9000true...hadoop.tmp.dir/home/hadoop/hadoop/data/hdfs/tmp配置mapred-site.xmlmapred.job.trackermaster:9001配置hdfs-site.xml

7、ame>dfs.name.dir/home/hadoop/hadoop/data/hdfs/nametruedfs.data.dir/home/hadoop/hadoop/data/hdfs/datatrue