hadoop入门-分布式hadoop安装

《hadoop入门-分布式hadoop安装》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、一:准备4台集群的机器.192.168.1.31192.168.1.32192.168.1.33192.168.1.34root账号修改4台机器的/etc/hosts文件,添加如下:192.168.1.31hadoop1#namenode+job192.168.1.32hadoop2#datanode192.168.1.33hadoop3#datanode192.168.1.34hadoop4#datanode在每台服务器上创建4个一样的hadoop用户:1.useraddhadoop#加用户2.passwdhad

2、oop#修改密码二:创建SSH免密码登录hadoop账号在namenode节点上,查看sshlocalhost是否是免密码的,如果不是,执行下面命令:a)生成公钥密钥:ssh-keygen-tdsa-P''-f~/.ssh/id_dsa执行完成之后,会在~/.ssh/下生成id_dsa.pub和id_dsab)把id_dsa.pub文件内容复制到authorized_keys文件里:cat~/.ssh/id_dsa.pub>>~/.ssh/authorized_keys执行完成后会在~/.ssh/下生成author

3、ized_keys删除id_dsa.pub文件:rm–rfid_dsa.pubc)将authorized_keys复制到各个datanode节点上:在各个子节点上创建.ssh文件夹:mkdir/home.hadoop/.ssh将证书文件拷贝到各台机器上(会提示输入密码):scpauthorized_keyshadoop2:/home/hadoop/.ssh/scpauthorized_keyshadoop3:/home/hadoop/.ssh/scpauthorized_keyshadoop4:/home/hado

4、op/.ssh/d)将各个datanode节点下的.ssh权限改成700,authorized_keys改成600改完后应该就差不多可以了,在hadoop1机器上依次执行:sshhadoop2,sshhadoop3,sshhadoop4,如果都不用密码的话,表是成功.三:安装jdk用root账号安装,可以下载个bin文件,我的是jdk-6u23-linux-i586-rpm.bin,直接./jdk-6u23-linux-i586-rpm.bin安装,默认路径在/usr/java/jdk1.6.0_23/下,4台机器

5、都一样,注意点:安装的路径一定要一样.四:安装配置Hadoop用hadoop账号,下载hadoop-0.20.205.0.tar.gz,放在/home/hadoop/下,修改hadoop-0.20.205.0.tar.gz权限:chmod700hadoop-0.20.205.0.tar.gz解压:tarzxvfhadoop-0.20.205.0.tar.gz,完成后会生成hadoop-0.20.205.0目录,进入hadoop-0.20.205.0/conf/下按照如下方式修改下面文件:1.Namenode配置:c

6、ore-site.xml:hadoop.tmp.dir/tmp/hadooptmpAbaseforothertemporarydir

7、ectories.fs.default.namehdfs://hadoop1:9000Thenameofthedefaultfilesystem.AURIwhoseschemeandauthoritydeterminetheFileSystemimplementation.Theuri'sschemedeterminestheconfigproperty(fs.SC

8、HEME.impl)namingtheFileSystemimplementationclass.Theuri'sauthorityisusedtodeterminethehost,port,etc.forafilesystem.2.Tracker配制:Mapred-site.