hadoop大数据应用开发环境配置

《hadoop大数据应用开发环境配置》由会员上传分享，免费在线阅读，更多相关内容在工程资料-天天文库。

1、Hadoop大数据应用开发环境配置一、运行环境计算机名IP角色备注Kingo-mn192.168.0.230主节点(namenode),masters负责总管分布式数裾的调度。Kingo-nis192.168.0.231副节点(JobTrackor),masters分解任务的执行。资源控制器Kingo-dl192.168.0.232数据节点1(Datanodel)Slaves/TaskTracker,负责分布式数据存储以及任务的执行。Kingo~d2192.168.0.238数据节点2(Datan

2、ode2)Kingo~d3192.168.0.239数据节点3(Da.ta.node3)说明修改/etc/sysconfig/network文件可以修改主机名；主机名不能含Ip主机名程序进程192.168.0.230Kingo-mnJdkHadoophbaseNamenodeDFSZKEailoverControllerHamsterresourceManager192.168.0.231Kingo-msJdkHadoophbaseNamenodeSecondaryNamenodeJobTrack

3、erDFSZKEailoverControllerHamsterresourccManager192.168.0.232Kingo-dlJdkHadoopZookeeperhbaseDatanodenodeManagerJournalNodeQuorumPeerMainIlregionServerTaskTracker192.168.0.238Kingo-d2JdkHadoopZooKeeperhbaseDatanodenodeManagerJournalNodeQuorumPeerMainIlr

4、egionServerTaskTracker192.168.0.239Kingo-d3JdkHadoopZookeeperHbaseDatanodenodeManagerJournalNodeQuorumPeerMainIlregionServerTaskTracker二、创建帐号四台机器均创建king。用户组和bigdata帐号，密码统一为Kingol23o#groupaddkingo#useradd-d/home/bigdata-gkingobigdata#passwdbigdata//设賈密

5、码为Kingol23#cd/home#chgrp-Rkingobigdata//设置0录所属组#chmod-Rg=nvxbigdata//设置H录所属组的权限为读、写、执行三、绑定hostname与IP#vi/etc/hosts，增1加内容如下:192.168.0.230kingo-mn192.168.0.231kingo-ms192.168.0.238kingo-rm192.168.0.232kingo-dl192.168.0.239kingo—d2四、设置节点之间的免密码通信1、向各节点复制H

6、osts文件scp/etc/hostskingo-ms:/etc/hostsscp/etc/hostskingo-dl:/etc/hostsscp/etc/hostskingo~d2:/etc/hostsscp/etc/hostskingo-d3:/etc/hosts2、在各节点上生成各自SSH秘钥对#ssh-keygen—t.dsa-P’’-f〜/.ssh/id_dsa#cdV.ssh#catid_dsa.pub〉＞authorized_keys#chmod600authorized_keys3

7、、复制公钥到某一节点#ssh-c()py-id-i~/.ssh/iddsa.pubkingo-mn4、复制密钥到各个节点。在kingo-mn上执行如下命令：#scp/root/,ssh/authorizedkeyskingo-ms:/root/,ssh/;scp/root/,ssh/authorizedkeyskingo-dl:/root/,ssh/;scp/root/,ssh/authorizedkeyskingo-d2:/root/.ssh/;scp/root/.ssh/authorized_

8、keyskingo-d3:/root/.ssh/五、安装JDK把jdk-7u80-linux~x64.rpm安装包复制到/bigdata/setup文件夹下，执行:#rpm-ivhjdk-7u80-linux~x64.rpmJdk安装到默汄目录/usr/java/jdkl.7.0_80下。#vi/etc/profile加入如下配置信息：exportJAVAII0ME=/usr/java/jdk1.7.080exportCLASSPATH^.:$JAVA_HOME/lib/tools