目前本系列文章有:
搭建大數(shù)據(jù)平臺(tái)系列(0)-機(jī)器準(zhǔn)備
搭建大數(shù)據(jù)平臺(tái)系列(1)-Hadoop環(huán)境搭建[hdfs,yarn,mapreduce]
搭建大數(shù)據(jù)平臺(tái)系列(2)-zookeeper環(huán)境搭建
搭建大數(shù)據(jù)平臺(tái)系列(3)-hbase環(huán)境搭建
搭建大數(shù)據(jù)平臺(tái)系列(4)-hive環(huán)境搭建
1.安裝步驟
Zookeeper集群一般配置奇數(shù)個(gè),在本次測(cè)試機(jī)是部署到slave1,slave2,slave3這3臺(tái)機(jī)器上。
1.下載zookeeper-3.4.5-cdh5.5.0.tar.gz到slave1機(jī)器的~/bigdataspacce文件夾下
2.解壓安裝包的命令:
[hadoop@slave1 bigdataspace]$ tar -zxvf zookeeper-3.4.5-cdh5.5.0.tar.gz
3.解壓完成后刪除壓縮包:
[hadoop@slave1 bigdataspace]$ rm zookeeper-3.4.5-cdh5.5.0.tar.gz
4.配置zoo.cfg文件
$ cd /home/hadoop/bigdataspace/zookeeper-3.4.5-cdh5.5.0/conf
$ cp zoo_sample.cfg zoo.cfg
$ vi zoo.cfg
(修改文件中的dataDir配置)
dataDir=/data/zookeeper-3.4.5-cdh5.5.0/var/data
dataLogDir=/data/zookeeper-3.4.5-cdh5.5.0/var/dataLog/
(并在clientPort下面新增如下配置)
server.1=slave1:2888:3888
server.2=slave2:2888:3888
server.3=slave3:2888:3888
5.建立dataDir對(duì)應(yīng)路徑的文件夾,并進(jìn)入該data文件夾下新建一個(gè)文件myid:
$ mkdir -p /data/zookeeper-3.4.5-cdh5.5.0/var/dataLog
$ mkdir -p /data/zookeeper-3.4.5-cdh5.5.0/var/data
$ cd /data/zookeeper-3.4.5-cdh5.5.0/var/data
$ vi myid
(myid文件內(nèi)容為zoo.cfg中配的server號(hào)碼,如server.1則myid文件中只保存1,每臺(tái)機(jī)器都配自己對(duì)應(yīng)的號(hào)碼)
$ cat myid
1
$
6.以上對(duì)zookeeper的配置基本完成,下面使用scp把zookeeper發(fā)到各個(gè)節(jié)點(diǎn):
$ scp -r zookeeper-3.4.5-cdh5.5.0/ hadoop@slave2:~/bigdataspace/
$ scp -r zookeeper-3.4.5-cdh5.5.0/ hadoop@slave3:~/bigdataspace/
7.通過scp把myid傳到各個(gè)節(jié)點(diǎn),并修改其zoo.cfg配置文件對(duì)應(yīng)的server號(hào)碼
(如server.2=slave1:52888:53888則myid文件存入2)
$ scp -r /data/zookeeper-3.4.5-cdh5.5.0/ hadoop@slave2:/data/
$ scp -r /data/zookeeper-3.4.5-cdh5.5.0/ hadoop@slave3:/data/
(然后到到各個(gè)節(jié)點(diǎn)上修改/data/zookeeper-3.4.5-cdh5.5.0/var/data/myid文件),如:
[hadoop@slave2 ~]$ vi /data/zookeeper-3.4.5-cdh5.5.0/var/data/myid
2
[hadoop@slave2 ~]$
8.zookeeper.out以及l(fā)og4j日志文件的設(shè)置
[hadoop@slave1 ~]$ cd /home/hadoop/bigdataspace/zookeeper-3.4.5-cdh5.5.0/conf
[hadoop@slave1 conf]$ vi log4j.properties
# Define some default values that can be overridden by system properties
zookeeper.root.logger=INFO, ROLLINGFILE
……
log4j.appender.ROLLINGFILE=org.apache.log4j.DailyRollingFileAppender
查看zkServer.sh腳本,發(fā)現(xiàn)運(yùn)行時(shí)會(huì)先加載zookeeper-3.4.5-cdh5.5.0/libexec/zkEnv.sh,不存在會(huì)加載zookeeper-3.4.5-cdh5.5.0/bin/zkEnv.sh
[hadoop@slave1 ~]$ cd /home/hadoop/bigdataspace/zookeeper-3.4.5-cdh5.5.0/libexec
[hadoop@slave1 libexec]$ vi zkEnv.sh
if [ "x${ZOO_LOG_DIR}" = "x" ]
then
ZOO_LOG_DIR="/data/zookeeper-3.4.5-cdh5.5.0/logs"
fi
if [ "x${ZOO_LOG4J_PROP}" = "x" ]
then
ZOO_LOG4J_PROP="INFO,ROLLINGFILE"
也可以把zookeeper-3.4.5-cdh5.5.0/bin/zkEnv.sh文件的配置修改成上面一樣.
9.啟動(dòng)zookeeper服務(wù):
[hadoop@slave1 ~]$ cd /home/hadoop/bigdataspace/zookeeper-3.4.5-cdh5.5.0/bin
[hadoop@slave1 bin]$ ./ zkServer.sh start #啟動(dòng)zookeeper(每臺(tái)機(jī)器都要執(zhí)行此命令)
[hadoop@slave1 bin]$ jps #使用jps命令
25906 Jps
20536 QuorumPeerMain #zookeeper的進(jìn)程
19994 JobHistoryServer
19068 NameNode
19422 ResourceManager
19263 SecondaryNameNode
如上,含有QuorumPeerMain表明安裝成功
[hadoop@master bin]$ ./ zkServer.sh stop #停止zookeeper(每臺(tái)機(jī)器都要執(zhí)行此命令)
[hadoop@master bin]$ ./zkServer.sh status #查看角色狀態(tài)命令
JMX enabled by default
Using config: /home/hadoop/bigdataspace/zookeeper-3.4.5-cdh5.5.0/bin/../conf/zoo.cfg
Mode: follower
(Mode: follower/leader,leader這個(gè)角色只有一臺(tái)機(jī)器,是通過zookeeper的選舉算法產(chǎn)生)
如果出現(xiàn)如下錯(cuò)誤,
[hadoop@master bin]$ ./zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/bigdataspace/zookeeper-3.4.5-cdh5.5.0/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
極大可能是因?yàn)榉阑饓Χ丝诒幌拗屏耍梢源蜷_這些被用到的端口
(注意:只啟用一臺(tái)zookeeper也是會(huì) 出現(xiàn)這個(gè)錯(cuò)誤,需要啟動(dòng)2臺(tái)以上的節(jié)點(diǎn))
#進(jìn)入zookeeper客戶端的命令
[hadoop@master bin]$ bin/zkCli.sh -server master:2181