1. ZooKeeper應用實踐

qdice007@gmail.com 2017-12-21 21:17

寫在前面的廢話

大數據之所以日漸崛起,是因為海量數據的處理,必須依靠集群的威力,而集群又帶來了一個新問題,那就是成百上千的這么多機器,如何協調一致,如何滿足既可用又一致的要求? ZooKeeper就是其中一個答案.
推薦一本書可以看看,阿里的人寫的


s28377338.jpg

開始

ZooKeeper環(huán)境搭建前提條件

  • 已經按照之前的文章裝好了三臺虛擬機,并已配置好CentOS 7,保證三臺機器之間可以網絡互通,可以ssh免密登錄
  • JDK已安裝,并配好環(huán)境變量
  • 在ZooKeeper官網下載了軟件包(zookeeper-3.4.10.tar.gz),并放在 /home/libing/softwares 目錄下

ZooKeeper環(huán)境搭建步驟(都以提前建好的普通用戶libing來操作)

1. 解壓軟件包至用戶libing的家目錄下
[libing@bd-1 ~]$ tar -xzf softwares/zookeeper-3.4.10.tar.gz -C /home/libing/
[libing@bd-1 ~]$ ll
總用量 8
drwxr-xr-x.  8 libing libing  255 9月  14 17:27 jdk1.8.0_152
drwxrwxr-x.  2 libing libing  148 12月 20 20:09 softwares
drwxr-xr-x  10 libing libing 4096 3月  23 2017 zookeeper-3.4.10
2. 編輯 /etc/profile 文件,給系統增加環(huán)境變量ZOOKEEPER_HOME,以及修改PATH環(huán)境變量
[libing@bd-1 ~]$ sudo vi /etc/profile
[sudo] libing 的密碼：

在文件內容的最后添加以下內容

# Added for ZooKeeper
ZOOKEEPER_HOME=/home/libing/zookeeper-3.4.10
export ZOOKEEPER_HOME
PATH=${ZOOKEEPER_HOME}/bin:$PATH
export PATH

保存退出vi

3. 使之馬上生效,以及進行驗證
[libing@bd-1 ~]$ source /etc/profile
[libing@bd-1 ~]$ env | grep ZOOK
ZOOKEEPER_HOME=/home/libing/zookeeper-3.4.10
[libing@bd-1 ~]$ ls zookeeper-3.4.10/bin
README.txt  zkCleanup.sh  zkCli.cmd  zkCli.sh  zkEnv.cmd  zkEnv.sh  zkServer.cmd  zkServer.sh
[libing@bd-1 ~]$ which zkServer.sh
~/zookeeper-3.4.10/bin/zkServer.sh

簡單解釋一下上面的命令

  • source是執(zhí)行腳本的命令,將讀取/etc/profile的內容,并逐一執(zhí)行
  • env是輸出當前用戶所有的環(huán)境變量
  • grep是搜索過濾的命令,env | grep ZOOK 表示把env的輸出,作為grep命令的輸入,并在其中查找包含ZOOK字符的行,打印出來
  • which命令可以在當前的PATH環(huán)境變量包含的所有路徑中,尋找某個文件,如果找到了,就把這個文件的路徑加文件名打印出來
4. 進入ZooKeeper的conf目錄,復制并編輯zoo.cfg配置文件
[libing@bd-1 ~]$ cd zookeeper-3.4.10/conf
[libing@bd-1 conf]$ ls
configuration.xsl  log4j.properties  zoo_sample.cfg
[libing@bd-1 conf]$ cp zoo_sample.cfg zoo.cfg
[libing@bd-1 conf]$ vi zoo.cfg

在文件的最后增加如下內容

# store snapshot files
dataDir=/home/libing/zookeeper-3.4.10/data
# store transaction log files
dataLogDir=/home/libing/zookeeper-3.4.10/log

server.1=bd-1:2888:3888
server.2=bd-2:2888:3888
server.3=bd-3:2888:3888

保存后退出vi.
注意上面內容中的路徑,以及bd-1等主機名,需要根據你自己虛擬機的情況進行改動.

5. 在ZooKeeper軟件目錄下創(chuàng)建data和log目錄,并修改權限屬性
[libing@bd-1 conf]$ cd ~/zookeeper-3.4.10
[libing@bd-1 zookeeper-3.4.10]$ mkdir data log
[libing@bd-1 zookeeper-3.4.10]$ chmod 755 data log
6. 創(chuàng)建myid文件并填入內容(以bd-1虛擬機上操作為例)
[libing@bd-1 zookeeper-3.4.10]$ echo 1 > data/myid
[libing@bd-1 zookeeper-3.4.10]$ cat data/myid 
1
7. 啟動ZooKeeper服務,并查看狀態(tài)

這是bd-1虛擬機的

[libing@bd-1 zookeeper-3.4.10]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/libing/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[libing@bd-1 zookeeper-3.4.10]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/libing/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

這是bd-2虛擬機的

[libing@bd-2 zookeeper-3.4.10]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/libing/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[libing@bd-2 zookeeper-3.4.10]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/libing/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: leader

這是bd-3虛擬機的

[libing@bd-3 zookeeper-3.4.10]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/libing/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[libing@bd-3 zookeeper-3.4.10]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/libing/zookeeper-3.4.10/bin/../conf/zoo.cfg
Mode: follower

可以看到bd-2虛擬機的ZooKeeper是leader,其他兩臺都是follower.

8. 停止ZooKeeper

在每臺虛擬機上執(zhí)行(以bd-1虛擬機為例)

[libing@bd-1 zookeeper-3.4.10]$ zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /home/libing/zookeeper-3.4.10/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
9. 使用一個腳本實現所有虛擬機的ZooKeeper啟動/關閉

創(chuàng)建ZooKeeper集群啟動腳本(存放于bd-1虛擬機并執(zhí)行)

[libing@bd-1 ~]$ vi zkStartAll.sh

腳本內容如下

#!/bin/bash
echo "Start ZooKeeper service ..."
for i in 1 2 3
do
    ssh bd-$i "source /etc/profile ; zkServer.sh start"
done
echo "ZooKeeper started!"

執(zhí)行

[libing@bd-1 ~]$ chmod u+x zkStartAll.sh 
[libing@bd-1 ~]$ ./zkStartAll.sh

創(chuàng)建ZooKeeper集群關閉腳本(存放于bd-1虛擬機并執(zhí)行)

[libing@bd-1 ~]$ vi zkStopAll.sh

腳本內容如下

#!/bin/bash
echo "Stop ZooKeeper service ..."
for i in 1 2 3
do
    ssh bd-$i "source /etc/profile ; zkServer.sh stop"
done
echo "ZooKeeper stopped!"

執(zhí)行

[libing@bd-1 ~]$ chmod u+x zkStopAll.sh 
[libing@bd-1 ~]$ ./zkStopAll.sh

可以分別在每臺虛擬機上使用 zkServer.sh status驗證腳本的效果.

注意點: 三臺虛擬機之間的ssh免密登錄一定要配置好,包括每個虛擬機ssh登錄本身自己也要免密

ZooKeeper運維操作實驗

1. 進入客戶端zkCli.sh

不使用任何參數的話,就是連接本地的ZooKeeper.

[libing@bd-1 ~]$ zkCli.sh
Connecting to localhost:2181
2017-12-25 10:31:20,623 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2017-12-25 10:31:20,628 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=bd-1
2017-12-25 10:31:20,628 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_152
2017-12-25 10:31:20,633 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2017-12-25 10:31:20,634 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/home/libing/jdk1.8.0_152/jre
2017-12-25 10:31:20,635 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/home/libing/zookeeper-3.4.10/bin/../build/classes:/home/libing/zookeeper-3.4.10/bin/../build/lib/*.jar:/home/libing/zookeeper-3.4.10/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/libing/zookeeper-3.4.10/bin/../lib/slf4j-api-1.6.1.jar:/home/libing/zookeeper-3.4.10/bin/../lib/netty-3.10.5.Final.jar:/home/libing/zookeeper-3.4.10/bin/../lib/log4j-1.2.16.jar:/home/libing/zookeeper-3.4.10/bin/../lib/jline-0.9.94.jar:/home/libing/zookeeper-3.4.10/bin/../zookeeper-3.4.10.jar:/home/libing/zookeeper-3.4.10/bin/../src/java/lib/*.jar:/home/libing/zookeeper-3.4.10/bin/../conf:
2017-12-25 10:31:20,636 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-12-25 10:31:20,636 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2017-12-25 10:31:20,636 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2017-12-25 10:31:20,637 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2017-12-25 10:31:20,637 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2017-12-25 10:31:20,637 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=3.10.0-693.11.1.el7.x86_64
2017-12-25 10:31:20,638 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=libing
2017-12-25 10:31:20,638 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/libing
2017-12-25 10:31:20,638 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/home/libing
2017-12-25 10:31:20,642 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@7aec35a
Welcome to ZooKeeper!
2017-12-25 10:31:20,692 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2017-12-25 10:31:20,836 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@876] - Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2017-12-25 10:31:20,855 [myid:] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x1608b7c924d0001, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0]

退出zkCli

[zk: localhost:2181(CONNECTED) 0] quit
Quitting...
2017-12-25 10:31:26,565 [myid:] - INFO  [main:ZooKeeper@684] - Session: 0x1608b7c924d0001 closed
2017-12-25 10:31:26,569 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@519] - EventThread shut down for session: 0x1608b7c924d0001
[libing@bd-1 ~]$

在bd-1虛擬機上連接bd-2上的ZooKeeper

[libing@bd-1 ~]$ zkCli.sh -server bd-2:2181
Connecting to bd-2:2181
2017-12-25 10:31:57,966 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2017-12-25 10:31:57,970 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=bd-1
2017-12-25 10:31:57,970 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_152
2017-12-25 10:31:57,972 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2017-12-25 10:31:57,972 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/home/libing/jdk1.8.0_152/jre
2017-12-25 10:31:57,973 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/home/libing/zookeeper-3.4.10/bin/../build/classes:/home/libing/zookeeper-3.4.10/bin/../build/lib/*.jar:/home/libing/zookeeper-3.4.10/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/libing/zookeeper-3.4.10/bin/../lib/slf4j-api-1.6.1.jar:/home/libing/zookeeper-3.4.10/bin/../lib/netty-3.10.5.Final.jar:/home/libing/zookeeper-3.4.10/bin/../lib/log4j-1.2.16.jar:/home/libing/zookeeper-3.4.10/bin/../lib/jline-0.9.94.jar:/home/libing/zookeeper-3.4.10/bin/../zookeeper-3.4.10.jar:/home/libing/zookeeper-3.4.10/bin/../src/java/lib/*.jar:/home/libing/zookeeper-3.4.10/bin/../conf:
2017-12-25 10:31:57,973 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2017-12-25 10:31:57,973 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2017-12-25 10:31:57,973 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2017-12-25 10:31:57,973 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2017-12-25 10:31:57,973 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2017-12-25 10:31:57,973 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=3.10.0-693.11.1.el7.x86_64
2017-12-25 10:31:57,974 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=libing
2017-12-25 10:31:57,974 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/home/libing
2017-12-25 10:31:57,974 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/home/libing
2017-12-25 10:31:57,976 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=bd-2:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@7aec35a
Welcome to ZooKeeper!
2017-12-25 10:31:58,022 [myid:] - INFO  [main-SendThread(bd-2:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server bd-2/192.168.206.133:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2017-12-25 10:31:58,152 [myid:] - INFO  [main-SendThread(bd-2:2181):ClientCnxn$SendThread@876] - Socket connection established to bd-2/192.168.206.133:2181, initiating session
2017-12-25 10:31:58,173 [myid:] - INFO  [main-SendThread(bd-2:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server bd-2/192.168.206.133:2181, sessionid = 0x2608b7c91640001, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: bd-2:2181(CONNECTED) 0]

查看help幫助信息

[zk: bd-2:2181(CONNECTED) 0] help
ZooKeeper -server host:port cmd args
    stat path [watch]
    set path data [version]
    ls path [watch]
    delquota [-n|-b] path
    ls2 path [watch]
    setAcl path acl
    setquota -n|-b val path
    history 
    redo cmdno
    printwatches on|off
    delete path [version]
    sync path
    listquota path
    rmr path
    get path [watch]
    create [-s] [-e] path data acl
    addauth scheme auth
    quit 
    getAcl path
    close 
    connect host:port

總結:
三臺機器的配置和操作幾乎完全一樣,區(qū)別就在于myid文件內容不一樣,bd-1虛擬機的myid文件內容是1,bd-2虛擬機的myid文件內容是2,bd-3虛擬機的myid文件內容是3,其他都完全一樣.

其他的運維命令什么的,大家就把ZooKeeper集群服務起來后,照著文檔慢慢嘗試吧.

結束

最后編輯于 ：
?著作權歸作者所有,轉載或內容合作請聯系作者
【社區(qū)內容提示】社區(qū)部分內容疑似由AI輔助生成，瀏覽時請結合常識與多方信息審慎甄別。
平臺聲明：文章內容（如有圖片或視頻亦包括在內）由作者上傳并發(fā)布，文章內容僅代表作者本人觀點，簡書系信息發(fā)布平臺，僅提供信息存儲服務。

相關閱讀更多精彩內容

友情鏈接更多精彩內容