Redis 4 集群搭建

一、概述

Redis3.0版本之后支持Cluster,目前redis cluster支持的特性有:

節(jié)點(diǎn)自動(dòng)發(fā)現(xiàn),slave->master 選舉,集群容錯(cuò),Hot resharding:在線(xiàn)分片
進(jìn)群管理:cluster xxx,基于配置(nodes-port.conf)的集群管理,ASK 轉(zhuǎn)向/MOVED 轉(zhuǎn)向機(jī)制.

二、redis cluster安裝

1、下載和解包
cd /usr/local/
wget http://download.redis.io/releases/redis-4.0.1.tar.gz
 tar -xvzf redis-4.0.1.tar.gz

2、 編譯安裝

 cd redis-4.0.1
 make && make install

3、創(chuàng)建redis節(jié)點(diǎn)

選擇2臺(tái)服務(wù)器,分別為:192.168.59.131,192.168..59.130.每分服務(wù)器有3個(gè)節(jié)點(diǎn)。

先在192.168.59.131創(chuàng)建3個(gè)節(jié)點(diǎn):

  cd /usr/local/
//創(chuàng)建集群目錄
  mkdir redis_cluster  
//分別代表三個(gè)節(jié)點(diǎn)    其對(duì)應(yīng)端口 7000 7001 7002
  mkdir 7000 7001 7002  
 //以創(chuàng)建7000節(jié)點(diǎn)為例,拷貝到7000目錄
 cp /usr/local/redis-4.0.1/redis.conf  ./redis_cluster/7000/   
 //拷貝到7001目錄
 cp /usr/local/redis-4.0.1/redis.conf  ./redis_cluster/7001/   
 //拷貝到7002目錄
 cp /usr/local/redis-4.0.1/redis.conf  ./redis_cluster/7002/ 

分別對(duì)7001,7002、7003文件夾中的3個(gè)文件修改對(duì)應(yīng)的配置

daemonize    yes                             //redis后臺(tái)運(yùn)行
pidfile  /var/run/redis_7000.pid         //pidfile文件對(duì)應(yīng)7000,7002,7003
bind 192.168.59.131                         //綁定ip地址
port  7000                                          //端口7000,7002,7003
cluster-enabled  yes                           //開(kāi)啟集群  把注釋#去掉
cluster-config-file  nodes_7000.conf   //集群的配置  配置文件首次啟動(dòng)自動(dòng)生成 7000,7001,7002
cluster-node-timeout  5000                //請(qǐng)求超時(shí)  設(shè)置5秒夠了
appendonly  yes                          //aof日志開(kāi)啟  有需要就開(kāi)啟,它會(huì)每次寫(xiě)操作都記錄一條日志

在192.168.59.130創(chuàng)建3個(gè)節(jié)點(diǎn):對(duì)應(yīng)的端口改為7003,7004,7005.配置對(duì)應(yīng)的改一下就可以了
4、兩臺(tái)機(jī)啟動(dòng)各節(jié)點(diǎn)

第一臺(tái)機(jī)器上執(zhí)行
redis-server redis_cluster/7000/redis.conf
redis-server redis_cluster/7001/redis.conf
redis-server redis_cluster/7002/redis.conf
 
另外一臺(tái)機(jī)器上執(zhí)行
redis-server redis_cluster/7003/redis.conf
redis-server redis_cluster/7004/redis.conf
redis-server redis_cluster/7005/redis.conf 

5、查看服務(wù)

  ps -ef | grep redis   #查看是否啟動(dòng)成功

 netstat -tnlp | grep redis #可以看到redis監(jiān)聽(tīng)端口

三、創(chuàng)建集群

前面已經(jīng)準(zhǔn)備好了搭建集群的redis節(jié)點(diǎn),接下來(lái)我們要把這些節(jié)點(diǎn)都串連起來(lái)搭建集群。
官方提供了一個(gè)工具:redis-trib.rb(/usr/local/redis-4.0.1/src/redis-trib.rb) 看后綴就知道它是用ruby寫(xiě)的一個(gè)程序,所以我們還得安裝ruby.

yum -y install ruby ruby-devel rubygems rpm-build

再用 gem 這個(gè)命令來(lái)安裝 redis接口 gem是ruby的一個(gè)工具包.

gem install redis
接下來(lái)運(yùn)行一下redis-trib.rb

[root@localhost local]# /usr/local/redis-4.0.1/src/redis-trib.rb
Usage: redis-trib <command> <options> <arguments ...>

  create          host1:port1 ... hostN:portN
                  --replicas <arg>
  check           host:port
  info            host:port
  fix             host:port
                  --timeout <arg>
  reshard         host:port
                  --from <arg>
                  --to <arg>
                  --slots <arg>
                  --yes
                  --timeout <arg>
                  --pipeline <arg>
  rebalance       host:port
                  --weight <arg>
                  --auto-weights
                  --use-empty-masters
                  --timeout <arg>
                  --simulate
                  --pipeline <arg>
                  --threshold <arg>
  add-node        new_host:new_port existing_host:existing_port
                  --slave
                  --master-id <arg>
  del-node        host:port node_id
  set-timeout     host:port milliseconds
  call            host:port command arg arg .. arg
  import          host:port
                  --from <arg>
                  --copy
                  --replace
  help            (show this help)

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

確認(rèn)所有的節(jié)點(diǎn)都啟動(dòng),接下來(lái)使用參數(shù)create 創(chuàng)建 (在192.168.59.131中來(lái)創(chuàng)建)

/usr/local/redis-4.0.1/src/redis-trib.rb create --replicas 1 192.168.59.131:7000 192.168.59.131:7001 192.168.59.131:7003 192.168.59.130:7003 192.168.59.130:7004 192.168.59.130:7005

--replicas 1 表示 自動(dòng)為每一個(gè)master節(jié)點(diǎn)分配一個(gè)slave節(jié)點(diǎn) 上面有6個(gè)節(jié)點(diǎn),程序會(huì)按照一定規(guī)則生成 3個(gè)master(主)3個(gè)slave(從)

防火墻一定要開(kāi)放監(jiān)聽(tīng)的端口,否則會(huì)創(chuàng)建失敗。
運(yùn)行中,提示Can I set the above configuration? (type 'yes' to accept): yes //輸入yes

[root@localhost local]#  /usr/local/redis-4.0.1/src/redis-trib.rb  create  --replicas  1  192.168.59.131:7000 192.168.59.131:7001  192.168.59.131:7003 192.168.59.130:7003  192.168.59.130:7004  192.168.59.130:7005
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.59.131:7000
192.168.59.130:7003
192.168.59.131:7001
Adding replica 192.168.59.130:7004 to 192.168.59.131:7000
Adding replica 192.168.59.131:7002 to 192.168.59.130:7003
Adding replica 192.168.59.130:7005 to 192.168.59.131:7001
M: 048937fd4ac542c135e7527f681be12348e8c8d0 192.168.59.131:7000
   slots:0-5460 (5461 slots) master
M: 2039bc32bf55d21d74296cdc91a316022a7afc4b 192.168.59.131:7001
   slots:10923-16383 (5461 slots) master
S: 76514c28d0a40f9751ccaf117d1f8f1d0878475a 192.168.59.131:7002
   replicates a58a304964ff10d10f4323341f7084c9f60f5124
M: a58a304964ff10d10f4323341f7084c9f60f5124 192.168.59.130:7003
   slots:5461-10922 (5462 slots) master
S: 7512d76b416d09619d3b639a94945a676d7b8826 192.168.59.130:7004
   replicates 048937fd4ac542c135e7527f681be12348e8c8d0
S: 7824b9721cb21de60c8d132d668fd3ac5c4eae49 192.168.59.130:7005
   replicates 2039bc32bf55d21d74296cdc91a316022a7afc4b
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 192.168.59.131:7000)
M: 048937fd4ac542c135e7527f681be12348e8c8d0 192.168.59.131:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 76514c28d0a40f9751ccaf117d1f8f1d0878475a 192.168.59.131:7002
   slots: (0 slots) slave
   replicates a58a304964ff10d10f4323341f7084c9f60f5124
M: a58a304964ff10d10f4323341f7084c9f60f5124 192.168.59.130:7003
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 7512d76b416d09619d3b639a94945a676d7b8826 192.168.59.130:7004
   slots: (0 slots) slave
   replicates 048937fd4ac542c135e7527f681be12348e8c8d0
M: 2039bc32bf55d21d74296cdc91a316022a7afc4b 192.168.59.131:7001
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 7824b9721cb21de60c8d132d668fd3ac5c4eae49 192.168.59.130:7005
   slots: (0 slots) slave
   replicates 2039bc32bf55d21d74296cdc91a316022a7afc4b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

到這里集群已經(jīng)初步搭建好了。

四、測(cè)試

  1. get 和 set數(shù)據(jù)
    [root@localhost local]# redis-cli -h 192.168.59.131 -c -p 7002
    192.168.59.131:7002> keys *
    (empty list or set)

[root@localhost local]# redis-cli -h 192.168.59.130 -c -p 7005
192.168.59.130:7005> set hello world
-> Redirected to slot [866] located at 192.168.59.131:7000
OK
192.168.59.131:7000>

[root@localhost local]# redis-cli -h 192.168.59.131 -c -p 7002
192.168.59.131:7002> get hello
-> Redirected to slot [866] located at 192.168.59.131:7000
"world"
192.168.59.131:7000>

  1. 模擬故障切換

A. 先把192.168.59.131 其中一個(gè)Master服務(wù)Down掉,(192.168.59.131有2個(gè)Master, 1個(gè)Slave) ,
測(cè)試一下,依然沒(méi)有問(wèn)題,集群依然能繼續(xù)工作。

原因:  redis集群  通過(guò)選舉方式進(jìn)行容錯(cuò),保證一個(gè)服務(wù)掛了還能跑,這個(gè)選舉是全部集群超過(guò)半數(shù)以上的Master發(fā)現(xiàn)其他Master掛了后,會(huì)將其他對(duì)應(yīng)的Slave節(jié)點(diǎn)升級(jí)成Master.

B. 要是掛的是192.168.59.131怎么辦? 結(jié)果是,(error) CLUSTERDOWN The cluster is down 沒(méi)辦法,超過(guò)半數(shù)掛了那整個(gè)集群就無(wú)法工作了。
五 、常用命令

  1. 查看集群情況
    redis-trib.rb check ip:port #檢查集群狀態(tài)
    redis-cli -c -h ip -p port #使用-c進(jìn)入集群命令模式
    redis-trib.rb rebalance ip:port --auto-weights #重新分配權(quán)重
[root@localhost script]# /usr/local/redis-4.0.1/src/redis-trib.rb  info  192.168.59.131:7000
192.168.59.130:7003 (a58a3049...) -> 0 keys | 5462 slots | 1 slaves.
192.168.59.130:7005 (7824b972...) -> 0 keys | 5461 slots | 1 slaves.
192.168.59.130:7004 (7512d76b...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
  1. 增加、刪除集群節(jié)點(diǎn)
    redis-trib.rb add-node ip:port(新增節(jié)點(diǎn)) ip:port(現(xiàn)有效節(jié)點(diǎn))
    redis-trib.rb del-node ip:port id(目標(biāo)節(jié)點(diǎn)的id) #刪除master節(jié)點(diǎn)之前首先要使用reshard移除master的全部slot

  2. 重新劃分slot
    redis-trib.rb reshard ip:port

[root@localhost script]#  /usr/local/redis-4.0.1/src/redis-trib.rb  reshard  192.168.59.131:7001
>>> Performing Cluster Check (using node 192.168.59.131:7001)
S: 2039bc32bf55d21d74296cdc91a316022a7afc4b 192.168.59.131:7001
   slots: (0 slots) slave
   replicates 7824b9721cb21de60c8d132d668fd3ac5c4eae49
M: 7824b9721cb21de60c8d132d668fd3ac5c4eae49 192.168.59.130:7005
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 048937fd4ac542c135e7527f681be12348e8c8d0 192.168.59.131:7000
   slots: (0 slots) slave
   replicates 7512d76b416d09619d3b639a94945a676d7b8826
M: 7512d76b416d09619d3b639a94945a676d7b8826 192.168.59.130:7004
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 76514c28d0a40f9751ccaf117d1f8f1d0878475a 192.168.59.131:7002
   slots: (0 slots) slave
   replicates a58a304964ff10d10f4323341f7084c9f60f5124
M: a58a304964ff10d10f4323341f7084c9f60f5124 192.168.59.130:7003
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 5461
What is the receiving node ID? 7824b9721cb21de60c8d132d668fd3ac5c4eae49
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1:7512d76b416d09619d3b639a94945a676d7b8826
Source node #2:done

Ready to move 5461 slots.
  Source nodes:
    M: 7512d76b416d09619d3b639a94945a676d7b8826 192.168.59.130:7004
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
  Destination node:
    M: 7824b9721cb21de60c8d132d668fd3ac5c4eae49 192.168.59.130:7005
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
  Resharding plan:
    Moving slot 0 from 7512d76b416d09619d3b639a94945a676d7b8826
    ........................
    .........................
    Moving slot 5460 from 7512d76b416d09619d3b639a94945a676d7b8826
Do you want to proceed with the proposed reshard plan (yes/no)? yes
Moving slot 0 from 192.168.59.130:7004 to 192.168.59.130:7005:
........................
.........................
Moving slot 5460 from 192.168.59.130:7004 to 192.168.59.130:7005:
[root@localhost script]#
  1. 將master轉(zhuǎn)換為salve
    cluster replicate master-id #轉(zhuǎn)換前6380端必須沒(méi)有slots
[root@localhost script]#  redis-cli -h 192.168.59.130 -c -p 7004
192.168.59.130:7004> cluster replicate 7824b9721cb21de60c8d132d668fd3ac5c4eae49
OK
192.168.59.130:7004> quit

六、深入了解

  1. redis-cluster架構(gòu)圖



      (1)所有的redis節(jié)點(diǎn)彼此互聯(lián)(PING-PONG機(jī)制),內(nèi)部使用二進(jìn)制協(xié)議優(yōu)化傳輸速度和帶寬.
      (2)節(jié)點(diǎn)的fail是通過(guò)集群中超過(guò)半數(shù)的節(jié)點(diǎn)檢測(cè)失效時(shí)才生效.
      (3)客戶(hù)端與redis節(jié)點(diǎn)直連,不需要中間proxy層.客戶(hù)端不需要連接集群所有節(jié)點(diǎn),連接集群中任何一個(gè)可用節(jié)點(diǎn)即可
      (4)redis-cluster把所有的物理節(jié)點(diǎn)映射到[0-16383]slot上,cluster 負(fù)責(zé)維護(hù)node<->slot<->value

    1. redis-cluster選舉容錯(cuò)



        (1)領(lǐng)著選舉過(guò)程是集群中所有master參與,如果半數(shù)以上master節(jié)點(diǎn)與master節(jié)點(diǎn)通信超過(guò)(cluster-node-timeout),認(rèn)為當(dāng)前master節(jié)點(diǎn)掛掉.
        (2):什么時(shí)候整個(gè)集群不可用(cluster_state:fail),當(dāng)集群不可用時(shí),所有對(duì)集群的操作做都不可用,收到((error) CLUSTERDOWN The cluster is down)錯(cuò)誤
        a:如果集群任意master掛掉,且當(dāng)前master沒(méi)有slave.集群進(jìn)入fail狀態(tài),也可以理解成進(jìn)群的slot映射[0-16383]不完成時(shí)進(jìn)入fail狀態(tài).
        b:如果進(jìn)群超過(guò)半數(shù)以上master掛掉,無(wú)論是否有slave集群進(jìn)入fail狀態(tài).

參考:http://www.cnblogs.com/yuanermen/p/5717885.html

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書(shū)系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容