kubernetes Stolon部署高可用Postgresql

....前言

本文選用Stolon的方式搭建Postgresql高可用方案，主要為Harbor提供高可用數(shù)據(jù)庫，Harbor搭建可查看kubernetes搭建Harbor無坑及Harbor倉庫同步，之后會提供redis高可用及Harbor高可用方案搭建

方案比較

幾種postgresql高可用方案簡單比較：

引用https://studygolang.com/articles/19002?fr=sidebar

  • 首先repmgr這種方案的算法有明顯缺陷，非主流分布式算法，直接pass；
  • Stolon和Patroni相對于Crunchy更加Cloud Native， 后者是基于pgPool實現(xiàn)。
  • Crunchy和Patroni相對于Stolon有更多的使用者，并且提供了Operator對于以后的管理和擴(kuò)容

根據(jù)上面簡單的比較，最終選擇的stolon，作者選擇的是Patroni，感覺實際區(qū)別并不大。

一、Stolon概述：

Stolon（https://github.com/sorintlab/stolon
）
是由3個部分組成的：

  • keeper：他負(fù)責(zé)管理PostgreSQL的實例匯聚到由sentinel(s)提供的clusterview。
  • sentinel：it負(fù)責(zé)發(fā)現(xiàn)并且監(jiān)控keeper，并且計算最理想的clusterview。
  • proxy：客戶端的接入點。它強(qiáng)制連接到右邊PostgreSQL的master并且強(qiáng)制關(guān)閉連接到由非選舉產(chǎn)生的master。
    Stolon 用etcd或者consul作為主要的集群狀態(tài)存儲。

二、Installation

git clone https://github.com/sorintlab/stolon.git
cd XXX/stolon/examples/kubernetes

如圖所示

stolon

如有興趣可查看官網(wǎng)搭建：https://github.com/sorintlab/stolon/blob/master/examples/kubernetes/README.md
如下為yaml中注意修改的地方

  • stolon-keeper.yaml 中設(shè)置Postgresql用戶名
- name: STKEEPER_PG_SU_USERNAME
            value: "postgres"
  • stolon-keeper.yaml 中設(shè)置stolon掛載卷
 volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
        - "ReadWriteOnce"
      resources:
        requests:
          storage: "512Mi"
      storageClassName: nfs
  • secret.yaml中設(shè)置用戶密碼
apiVersion: v1
kind: Secret
metadata:
    name: stolon
type: Opaque
data:
    password: cGFzc3dvcmQx

如下是作者整理的完整的stolon的編排文件，可直接修改使用

# This is an example and generic rbac role definition for stolon. It could be
# fine tuned and split per component.
# The required permission per component should be:
# keeper/proxy/sentinel: update their own pod annotations
# sentinel/stolonctl: get, create, update configmaps
# sentinel/stolonctl: list components pods
# sentinel/stolonctl: get components pods annotations

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: stolon
  namespace: default
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - configmaps
  - events
  verbs:
  - "*"
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: stolon
  namespace: default
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: stolon
subjects:
- kind: ServiceAccount
  name: default
  namespace: default
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: stolon-sentinel
spec:
  replicas: 2
  template:
    metadata:
      labels:
        component: stolon-sentinel
        stolon-cluster: kube-stolon
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
    spec:
      containers:
      - name: stolon-sentinel
        image: sorintlab/stolon:master-pg10
        command:
          - "/bin/bash"
          - "-ec"
          - |
            exec gosu stolon stolon-sentinel
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: STSENTINEL_CLUSTER_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.labels['stolon-cluster']
          - name: STSENTINEL_STORE_BACKEND
            value: "kubernetes"
          - name: STSENTINEL_KUBE_RESOURCE_KIND
            value: "configmap"
          - name: STSENTINEL_METRICS_LISTEN_ADDRESS
            value: "0.0.0.0:8080"
          ## Uncomment this to enable debug logs
          #- name: STSENTINEL_DEBUG
          #  value: "true"
        ports:
          - containerPort: 8080
---
apiVersion: v1
kind: Secret
metadata:
    name: stolon
type: Opaque
data:
    password: cGFzc3dvcmQx
---
# PetSet was renamed to StatefulSet in k8s 1.5
# apiVersion: apps/v1alpha1
# kind: PetSet
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: stolon-keeper
spec:
  serviceName: "stolon-keeper"
  replicas: 2
  template:
    metadata:
      labels:
        component: stolon-keeper
        stolon-cluster: kube-stolon
      annotations:
        pod.alpha.kubernetes.io/initialized: "true"
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: stolon-keeper
        image: sorintlab/stolon:master-pg10
        command:
          - "/bin/bash"
          - "-ec"
          - |
            # Generate our keeper uid using the pod index
            IFS='-' read -ra ADDR <<< "$(hostname)"
            export STKEEPER_UID="keeper${ADDR[-1]}"
            export POD_IP=$(hostname -i)
            export STKEEPER_PG_LISTEN_ADDRESS=$POD_IP
            export STOLON_DATA=/stolon-data
            chown stolon:stolon $STOLON_DATA
            exec gosu stolon stolon-keeper --data-dir $STOLON_DATA
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: STKEEPER_CLUSTER_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.labels['stolon-cluster']
          - name: STKEEPER_STORE_BACKEND
            value: "kubernetes"
          - name: STKEEPER_KUBE_RESOURCE_KIND
            value: "configmap"
          - name: STKEEPER_PG_REPL_USERNAME
            value: "repluser"
            # Or use a password file like in the below supersuser password
          - name: STKEEPER_PG_REPL_PASSWORD
            value: "replpassword"
          - name: STKEEPER_PG_SU_USERNAME
            value: "postgres"
          - name: STKEEPER_PG_SU_PASSWORDFILE
            value: "/etc/secrets/stolon/password"
          - name: STKEEPER_METRICS_LISTEN_ADDRESS
            value: "0.0.0.0:8080"
          # Uncomment this to enable debug logs
          #- name: STKEEPER_DEBUG
          #  value: "true"
        ports:
          - containerPort: 5432
          - containerPort: 8080
        volumeMounts:
        - mountPath: /stolon-data
          name: data
        - mountPath: /etc/secrets/stolon
          name: stolon
      volumes:
        - name: stolon
          secret:
            secretName: stolon
  # Define your own volumeClaimTemplate. This example uses dynamic PV provisioning with a storage class named "standard" (so it will works by default with minikube)
  # In production you should use your own defined storage-class and configure your persistent volumes (statically or dynamically using a provisioner, see related k8s doc).
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
        - "ReadWriteOnce"
      resources:
        requests:
          storage: "512Mi"
      storageClassName: nfs
---
apiVersion: v1
kind: Service
metadata:
  name: stolon-proxy-service
spec:
  ports:
    - port: 5432
      targetPort: 5432
  selector:
    component: stolon-proxy
    stolon-cluster: kube-stolon
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: stolon-proxy
spec:
  replicas: 2
  template:
    metadata:
      labels:
        component: stolon-proxy
        stolon-cluster: kube-stolon
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
    spec:
      containers:
      - name: stolon-proxy
        image: sorintlab/stolon:master-pg10
        command:
          - "/bin/bash"
          - "-ec"
          - |
            exec gosu stolon stolon-proxy
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: STPROXY_CLUSTER_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.labels['stolon-cluster']
          - name: STPROXY_STORE_BACKEND
            value: "kubernetes"
          - name: STPROXY_KUBE_RESOURCE_KIND
            value: "configmap"
          - name: STPROXY_LISTEN_ADDRESS
            value: "0.0.0.0"
          - name: STPROXY_METRICS_LISTEN_ADDRESS
            value: "0.0.0.0:8080"
          ## Uncomment this to enable debug logs
          #- name: STPROXY_DEBUG
          #  value: "true"
        ports:
          - containerPort: 5432
          - containerPort: 8080
        readinessProbe:
          tcpSocket:
            port: 5432
          initialDelaySeconds: 10
          timeoutSeconds: 5

三、部署stolon

kubectl applay -f stolon.yaml

Initialize the cluster（大概意思是stolon初始化k8s集群，可以大概看下官網(wǎng)解釋）

All the stolon components wait for an existing clusterdata entry in the store. So the first time you have to initialize a new cluster. For more details see the cluster initialization doc. You can do this step at every moment, now or after having started the stolon components.
You can execute stolonctl in different ways:

  • as a one shot command executed inside a temporary pod:
kubectl run -i -t stolonctl --image=sorintlab/stolon:master-pg10 --restart=Never --rm -- /usr/local/bin/stolonctl --cluster-name=kube-stolon --store-backend=kubernetes --kube-resource-kind=configmap init
  • from a machine that can access the store backend:
stolonctl --cluster-name=kube-stolon --store-backend=kubernetes --kube-resource-kind=configmap init
  • later from one of the pods running the stolon components.

執(zhí)行

kubectl run -i -t stolonctl --image=sorintlab/stolon:master-pg10 --restart=Never --rm -- /usr/local/bin/stolonctl --cluster-name=kube-stolon --store-backend=kubernetes --kube-resource-kind=configmap init

如圖所示，部署成功

stolon pod

四、卸載Postgresql數(shù)據(jù)庫

kubectl delete -f stolon.yaml
kubectl delete pvc data-stolon-keeper-0 data-stolon-keeper-1

五、驗證Postgresql安裝成功（也可簡單測試下）

1、驗證數(shù)據(jù)同步

連接master并且建立test表

psql --host <IP> --port 30543 postgres -U stolon -W
postgres=# create table test (id int primary key not null,
value text not null);
CREATE TABLE
postgres=# insert into test values (1, 'value1');
INSERT 0 1
postgres=# select * from test;
id | value
---- --------
1 | value1
(1 row)

也可進(jìn)入Pod執(zhí)行postgresql命令

kubectl exec -ti stolon-proxy-5977cdbcfc-csnkq bash
#登入sql
psql --host localhost --port 5432 postgres -U postgres 
\l                      #列出所有數(shù)據(jù)庫
\c dbname               #切換數(shù)據(jù)庫
CREATE TABLE
insert into test values (1, 'value1');
INSERT 0 1
select * from test;
\d                      #列出當(dāng)前數(shù)據(jù)庫的所有表
\q                      #退出數(shù)據(jù)庫

連接slave并且檢查數(shù)據(jù)。你可以寫一些信息以便確認(rèn)請求已經(jīng)被slave處理了。

psql --host <IP> --port 30544 postgres -U stolon -W
postgres=# select * from test;
id | value
---- --------
1 | value1
(1 row)

2、測試failover

這個案例是官方代碼庫中statefullset的一個例子。
簡單的說，就是為模擬了master掛掉，我們先刪除了master的statefulset又刪除了master的pod。

kubectl delete statefulset stolon-keeper --cascade=false
kubectl delete pod stolon-keeper-0

然后，在sentinel的log中我們可以看到新的master被選舉出來了。

no keeper info available db=cb96f42d keeper=keeper0
no keeper info available db=cb96f42d keeper=keeper0
master db is failed db=cb96f42d keeper=keeper0
trying to find a standby to replace failed master
electing db as the new master db=087ce88a keeper=keeper1

現(xiàn)在，在剛才的那兩個終端中如果我們重復(fù)上一個命令，我們可以看到如下輸出。

postgres=# select * from test;
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset:
Succeeded.
postgres=# select * from test;
id | value
---- --------
1 | value1
(1 row)</pre>

Kubernetes的service把不可用的pod去掉，把請求轉(zhuǎn)到可用的pod上。所以新的讀取連接被路由到了健康的pod上。

.也可用chaoskube模擬隨機(jī)的pod掛掉（準(zhǔn)生產(chǎn)可以測試下）

另一個測試集群彈性（resilience）的好方法是用chaoskube。Chaoskube是一個小的服務(wù)程序，它可以周期性的在集群里隨機(jī)的kill掉一些的pod。它也可以用helm charts部署。

helm install --set labels="release=factualcrocodile,
component!=factual-crocodine-etcd" --set
interval=5m stable/chaoskube

這條命令會運行chaoskube，它會每5分鐘刪除一個pod。它會選擇label中release=factual-crocodile的pod，但是會忽略etcd的pod。

本文按照官網(wǎng)搭建，主要為之后的Harbor高可用做準(zhǔn)備，有情趣的伙伴點個贊，之后會續(xù)寫redis、Harbor高可用

參考資料：
https://my.oschina.net/u/2306127/blog/2991474
https://github.com/sorintlab/stolon/tree/master/examples/kubernetes
https://studygolang.com/articles/19002?fr=sidebar

最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點，簡書系信息發(fā)布平臺，僅提供信息存儲服務(wù)。

友情鏈接更多精彩內(nèi)容