二進(jìn)制

K8S
一、二進(jìn)制搭建
1.安裝要求
(1)CentOS 7
(2)禁止swap
(3)集群間互通

2.操作系統(tǒng)初始化
(1)關(guān)閉防火墻
(2)關(guān)閉selinux
(3)關(guān)閉swap
swapoff -a
/etc/fstab 注釋swap掛載
(4)設(shè)置主機(jī)名
(5)設(shè)置hosts
(6)時(shí)間同步
(7)將IPV4流量傳遞到iptables鏈
[root@localhost ~]# cat /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
sysctl --system //生效

3.為etcd和apiserver自簽證書
(1)cfssl：json形式
chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo
mkdir -p ~/TLS/{etcd,k8s} //創(chuàng)建目錄
生成證書配置:
cat > ca-config.json<< EOF
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"www": {
"expiry": "87600h",
"usages": [
"signing","key encipherment",
"server auth",
"client auth"
]
}
}
}
}
EOF

cat > ca-csr.json<< EOF
{
"CN": "etcd CA",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing"
}
]
}
EOF

cfssl gencert -initca ca-csr.json | cfssljson -bare ca - //生成
ls *pem //查看生成的證書

使用自簽CA簽發(fā)ETCD HTTPS證書
創(chuàng)建證書申請(qǐng)文件：
cat > server-csr.json<< EOF
{
"CN": "etcd",
"hosts": [
"11.61.21.166", //該IP為所有ETCD節(jié)點(diǎn)的集群內(nèi)部通信IP
"11.61.21.167",
"11.61.21.168",
"11.61.21.169"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing"
}
]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
ls *server

(2)Openssl

4.部署etcd集群
上傳etcd軟件包
etcd-v3.4.9-linux-amd64.tar.gz
mkdir -p /opt/etcd/{cfg,bin,ssl}

創(chuàng)建etcd配置文件
cat /opt/etcd/cfg/etcd.conf
[root@master cfg]# cat etcd.conf | grep -Ev "$|#"
ETCD_NAME="etcd-1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://11.61.21.166:2380"
ETCD_LISTEN_CLIENT_URLS="https://11.61.21.166:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://11.61.21.166:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://11.61.21.166:2379"
ETCD_INITIAL_CLUSTER="etcd-1=https://11.61.21.166:2380,etcd-2=https://11.61.21.167:2380,etcd-3=https://11.61.21.168:2380,etcd-4=https://11.61.21.169:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

拷貝之前創(chuàng)建的ssl證書至/opt/etcd/ssl
[root@master ssl]# cp ~/TLS/etcd/{ca,server,server-key}.pem .

添加systemd管理etcd
[root@master system]# cat etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=/opt/etcd/cfg/etcd.conf
ExecStart=/opt/etcd/bin/etcd
--cert-file=/opt/etcd/ssl/server.pem
--key-file=/opt/etcd/ssl/server-key.pem
--peer-cert-file=/opt/etcd/ssl/server.pem
--peer-key-file=/opt/etcd/ssl/server-key.pem
--trusted-ca-file=/opt/etcd/ssl/ca.pem
--peer-trusted-ca-file=/opt/etcd/ssl/ca.pem
--logger=zap
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

將master上/opt/etcd/ etcd.service 下發(fā)至Node
systemctl daemon-reload
systemctl start etcd
systemctl status etcd
systemctl enable etcd

5.安裝docker
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
[Service]
Type=notify
ExecStart=/usr/bin/dockerd
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target

6.使用harbor做為鏡像倉庫
依賴docker-compose
cp docker-compose /usr/local/bin
軟件包：harbor-offline-installer-v2.1.0.tgz
cd harbor
vim harbor.yml
修改hostname //建議使用ip或自定義倉庫name
./install.sh

vim /etc/docker/daemon.json
{
"insecure-registries": ["11.61.21.166"] //該地址為harbor地址
}

7.APIserver自簽證書
cat > ca-config.json<< EOF
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"kubernetes": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}}
}
EOF

cat > ca-csr.json<< EOF
{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "Beijing",
"ST": "Beijing",
"O": "k8s",
"OU": "System"
}
]
}
EOF

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

cat > server-csr.json<< EOF
{
"CN": "kubernetes",
"hosts": [
"11.61.21.166",
"11.61.21.167",
"11.61.21.168",
"11.61.21.169",
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing","O": "k8s",
"OU": "System"
}
]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server
ls server*pem

8.部署master組件
上傳軟件包：kubernetes-server-linux-amd64.tar.gz
mkdir -p /opt/kubernetes/{bin,cfg,ssl,logs}
cp kube-apiserver kube-scheduler kube-controller-manager /opt/kubernetes/bin
cp kubectl /usr/bin/

(1)Kube-apiserver
KUBE_APISERVER_OPTS="--logtostderr=false
--v=2
--log-dir=/opt/kubernetes/logs
--etcd-servers=https://11.61.21.166:2379,https://11.61.21.167:2379,https://11.61.21.168:2379,https://11.61.21.169:2379
--bind-address=11.61.21.166
--secure-port=6443
--advertise-address=11.61.21.166
--allow-privileged=true
--service-cluster-ip-range=10.0.0.0/24
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction
--authorization-mode=RBAC,Node
--enable-bootstrap-token-auth=true
--token-auth-file=/opt/kubernetes/cfg/token.csv
--service-node-port-range=30000-32767
--kubelet-client-certificate=/opt/kubernetes/ssl/server.pem
--kubelet-client-key=/opt/kubernetes/ssl/server-key.pem
--tls-cert-file=/opt/kubernetes/ssl/server.pem
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem
--client-ca-file=/opt/kubernetes/ssl/ca.pem
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem
--etcd-cafile=/opt/etcd/ssl/ca.pem
--etcd-certfile=/opt/etcd/ssl/server.pem
--etcd-keyfile=/opt/etcd/ssl/server-key.pem
--audit-log-maxage=30
--audit-log-maxbackup=3
--audit-log-maxsize=100
--audit-log-path=/opt/kubernetes/logs/k8s-audit.log"

-logtostderr：?jiǎn)⒂萌罩?br> -v：日志等級(jí)
-log-dir：日志目錄
-etcd-servers：etcd 集群地址
-bind-address：監(jiān)聽地址
-secure-port：https 安全端口
-advertise-address：集群通告地址
-allow-privileged：?jiǎn)⒂檬跈?quán)
-service-cluster-ip-range：Service 虛擬 IP 地址段
-enable-admission-plugins：準(zhǔn)入控制模塊
-authorization-mode：認(rèn)證授權(quán)，啟用 RBAC 授權(quán)和節(jié)點(diǎn)自管理
-enable-bootstrap-token-auth：?jiǎn)⒂?TLS bootstrap 機(jī)制
-token-auth-file：bootstrap token 文件
-service-node-port-range：Service nodeport 類型默認(rèn)分配端口范圍
-kubelet-client-xxx：apiserver 訪問 kubelet 客戶端證書
-tls-xxx-file：apiserver https 證書
-etcd-xxxfile：連接 Etcd 集群證書
-audit-log-xxx：審計(jì)日志

生成token.csv用于第一次沒有證書時(shí)連接apiserver
echo "head -c 16 /dev/urandom | od -An -t x | tr -d ' ',kubelet-bootstrap,10001,"system:kubelet-bootstrap"" > token.csv

Systemd管理
cat /usr/lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-apiserver.conf
ExecStart=/opt/kubernetes/bin/kube-apiserver $KUBE_APISERVER_OPTS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

授權(quán)kubelet-bootstrap用戶允許請(qǐng)求證書
[root@master cfg]# kubectl create clusterrolebinding kubelet-bootstrap \

--clusterrole=system:node-bootstrapper
--user=kubelet-bootstrap
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created

(2)Kube-controller-manager
KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=false
--v=2
--log-dir=/opt/kubernetes/logs
--leader-elect=true
--master=127.0.0.1:8080
--bind-address=127.0.0.1
--allocate-node-cidrs=true
--cluster-cidr=10.244.0.0/16
--service-cluster-ip-range=10.0.0.0/24
--cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem
--cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem
--root-ca-file=/opt/kubernetes/ssl/ca.pem
--service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem
--experimental-cluster-signing-duration=87600h0m0s"

生成kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes
[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-controller-manager.conf
ExecStart=/opt/kubernetes/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_OPTS
Restart=on-failure
[Install]
WantedBy=multi-user.target

systemctl daemon-reload
systemctl start kube-controller-manager
systemctl enable kube-controller-manager

(3)Kube-scheduler
cat kube-scheduler.conf
KUBE_SCHEDULER_OPTS="--logtostderr=false
--v=2
--log-dir=/opt/kubernetes/logs
--leader-elect
--master=127.0.0.1:8080
--bind-address=127.0.0.1"

cat kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes
[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-scheduler.conf
ExecStart=/opt/kubernetes/bin/kube-scheduler $KUBE_SCHEDULER_OPTS
Restart=on-failure
[Install]
WantedBy=multi-user.target

systemctl daemon-reload
systemctl start kube-scheduler
systemctl enable kube-scheduler

Master所有組件部署完成，檢查狀態(tài)
kubelet get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-1 Healthy {"health":"true"}
etcd-2 Healthy {"health":"true"}
etcd-3 Healthy {"health":"true"}
etcd-0 Healthy {"health":"true"}

9.部署node組件
mkdir -p /opt/kubernetes/{bin,cfg,ssl,logs}
將kubelet,kube-proxy拷貝至/opt/kubernets/bin
將kubectl 拷貝至/usr/bin
需要的證書ca.pem ca-key.pem kube-proxy.pem kube-proxy-key.pem server.pem server-key.pem

生成bootstrap.kubeconfig
先添加環(huán)境變量
KUBE_APISERVER="https://11.61.21.166:6443" #apiserver
TOKEN="c659724a32f7ec103ddfa5ae62d2619d" #token.csv

kubectl config set-cluster kubernetes
--certificate-authority=/opt/kubernetes/ssl/ca.pem
--embed-certs=true
--server=${KUBE_APISERVER}
--kubeconfig=bootstrap.kubeconfig

kubectl config set-credentials "kubelet-bootstrap"
--token=${TOKEN}
--kubeconfig=bootstrap.kubeconfig
kubectl config set-context default
--cluster=kubernetes
--user="kubelet-bootstrap"
--kubeconfig=bootstrap.kubeconfig

kubectl config use-context default --kubeconfig=bootstrap.kubeconfig

(1)Kubelet
cat kubelet.conf
KUBELET_OPTS="--logtostderr=false
--v=2
--log-dir=/opt/kubernetes/logs
--hostname-override=k8s-master
--network-plugin=cni
--kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig
--bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig
--config=/opt/kubernetes/cfg/kubelet-config.yml
--cert-dir=/opt/kubernetes/ssl
--pod-infra-container-image=11.61.21.166/k8s/pause-amd64:3.0" //pause容器鏡像地址

cat kubelet-config.yml
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: 0.0.0.0
port: 10250
readOnlyPort: 10255
cgroupDriver: cgroupfs
clusterDNS:

  • 10.0.0.2
    clusterDomain: cluster.local
    failSwapOn: false
    authentication:
    anonymous:
    enabled: false
    webhook:
    cacheTTL: 2m0s
    enabled: true
    x509:
    clientCAFile: /opt/kubernetes/ssl/ca.pem
    authorization:
    mode: Webhook
    webhook:
    cacheAuthorizedTTL: 5m0s
    cacheUnauthorizedTTL: 30s
    evictionHard:
    imagefs.available: 15%
    memory.available: 100Mi
    nodefs.available: 10%
    nodefs.inodesFree: 5%
    maxOpenFiles: 1000000
    maxPods: 110

vim kubelet.service
[Unit]
Description=Kubernetes Kubelet
After=docker.service
[Service]
EnvironmentFile=/opt/kubernetes/cfg/kubelet.conf
ExecStart=/opt/kubernetes/bin/kubelet $KUBELET_OPTS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

systemctl daemon-reload
systemctl start kubelet
systemctl enable kubelet

在master上查看證書申請(qǐng)
[root@master cfg]# kubectl get csr
NAME AGE SIGNERNAME REQUESTOR CONDITION
node-csr-nx6sszAqMpBMAybiezfy-Olv-1tyHZ800Lun-AmdfEI 5m20s kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Pending
通過證書申請(qǐng)
[root@master cfg]# kubectl certificate approve node-csr-nx6sszAqMpBMAybiezfy-Olv-1tyHZ800Lun-AmdfEI
certificatesigningrequest.certificates.k8s.io/node-csr-nx6sszAqMpBMAybiezfy-Olv-1tyHZ800Lun-AmdfEI approved

(2)Kube-proxy
vim kube-proxy.conf
KUBE_PROXY_OPTS="--logtostderr=false
--v=2
--log-dir=/opt/kubernetes/logs
--config=/opt/kubernetes/cfg/kube-proxy-config.yml"

vim kube-proxy-config.yml
kind: KubeProxyConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
metricsBindAddress: 0.0.0.0:10249
clientConnection:
kubeconfig: /opt/kubernetes/cfg/kube-proxy.kubeconfig
hostnameOverride: k8s-master
clusterCIDR: 10.0.0.0/24

創(chuàng)建proxy的證書(master上)
vim kube-proxy-csr.json
{
"CN": "system:kube-proxy",
"hosts": [],"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"L": "BeiJing",
"ST": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}

生成證書
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy
ls kube-proxy*pem

生成kubeconfig文件
kubectl config set-cluster kubernetes
--certificate-authority=/opt/kubernetes/ssl/ca.pem
--embed-certs=true
--server=${KUBE_APISERVER}
--kubeconfig=kube-proxy.kubeconfig

kubectl config set-credentials kube-proxy
--client-certificate=/opt/kubernetes/ssl/kube-proxy.pem
--client-key=/opt/kubernetes/ssl/kube-proxy-key.pem
--embed-certs=true
--kubeconfig=kube-proxy.kubeconfig

kubectl config set-context default
--cluster=kubernetes
--user=kube-proxy
--kubeconfig=kube-proxy.kubeconfig

kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig

創(chuàng)建kube-proxy.servce
[Unit]
Description=Kubernetes Proxy
After=network.target
[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-proxy.conf
ExecStart=/opt/kubernetes/bin/kube-proxy $KUBE_PROXY_OPTS
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

10.部署集群網(wǎng)絡(luò)插件
CNI網(wǎng)絡(luò)
Kube-flannel.yml配置文件
wget
https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-
flannel.yml
Kube-flannel.yml中需要修改flannel鏡像地址
kubectl apply -f kube-flannel.yml //master執(zhí)行
kubectl get pods -n kube-system //檢查執(zhí)行狀態(tài)，-n 表示namespace
kubectl describe pod POD_NAME -n NAMESPACE
kubectl get nodes

11.授權(quán)apiserver訪問kubelet
vim apiserver-to-kubelet-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:kube-apiserver-to-kubelet
rules:

  • apiGroups:
    • ""
      resources:
    • nodes/proxy
    • nodes/stats
    • nodes/log
    • nodes/spec
    • nodes/metrics
    • pods/log
      verbs:
    • "*"

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: system:kube-apiserver
namespace: ""
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:kube-apiserver-to-kubelet
subjects:

  • apiGroup: rbac.authorization.k8s.io
    kind: User
    name: kubernetes

二、kubectl
1.語法：kubectl [COMMAND] [TYPE] [NAME] [flags]
command:對(duì)資源執(zhí)行的操作，create、get、describe和delete等
TYPE:指定資源類型，資源類型大小寫敏感，開發(fā)者能以單數(shù)、復(fù)數(shù)和縮略的形式，例如kubectl get pod/pods/po
NAME:即POD_NAME
flags:指定可選參數(shù)，例如可用-s或者-server指定k8s API server的地址和端口

三、yaml
K8s中對(duì)資源管理和資源對(duì)象編排都通過聲明樣式(yaml)文件來定義，通常把這樣的yaml文件叫做資源清單文件，通過Kubectl直接使用資源清單文件來進(jìn)行編排部署。
yaml是一種標(biāo)記語言，為了強(qiáng)調(diào)，這種語言以數(shù)據(jù)為中心，不以標(biāo)記語言為重點(diǎn)?？勺x性高，用來表達(dá)數(shù)據(jù)序列的格式。以縮進(jìn)表示層級(jí)關(guān)系，使用空格來做為縮進(jìn)，空格數(shù)量不重要，只要相同層級(jí)的元素左側(cè)對(duì)其即可。
語法:
縮進(jìn)表示層級(jí)關(guān)系
不能使用tab進(jìn)行縮進(jìn)
一般開頭縮進(jìn)兩個(gè)空格
字符后縮進(jìn)一個(gè)空格，比如: , -
使用 --- 表示一個(gè)新的yaml文件
使用#標(biāo)識(shí)注釋

組成部分:
控制器定義: template前

被控制對(duì)象:template后

常用字段含義:
apiVersion: API版本 //kubectl api-versions 查看所有版本
kind:資源類型 // kubectl api-resources .
metadata:資源元數(shù)據(jù) //name , namespace
spec:資源規(guī)格
replicas:副本數(shù)量
selector:標(biāo)簽選擇器
template:POD模板
metadata:POD元數(shù)據(jù)
spec:POD規(guī)格
containers:容器配置 //名字，鏡像版本，端口等等

快速編寫一個(gè)yaml文件
1.使用kubectl create 生成yaml文件 //資源未生成
kubectl create deployment web --image=nginx -o yaml --dry-run > n1.yaml
2.使用kubectl get 導(dǎo)出yaml文件 //資源已生成
kubectl get deploy nginx -o=yaml --export > n2.yaml

四、POD
1.基本概念
?可以創(chuàng)建的最小的管理單元
?k8s不直接處理容器，而是通過pod，pod是一個(gè)或一組容器的集合
?一個(gè)pod中的容器共享網(wǎng)絡(luò)命名空間
?pod是短暫存在的
?每個(gè)pod都有一個(gè)pause容器，pause容器對(duì)應(yīng)的鏡像屬于k8s平臺(tái)的一部分、
2.pod存在的意義
?創(chuàng)建容器使用docker，一個(gè)docker對(duì)應(yīng)的是一個(gè)容器，一個(gè)容器對(duì)應(yīng)一個(gè)應(yīng)用程序
?pod是多進(jìn)程設(shè)計(jì)，運(yùn)行多個(gè)應(yīng)用程序
?pod存在為了親密性應(yīng)用
多個(gè)應(yīng)用之間進(jìn)行交互
網(wǎng)絡(luò)間的調(diào)用(通過socket或者127.0.0.1)
兩個(gè)應(yīng)用需要頻繁進(jìn)行調(diào)用
3.實(shí)現(xiàn)機(jī)制
容器間相互隔離，利用Linux的namespace 和group
?共享網(wǎng)絡(luò)
前提條件：多個(gè)容器在同一個(gè)namespace里
POD中先創(chuàng)建pause容器，再創(chuàng)建業(yè)務(wù)容器，再將業(yè)務(wù)容器加入至pause容器，共享一個(gè)ip\mac\port
?共享存儲(chǔ)
pod持久化數(shù)據(jù):日志、業(yè)務(wù)數(shù)據(jù)等
使用volume數(shù)據(jù)卷持久化數(shù)據(jù)，POD從volume中讀寫

4.鏡像拉取策略
imagePullPolicy(yml spec.containers中定義)
IFNotPresent:默認(rèn)值，鏡像再宿主機(jī)上不存在時(shí)才拉取
Always:每次創(chuàng)建Pod都會(huì)重新拉取一次鏡像
Never:Pod永遠(yuǎn)不會(huì)主動(dòng)拉取這個(gè)鏡像
5.Pod中資源限制
spec.containers.resource.requests.cpu //最低,調(diào)度大小 cpu單位m 1c=1000m
spec.containers.resource.limits.cpu //最大
6.Pod重啟策略
spec.restarPolicy
Always:當(dāng)容器終止退出后，總是重啟容器，默認(rèn)策略
OnFailure:當(dāng)容器異常退出（退出狀態(tài)碼非0）時(shí)，才重啟容器
Never:從不重啟容器
7.健康檢查
容器檢查 state
應(yīng)用層面健康檢查

spec.containers.livenessProbe:
exec:
command:
兩種檢查機(jī)制:
livenessProbe(存活檢查)
如果檢查失敗，將殺死容器，根據(jù)Pod的restartPolicy來操作
readinessProbe(就緒檢查)
如果檢查失敗，k8s會(huì)把Pod從service endpoints中剔除

Probe檢查方法:
httpGet
發(fā)送http請(qǐng)求，返回200-400狀態(tài)碼為成功
exec
執(zhí)行shell命令，返回狀態(tài)碼為0為成功
tcpSocket
發(fā)起TCP socket建立 成功
8.調(diào)度策略
創(chuàng)建流程：
master節(jié)點(diǎn)
create pod --> apiserver -->etcd(存儲(chǔ))
scheduler -->apiserver (watch是否有新pod創(chuàng)建)--> etcd(讀取) --> 調(diào)度算法，把Pod調(diào)度到某個(gè)節(jié)點(diǎn)上
node節(jié)點(diǎn)
kubelet --> apiserver --> etcd(讀取) --> docker --> update pod status

影響調(diào)度的屬性
(1)資源限制對(duì)Pod調(diào)度產(chǎn)生影響
根據(jù)request找到滿足需求的node節(jié)點(diǎn)進(jìn)行調(diào)度
(2)節(jié)點(diǎn)選擇器標(biāo)簽
spec.nodeSelector:
env_role:dev

首先給節(jié)點(diǎn)起別名
kubectl label node node1(hostname) env_role=dev
查看標(biāo)簽
kubectl get nodes (hostname) --show-lables
(3)節(jié)點(diǎn)親和性
spec.affinity.nodeAffinity
和nodeSelector類似，根據(jù)節(jié)點(diǎn)上約束來決定Pod調(diào)度到哪些節(jié)點(diǎn)上,支持表達(dá)式 - matchExpressions
硬親和性
requiredDuringSchedulingIgnoredDuringExecution
表示約束條件必須滿足

nodeSelectorTerms:

  • matchExpressions: //表達(dá)式
  • key: env_role
    operation: In //支持 In\NotIn\Exists\Gt\Lt\DoesNotExists
    values:
    • dev
    • test
      軟親和性
      嘗試滿足條件，不保證一定滿足
      preferredDuringSchedulingIgnoredDuringExecution:
  • weigh: 1 //權(quán)重
    preference:
  • matchExpressions:
  • key: group
    operation: In
    values:
    • otherprod
      反親和性
      NotIn,DoesNotExists

污點(diǎn)和污點(diǎn)容忍
nodeSelector和nodeAffinity:根據(jù)這兩個(gè)中的配置把Pod調(diào)度到某些節(jié)點(diǎn)上實(shí)現(xiàn)，屬于Pod屬性，在調(diào)度中實(shí)現(xiàn)
Taint污點(diǎn)：節(jié)點(diǎn)不做普通分配調(diào)度 屬于節(jié)點(diǎn)屬性

應(yīng)用場(chǎng)景：
專用節(jié)點(diǎn)，針對(duì)某些業(yè)務(wù)或用戶特定分配
配置特殊硬件節(jié)點(diǎn)
基于Taint驅(qū)逐

查看當(dāng)前節(jié)點(diǎn)污點(diǎn)情況:kubectl describe nodes HOSTNAME | grep Taint
污點(diǎn)值:NoSchedule(該節(jié)點(diǎn)不被調(diào)度)/PreferNoSchedule(盡量不被調(diào)度)/NoExecute(不會(huì)調(diào)度，并且還會(huì)驅(qū)逐已有的Pod)
為節(jié)點(diǎn)打上污點(diǎn)標(biāo)簽:kubectl taint node HOSTNAME key=value:污點(diǎn)值
eg.:[root@master ~]# kubectl create deployment web --image=11.61.21.166/k8s/nginx
deployment.apps/web created
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
web-74cf4dfdcd-gwgj2 1/1 Running 0 9s

[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web-74cf4dfdcd-gwgj2 1/1 Running 0 71s 10.244.1.2 node02 <none> <none>

[root@master ~]# kubectl scale deployment web --replicas=5 //設(shè)置副本數(shù)量
deployment.apps/web scaled

[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web-74cf4dfdcd-gwgj2 1/1 Running 0 4m58s 10.244.1.2 node02 <none> <none>
web-74cf4dfdcd-p4zs2 1/1 Running 0 26s 10.244.1.3 node02 <none> <none>
web-74cf4dfdcd-s6gqj 1/1 Running 0 26s 10.244.2.3 node03 <none> <none>
web-74cf4dfdcd-sz944 1/1 Running 0 26s 10.244.2.2 node03 <none> <none>
web-74cf4dfdcd-vxvpq 1/1 Running 0 26s 10.244.0.2 node01 <none> <none>

[root@master ~]# kubectl taint node node01 env_role=yes:NoSchedule
node/node01 tainted //給node01打上污點(diǎn)值，env_role=yes為自定義
[root@master ~]# kubectl describe nodes node01 | grep Taints
Taints: env_role=yes:NoSchedule

刪除剛才創(chuàng)建的pod：kubectl delete deployment web
重新創(chuàng)建
root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
web-74cf4dfdcd-5vwgl 1/1 Running 0 12s 10.244.2.5 node03 <none> <none>
web-74cf4dfdcd-7lxbx 1/1 Running 0 12s 10.244.2.6 node03 <none> <none>
web-74cf4dfdcd-kjglh 1/1 Running 0 12s 10.244.1.4 node02 <none> <none>
web-74cf4dfdcd-mjcz4 1/1 Running 0 32s 10.244.2.4 node03 <none> <none>
web-74cf4dfdcd-r79w5 1/1 Running 0 12s 10.244.1.5 node02 <none> <none>
刪除污點(diǎn)：
[root@master ~]# kubectl taint node node01 env_role:NoSchedule-
node/node01 untainted

污點(diǎn)容忍
spec:
tolerations:

  • key: “KEY” //設(shè)置污點(diǎn)時(shí)的KEY
    operator: “Equal”
    value: “VALUE” //設(shè)置污點(diǎn)時(shí)的value
    effect: “NoSchedule”

五、Controller---無狀態(tài)應(yīng)用
1.什么是controller
是實(shí)際存在的，是管理和運(yùn)行容器的對(duì)象，controller有很多種
2.Pod和controller間的關(guān)系
Pod通過controller來實(shí)現(xiàn)應(yīng)用的運(yùn)維
比如伸縮、滾動(dòng)升級(jí)等等
Pod和controller間通過label建立關(guān)系
3.Deployment控制器應(yīng)用場(chǎng)景
部署無狀態(tài)應(yīng)用
管理Pod和replicaset
部署、滾動(dòng)升級(jí)等功能
web服務(wù)、微服務(wù)
4.yaml文件字段
使用deployment部署
kubectl create deployment web --image=11.61.21.166/k8s/nginx --dry-run -o yaml > web.yaml //創(chuàng)建yaml文件
kubectl apply -f web.yaml //根據(jù)yaml文件部署
對(duì)外發(fā)布(暴露對(duì)外端口號(hào))
kubectl expose deployment web --port=80 --type=NodePort --target-port=80 --name=web1 -o yaml > web1.yaml

5.Deployment控制器部署應(yīng)用
6.滾動(dòng)升級(jí)/回滾
kubectl set image deployment web nginx=11.61.21.166/k8s/nginx:latest
kubectl rollout status deployment web
deployment "web" successfully rolled out

kubectl rollout history deployment web
deployment.apps/web
REVISION CHANGE-CAUSE
1 <none>
2 <none>
kubectl rollout undo deployment web //回滾至上一版本

[root@master ~]# kubectl rollout undo deployment web --to-revision=2 //回滾至指定版本
deployment.apps/web rolled back
[root@master ~]# kubectl rollout status deployment web
Waiting for deployment "web" rollout to finish: 4 out of 5 new replicas have been updated...
Waiting for deployment "web" rollout to finish: 4 out of 5 new replicas have been updated...
Waiting for deployment "web" rollout to finish: 4 out of 5 new replicas have been updated...
Waiting for deployment "web" rollout to finish: 2 old replicas are pending termination...
Waiting for deployment "web" rollout to finish: 2 old replicas are pending termination...
Waiting for deployment "web" rollout to finish: 2 old replicas are pending termination...
Waiting for deployment "web" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "web" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "web" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "web" rollout to finish: 4 of 5 updated replicas are available...
deployment "web" successfully rolled out

7.彈性伸縮
kubectl scale deployment web --replicas=10 //會(huì)把副本數(shù)量擴(kuò)至10個(gè)
kubectl autoscale deployment web --min=2 --max=10 //設(shè)置自動(dòng)伸縮
kubectl get hpa //檢查hpa自動(dòng)伸縮狀態(tài)

六、Service
1.service存在的意義
為了防止pod失聯(lián)(服務(wù)發(fā)現(xiàn))
定義一組關(guān)于pod訪問策略

2.Pod和Service的關(guān)系
根據(jù)label和selector建立關(guān)聯(lián)
selector:
app:nginx //service

labels:
app:nginx //pod
3.service類型
ClusterIP:一般用于集群內(nèi)部使用
NodePort:一般用于對(duì)外暴露應(yīng)用時(shí)使用
node內(nèi)網(wǎng)部署應(yīng)用，外網(wǎng)一般不能訪問到
找到一臺(tái)可以進(jìn)行外網(wǎng)訪問的機(jī)器，安裝nginx，進(jìn)行反向代理
手動(dòng)把可以訪問的節(jié)點(diǎn)加到nginx中
LoadBalancer:也是暴露應(yīng)用，一般用于公有云，采用這個(gè)模式的時(shí)候負(fù)載均衡是由公有云提供的，不需要內(nèi)網(wǎng)手動(dòng)進(jìn)行nginx添加
ExternalName:externalName Service是k8s中一個(gè)特殊的service類型，它不需要指定selector去選擇哪些pods實(shí)例提供服務(wù)，而是使用DNS CNAME機(jī)制把自己CNAME到你指定的另外一個(gè)域名上，你可以提供集群內(nèi)的名字，比如mysql.db.svc這樣的建立在db命名空間內(nèi)的mysql服務(wù)，也可以指定http://mysql.example.com這樣的外部真實(shí)域名。后期學(xué)習(xí)

spec:
clusterIP: 10.0.0.219
externalTrafficPolicy: Cluster
ports:

  • nodePort: 32048
    port: 80
    protocol: TCP
    targetPort: 80
    selector:
    app: web
    sessionAffinity: None
    type: NodePort //設(shè)為NodePort

七、Controller---有狀態(tài)應(yīng)用
1.無狀態(tài)和有狀態(tài)
deployment部署的都是無狀態(tài)應(yīng)用
無狀態(tài)特點(diǎn)：
認(rèn)為Pod都是一樣的
應(yīng)用沒有順序要求
不考慮應(yīng)用在哪個(gè)Node上運(yùn)行應(yīng)用
隨意進(jìn)行伸縮和擴(kuò)展

有狀態(tài)特點(diǎn)：
上面每個(gè)因素都要考慮到
讓每個(gè)Pod獨(dú)立，保持pod啟動(dòng)順序和唯一性
唯一網(wǎng)絡(luò)標(biāo)識(shí)符區(qū)分pod，并持久存儲(chǔ)
有序，比如mysql主從，先主后從

無頭service
CLUSTER-IP值=none

使用SatefulSet部署有狀態(tài)應(yīng)用
先部署一個(gè)無頭service
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:

  • port: 80
    name: web
    clusterIP: None //設(shè)置為none
    selector:
    app: nginx

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: nginx-statefulset
namespace: default
spec:
serviceName: nginx
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: 11.61.21.166/k8s/nginx:latest
ports:
- containerPort: 80

deployment和satefulset區(qū)別：有身份的（唯一標(biāo)識(shí)）
根據(jù)主機(jī)名+按照一的規(guī)則生成域名
每個(gè)pod有唯一主機(jī)名
唯一域名：主機(jī)名稱.service名稱.命名空間.svc.cluster.local

八、DaemonSet
1.守護(hù)進(jìn)程DaemonSet
在每個(gè)node上運(yùn)行同一個(gè)pod，新加入的node也同樣運(yùn)行在這個(gè)pod里頭
2.部署DaemonSet
例子：在每個(gè)Node節(jié)點(diǎn)上安裝數(shù)據(jù)采集工具
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: ds-test
labels:
app: filebeat
spec:
selector:
matchLabels:
app: filebeat
template:
metadata:
labels:
app: filebeat
spec:
containers:
- name: logs
image: 11.61.21.166/k8s/filebeat:7.8.0
ports:
- containerPort: 80
volumeMounts:
- name: varlog
mountPath: /tmp/log
volumes:
- name: varlog
hostPath:
path: /var/log

九、Job和cronjob
1.job
一次性
apiVersion: batch/v1
kind: Job
metadata:
name: pi
spec:
template:
spec:
containers:
- name: pi
image: perl
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never //重啟策略
backoffLimit: 4 //失敗后重試4次， 默認(rèn)6次

kubectl create -f job.yaml
kubectl get jobs
運(yùn)行完成后 pod狀態(tài)變成completed
kubectl logs $NAME 會(huì)返回執(zhí)行結(jié)果

2.cronjob
定時(shí)

apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: hello
spec:
schedule: "*/1 * * * *" //cron表達(dá)式
jobTemplate:
spec:
template:
spec:
containers:
- name: hello
image: busybox
args:
- /bin/sh
- -c
- date; echo Hello from the Kubernetes cluster
restartPolicy: OnFailure
每次執(zhí)行狀態(tài)變成completed并創(chuàng)建新的pod再執(zhí)行下個(gè)周期

十、Secret
作用：加密數(shù)據(jù)存在etcd中，讓pod 容器以掛載volume的方式進(jìn)行訪問
場(chǎng)景：憑證

創(chuàng)建secret加密數(shù)據(jù)
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
username: YWRtaW4=
password: MWYyZDFlMmU2N2Rm

[root@master ~]# kubectl create -f secret.yaml
secret/mysecret created

掛載到pod中
變量形式：
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: nginx
image: nginx
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef: //這里是valueFrom所以需要先創(chuàng)建secret
name: mysecret
key: username //key:value 來源于secret.yaml中的定義
- name: SECRET_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: password

volume形式：
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:

  • name: nginx
    image: nginx
    volumeMounts:
    • name: foo
      mountPath: "/etc/foo" //容器內(nèi)路徑
      readOnly: true
      volumes:
  • name: foo
    secret:
    secretName: mysecret

十一、ConfigMap
1.作用:存儲(chǔ)不加密數(shù)據(jù)到etcd，讓pod以變量或者volume掛載到容器中
場(chǎng)景:配置文件

2.創(chuàng)建配置文件
[root@master ~]# cat redis.properties
redis.host=0.0.0.0
redis.port=6379
redis.password=123456

3.創(chuàng)建ConfigMap
kubectl create configmap redis-config --from-file=redis.properties
查看
kubectl get cm / kubectl describe cm

4.volume形式掛載
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: busybox
image: busybox
command: [ "/bin/sh","-c","cat /etc/config/redis.properties" ]
volumeMounts:
- name: config-volume
mountPath: /etc/config
volumes:
- name: config-volume
configMap:
name: redis-config
restartPolicy: Never

5.var形式掛載
創(chuàng)建變量
apiVersion: v1
kind: ConfigMap
metadata:
name: myconfig
namespace: default
data:
special.level: info
special.type: hello

apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: busybox
image: busybox
command: [ "/bin/sh", "-c", "echo (LEVEL)(TYPE)" ]
env:
- name: LEVEL
valueFrom:
configMapKeyRef:
name: myconfig
key: special.level
- name: TYPE
valueFrom:
configMapKeyRef:
name: myconfig
key: special.type
restartPolicy: Never

十二、安全機(jī)制
1.概述
當(dāng)訪問K8S集群時(shí)，需要經(jīng)過三個(gè)步驟
?認(rèn)證
傳輸安全：對(duì)外不暴露8080端口，該端口只能內(nèi)部訪問，對(duì)外使用6443
認(rèn)證方式：https證書認(rèn)證(基于ca證書)
http token認(rèn)證,通過token識(shí)別用戶
http基本認(rèn)證，用戶名+密碼認(rèn)證
?鑒權(quán)
基于RBAC方式鑒權(quán)(基于角色訪問控制)
?準(zhǔn)入控制
準(zhǔn)入控制器的列表，如果列表有請(qǐng)求的內(nèi)容則通過，沒有則拒絕
進(jìn)行訪問的時(shí)候，都需要經(jīng)過apiServer,apiServer做統(tǒng)一協(xié)調(diào)，訪問過程中需要證書\token\或者用戶名+密碼，如果需要訪問POD還需要serviceAccount

2.RBAC
基于角色的訪問控制
角色:Role ClusterRole
角色-->資源對(duì)象(pod,node...)-->操作(get,create...)
role:特定命名空間的訪問權(quán)限
ClusterRole:所有命名空間訪問權(quán)限
主體:user 用戶 group 用戶組 serviceaccount服務(wù)賬號(hào)，一般用于pod訪問

給主體設(shè)置角色，主體的訪問限制由角色的定義來決定
角色綁定：RoleBinding,ClusterRoleBinding

3.實(shí)例
創(chuàng)建一個(gè)命名空間
kubectl create ns roledemo
在新命名空間下創(chuàng)建測(cè)試Pod
kubectl run nginx --image=11.61.21.166/k8s/nginx -n roledemo
創(chuàng)建一個(gè)角色
rbac-role.yaml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: roledemo //特定ns
name: pod-reader //角色名
rules:

  • apiGroups: [""] # "" indicates the core API group
    resources: ["pods"]
    verbs: ["get", "watch", "list"] //相應(yīng)操作
    創(chuàng)建角色綁定
    kind: RoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
    name: read-pods
    namespace: roledemo
    subjects:
  • kind: User
    name: mary # Name is case sensitive
    apiGroup: rbac.authorization.k8s.io
    roleRef:
    kind: Role #this must be Role or ClusterRole
    name: pod-reader # this must match the name of the Role or ClusterRole you wish to bind to
    apiGroup: rbac.authorization.k8s.io

使用證書來識(shí)別身份
教程在胡扯，不做記錄

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容