tag: k8s coredns

一、前言

有個(gè)同學(xué)不小心將CoreDNS干掉了，直接使用helm安裝，但是DNS SVC的IP會(huì)重新分配一個(gè)。部署的POD的dns服務(wù)器地址不會(huì)變成新的（老的也不會(huì)改變）。

二、集群示例

kubelet 通過--cluster-dns=<dns-service-ip> 傳遞dns的地址。當(dāng)創(chuàng)建POD時(shí)，如果使用ClusterDNS，則/etc/resolv.conf文件里的地址就使用這個(gè)。參考官網(wǎng)

kubelet狀態(tài)

查看kubelet的PID，及啟動(dòng)配置。

# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/kubelet.service.d
           └─10-node-kubeadm.conf
   Active: active (running) since Wed 2020-04-15 22:37:12 CST; 1 months 24 days ago
     Docs: http://kubernetes.io/docs/
 Main PID: 1181 (kubelet)
   Memory: 354.7M
   CGroup: /system.slice/kubelet.service
           └─1181 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/node-bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/etc/kubelet/no...

kubelet啟動(dòng)參數(shù)

--config=/etc/kubelet/node-config.yaml 節(jié)點(diǎn)配置，里面配置了dns的地址。

[root@k8s-node-vm6fqz-77jtilv0tc ~]# cat /proc/1181/cmdline  | strings -1
/usr/bin/kubelet
--bootstrap-kubeconfig=/etc/kubernetes/node-bootstrap-kubelet.conf
--kubeconfig=/etc/kubernetes/kubelet.conf
--config=/etc/kubelet/node-config.yaml
--network-plugin=cni
--pod-infra-container-image=jdcloud-cn-south-1.jcr.service.jdcloud.com/k8s/pause-amd64:3.1
--kube-reserved=cpu=80m,memory=2562Mi
--node-labels=group=default,failure-domain.beta.kubernetes.io/zone=cn-south-1b,failure-domain.beta.kubernetes.io/region=cn-south-1
--cloud-provider=external

clusterDNS配置

默認(rèn)dns地址為：172.16.56.10

# cat /etc/kubelet/node-config.yaml 
address: 0.0.0.0
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: cgroupfs
cgroupsPerQOS: true
clusterDNS:
- 172.16.56.10
clusterDomain: cluster.local
configMapAndSecretChangeDetectionStrategy: Watch

三、DNS地址由來

以kubeadm部署集群為例，kubeadm默認(rèn)獲取svc網(wǎng)段下的第10個(gè)IP作為DNS的地址。

dns地址

• CoreDNS在創(chuàng)建SVC時(shí)調(diào)用GetDNSIP，顯示的將ClusterIP字段進(jìn)行設(shè)置。
• kubelet初始化時(shí)，調(diào)用GetDNSIP將地址傳入到啟動(dòng)參數(shù)里。

cmd/kubeadm/app/constants/constants.go:GetDNSIP

// GetDNSIP returns a dnsIP, which is 10th IP in svcSubnet CIDR range
func GetDNSIP(svcSubnetList string, isDualStack bool) (net.IP, error) {
    // Get the service subnet CIDR
    svcSubnetCIDR, err := GetKubernetesServiceCIDR(svcSubnetList, isDualStack)
    if err != nil {
        return nil, errors.Wrapf(err, "unable to get internal Kubernetes Service IP from the given service CIDR (%s)", svcSubnetList)
    }

    // Selects the 10th IP in service subnet CIDR range as dnsIP
    dnsIP, err := utilnet.GetIndexedIP(svcSubnetCIDR, 10)
    if err != nil {
        return nil, errors.Wrap(err, "unable to get internal Kubernetes Service IP from the given service CIDR")
    }

    return dnsIP, nil
}

image-20200609120618743

四、POD DNS配置

dns內(nèi)部類型

用戶對(duì)pod的配置，會(huì)在內(nèi)部進(jìn)行轉(zhuǎn)換，轉(zhuǎn)為以下3種。

podDNSCluster k8s集群內(nèi)部的DNS
podDNSHost 主機(jī)節(jié)點(diǎn)的DNS
podDNSNone 不配置DNS

dns配置轉(zhuǎn)換

• “Default”: Pod從運(yùn)行所在的節(jié)點(diǎn)繼承名稱解析配置。
• “ClusterFirst”: dnsPolicy的默認(rèn)配置。與配置的群集域后綴不匹配的任何DNS查詢(例如 “www.kubernetes.io” )都將轉(zhuǎn)發(fā)到從節(jié)點(diǎn)繼承的上游名稱服務(wù)器。 群集管理員可能配置了額外的存根域和上游DNS服務(wù)器。
• “ClusterFirstWithHostNet”: hostNetwork=true的 Pod，應(yīng)顯式設(shè)置其DNS策略 “ClusterFirstWithHostNet”。
• “None”: 它允許 Pod 忽略 Kubernetes 環(huán)境中的 DN S設(shè)置。 應(yīng)該使用 Pod Spec 中的 dnsConfig 字段提供所有 DNS 設(shè)置。

pkg/kubelet/network/dns/dns.go:getPodDNSType

func getPodDNSType(pod *v1.Pod) (podDNSType, error) {
    dnsPolicy := pod.Spec.DNSPolicy
    switch dnsPolicy {
    case v1.DNSNone:
        return podDNSNone, nil
    case v1.DNSClusterFirstWithHostNet:
        return podDNSCluster, nil
    case v1.DNSClusterFirst:
        if !kubecontainer.IsHostNetworkPod(pod) {
            return podDNSCluster, nil
        }
        // Fallback to DNSDefault for pod on hostnetowrk.
        fallthrough
    case v1.DNSDefault:
        return podDNSHost, nil
    }
    // This should not happen as kube-apiserver should have rejected
    // invalid dnsPolicy.
    return podDNSCluster, fmt.Errorf(fmt.Sprintf("invalid DNSPolicy=%v", dnsPolicy))
}

https://kubernetes.io/zh/docs/concepts/services-networking/dns-pod-service/

dns地址

pkg/kubelet/network/dns/dns.go:331

func (c *Configurer) GetPodDNS(pod *v1.Pod) (*runtimeapi.DNSConfig, error) {
    dnsConfig, err := c.getHostDNSConfig()

    dnsType, err := getPodDNSType(pod)

    switch dnsType {
    case podDNSNone:
        dnsConfig = &runtimeapi.DNSConfig{}
    case podDNSCluster:
        // 從c.clusterDNS里獲取dns地址
        if len(c.clusterDNS) != 0 {
            dnsConfig.Servers = []string{}
            for _, ip := range c.clusterDNS {
                dnsConfig.Servers = append(dnsConfig.Servers, ip.String())
            }
            dnsConfig.Searches = c.generateSearchesForDNSClusterFirst(dnsConfig.Searches, pod)
            dnsConfig.Options = defaultDNSOptions
            break
        }
        fallthrough
    case podDNSHost:
        if c.ResolverConfig == "" {
            switch {
            case c.nodeIP == nil || c.nodeIP.To4() != nil:
                dnsConfig.Servers = []string{"127.0.0.1"}
            case c.nodeIP.To16() != nil:
                dnsConfig.Servers = []string{"::1"}
            }
            dnsConfig.Searches = []string{"."}
        }
    }

    if pod.Spec.DNSConfig != nil {
        dnsConfig = appendDNSConfig(dnsConfig, pod.Spec.DNSConfig)
    }
    return c.formDNSConfigFitsLimits(dnsConfig, pod), nil
}

五、DNS修復(fù)

1）重新創(chuàng)建， svc的IP設(shè)置為之前的地址。

helm install coredns -n kube-system stable/coredns --set service.clusterIP=10.0.56.10

2）更換DNS地址，需要修改kubelet配置，重啟kubelet。