【问题标题】:Kubernetes CoreDns livenessProbe url missing ip/hotsKubernetes CoreDns livenessProbe url 缺少 ip/hots
【发布时间】:2020-12-12 12:19:30
【问题描述】:

我在 kubernetes 上部署 coredns 发生了一些问题。

从部署描述消息中,它显示 coredns 部署启动失败,pod livenessProbe 在 http-get url 中缺少 IP/主机名。

有这个问题的详细信息。

    1. coredns的yaml文件内容如下:
# Warning: This is a file generated from the base underscore template file: coredns.yaml.base

apiVersion: v1
kind: ServiceAccount
metadata:
  name: coredns
  namespace: kube-system
  labels:
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
    addonmanager.kubernetes.io/mode: Reconcile
  name: system:coredns
rules:
- apiGroups:
  - ""
  resources:
  - endpoints
  - services
  - pods
  - namespaces
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
    addonmanager.kubernetes.io/mode: EnsureExists
  name: system:coredns
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:coredns
subjects:
- kind: ServiceAccount
  name: coredns
  namespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system
  labels:
      addonmanager.kubernetes.io/mode: EnsureExists
data:
  Corefile: |
    .:53 {
        errors
        health
        ready
        kubernetes cluster.local  in-addr.arpa ip6.arpa {
            pods insecure
            fallthrough in-addr.arpa ip6.arpa
            ttl 30
        }
        prometheus :9153
        forward . /etc/resolv.conf
        cache 30
        loop
        reload
        loadbalance
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: coredns
  namespace: kube-system
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/name: "CoreDNS"
spec:
  # replicas: not specified here:
  # 1. In order to make Addon Manager do not reconcile this replicas parameter.
  # 2. Default is 1.
  # 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  replicas: 1
  selector:
    matchLabels:
      k8s-app: kube-dns
  template:
    metadata:
      labels:
        k8s-app: kube-dns
      annotations:
        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
      priorityClassName: system-cluster-critical
      serviceAccountName: coredns
      tolerations:
        - key: "CriticalAddonsOnly"
          operator: "Exists"
      nodeSelector:
        beta.kubernetes.io/os: linux
      containers:
      - name: coredns
        image: k8s.gcr.io/coredns:1.6.2
        imagePullPolicy: IfNotPresent
        resources:
          #limits:
            #memory: $DNS_MEMORY_LIMIT
          requests:
            cpu: 100m
            memory: 70Mi
        args: [ "-conf", "/etc/coredns/Corefile" ]
        volumeMounts:
        - name: config-volume
          mountPath: /etc/coredns
          readOnly: true
        ports:
        - containerPort: 53
          name: dns
          protocol: UDP
        - containerPort: 53
          name: dns-tcp
          protocol: TCP
        - containerPort: 9153
          name: metrics
          protocol: TCP
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
        readinessProbe:
          httpGet:
            path: /ready
            port: 8181
            scheme: HTTP
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - all
          readOnlyRootFilesystem: true
      dnsPolicy: Default
      volumes:
        - name: config-volume
          configMap:
            name: coredns
            items:
            - key: Corefile
              path: Corefile
---
apiVersion: v1
kind: Service
metadata:
  name: kube-dns
  namespace: kube-system
  annotations:
    prometheus.io/port: "9153"
    prometheus.io/scrape: "true"
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/name: "CoreDNS"
spec:
  selector:
    k8s-app: kube-dns
  clusterIP: '10.0.0.1'
  # type: ClusterIP
  ports:
  - name: dns
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
    protocol: TCP
  - name: metrics
    port: 9153
    protocol: TCP

执行命令kubectl apply -f coredns.yaml后,返回follow:

[root@master CoreDns]# kubectl apply -f coredns.yaml
serviceaccount/coredns unchanged
clusterrole.rbac.authorization.k8s.io/system:coredns unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:coredns unchanged
configmap/coredns unchanged
deployment.apps/coredns unchanged
service/kube-dns unchanged

然后,我通过命令kubectl get deployments -o wide --all-namespaces 查询部署信息:

[root@master flannel]# kubectl get deployments   --all-namespaces
NAMESPACE              NAME                        READY   UP-TO-DATE   AVAILABLE   AGE
kube-system            coredns                     0/1     0            0           3h54m

这意味着部署无法正常工作。

通过命令kubectl describe deployment coredns -n kube-system*emphasized text* 获取描述部署coredns,输出消息如下:

[root@master CoreDns]# kubectl describe deployment coredns -n kube-system
Name:                   coredns
Namespace:              kube-system
CreationTimestamp:      Sat, 12 Dec 2020 03:09:21 -0500
Labels:                 addonmanager.kubernetes.io/mode=Reconcile
                        k8s-app=kube-dns
                        kubernetes.io/cluster-service=true
                        kubernetes.io/name=CoreDNS
Annotations:            Selector:  k8s-app=kube-dns
Replicas:               1 desired | 0 updated | 0 total | 0 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  1 max unavailable, 25% max surge
Pod Template:
  Labels:           k8s-app=kube-dns
  Annotations:      seccomp.security.alpha.kubernetes.io/pod: docker/default
  Service Account:  coredns
  Containers:
   coredns:
    Image:       k8s.gcr.io/coredns:1.6.2
    Ports:       53/UDP, 53/TCP, 9153/TCP
    Host Ports:  0/UDP, 0/TCP, 0/TCP
    Args:
      -conf
      /etc/coredns/Corefile
    Requests:
      cpu:        100m
      memory:     70Mi
    Liveness:     http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
    Readiness:    http-get http://:8181/ready delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /etc/coredns from config-volume (ro)
  Volumes:
   config-volume:
    Type:               ConfigMap (a volume populated by a ConfigMap)
    Name:               coredns
    Optional:           false
  Priority Class Name:  system-cluster-critical
OldReplicaSets:         <none>
NewReplicaSet:          <none>
Events:                 <none>

从上面,我认为要点是

Liveness:     http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
Readiness:    http-get http://:8181/ready delay=0s timeout=1s period=10s #success=1 #failure=3

livenessProbe'url 缺少主机/IP。 任何人都可以告诉我如何配置我的 coredns.yaml 来解决这个问题吗?谢谢!

【问题讨论】:

  • 您是否配置了集群网络?没有这个,Coredns 就不会启动。另外请添加kubectl get events --namespace=kube-system 输出。

标签: kubernetes coredns


【解决方案1】:

我终于找到了这个问题的原因。 文件/var/log/message中有很多错误日志:

Dec 13 06:55:51 localhost kubelet: E1213 19:55:51.469737   39884 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get container info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"
Dec 13 06:56:01 localhost kubelet: E1213 19:56:01.476011   39884 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/kubelet.service": failed to get cgroup stats for "/system.slice/kubelet.service": failed to get container info for "/system.slice/kubelet.service": unknown container "/system.slice/kubelet.service"

谷歌搜索此错误信息,然后大多数答案都说 docker 版本与 kubernetes 冲突。

通过执行命令行yum list installed | docker 得到版本:

[root@master kubernetes]# yum list installed | grep docker
docker.x86_64                       2:1.13.1-203.git0be3e21.el7.centos @extras
docker-client.x86_64                2:1.13.1-203.git0be3e21.el7.centos @extras
docker-common.x86_64                2:1.13.1-203.git0be3e21.el7.centos @extras

k8s 版本为v1.18.3

最后我为版本docker-19.03.9重新安装了docker,k8s工作正常。

【讨论】:

    猜你喜欢
    • 2017-11-06
    • 2019-06-25
    • 2019-05-02
    • 2021-12-22
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2019-12-05
    相关资源
    最近更新 更多