centos7.1.1503上启动Kubernetes API Server失败答案

【问题标题】：Failed to start Kubernetes API Server on centos7.1.1503centos7.1.1503上启动Kubernetes API Server失败
【发布时间】：2017-05-27 19:02:41
【问题描述】：

以下部分是：错误、配置和 kubernetes 版本和 etcd 版本。

[root@xt3 kubernetes]# for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do  systemctl restart $SERVICES; systemctl enable $SERVICES; systemctl status $SERVICES ; done
etcd.service - Etcd Server
   Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled)
   Active: active (running) since Fri 2016-03-25 11:11:25 CST; 58ms ago
 Main PID: 6382 (etcd)
   CGroup: /system.slice/etcd.service
           й╕йд6382 /usr/bin/etcd

Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: data dir = /var/lib/etcd/default.etcd
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: member dir = /var/lib/etcd/default.etcd/member
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: heartbeat = 100ms
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: election = 1000ms
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: snapshot count = 10000
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: advertise client URLs = http://localhost:2379,http://localhost:4001
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: loaded cluster information from store: default=http://localhost:2380,default=http://localhost:7001
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: restart member ce2a822cea30bfca in cluster 7e27652122e8b2ae at commit index 10686
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 raft: ce2a822cea30bfca became follower at term 8
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 raft: newRaft ce2a822cea30bfca [peers: [ce2a822cea30bfca], term: 8, commit: 10686, applied: 10001, lastindex: 10686, lastterm: 8]
Job for kube-apiserver.service failed. See 'systemctl status kube-apiserver.service' and 'journalctl -xn' for details.
kube-apiserver.service - Kubernetes API Server
   Loaded: loaded (/usr/lib/systemd/system/kube-apiserver.service; enabled)
   Active: activating (auto-restart) (Result: exit-code) since Fri 2016-03-25 11:11:35 CST; 58ms ago
     Docs: https://github.com/GoogleCloudPlatform/kubernetes
 Main PID: 6401 (code=exited, status=255)

Mar 25 11:11:35 xt3 systemd[1]: Failed to start Kubernetes API Server.
Mar 25 11:11:35 xt3 systemd[1]: Unit kube-apiserver.service entered failed state.
kube-controller-manager.service - Kubernetes Controller Manager
   Loaded: loaded (/usr/lib/systemd/system/kube-controller-manager.service; enabled)
   Active: active (running) since Fri 2016-03-25 11:11:35 CST; 73ms ago
     Docs: https://github.com/GoogleCloudPlatform/kubernetes
 Main PID: 6437 (kube-controller)
   CGroup: /system.slice/kube-controller-manager.service
           й╕йд6437 /usr/bin/kube-controller-manager --logtostderr=true --v=0 --master=http://127.0.0.1:8080

Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.954951    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.ReplicationController: Get http://127.0.0.1:8080/api/v1/replicationcontrollers: dia... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955075    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.PersistentVolume: Get http://127.0.0.1:8080/api/v1/persistentvolumes: dial tcp 127.... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955159    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods: dial tcp 127.0.0.1:8080: getsockopt: connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955222    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.PersistentVolume: Get http://127.0.0.1:8080/api/v1/persistentvolumes: dial tcp 127.... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955248    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: ge... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955331    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.PersistentVolumeClaim: Get http://127.0.0.1:8080/api/v1/persistentvolumeclaims: dia... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955379    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: ge... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955430    6437 resource_quota_controller.go:62] Synchronization error: Get http://127.0.0.1:8080/api/v1/resourcequotas: dial tcp 127.0.0.1:8080: getsockopt: connection refused (&url....or)(0xc8204f2000)})
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955576    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods: dial tcp 127.0.0.1:8080: getsockopt: connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955670    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.ServiceAccount: Get http://127.0.0.1:8080/api/v1/serviceaccounts?fieldSelector=meta... connection refused
Hint: Some lines were ellipsized, use -l to show in full.
kube-scheduler.service - Kubernetes Scheduler Plugin
   Loaded: loaded (/usr/lib/systemd/system/kube-scheduler.service; enabled)
   Active: active (running) since Fri 2016-03-25 11:11:36 CST; 71ms ago
     Docs: https://github.com/GoogleCloudPlatform/kubernetes
 Main PID: 6466 (kube-scheduler)
   CGroup: /system.slice/kube-scheduler.service
           й╕йд6466 /usr/bin/kube-scheduler --logtostderr=true --v=0 --master=http://127.0.0.1:8080

Mar 25 11:11:36 xt3 systemd[1]: Started Kubernetes Scheduler Plugin.
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031318    6466 reflector.go:180] pkg/scheduler/factory/factory.go:194: Failed to list *api.ReplicationController: Get http://127.0.0.1:8080/api/v1/replicationcontrollers: dial tcp 127.0.0.1:...: connection refused
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031421    6466 reflector.go:180] pkg/scheduler/factory/factory.go:189: Failed to list *api.Service: Get http://127.0.0.1:8080/api/v1/services: dial tcp 127.0.0.1:8080: getsockopt: connection refused
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031564    6466 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%21%3D: dial tcp 127....: connection refused
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031644    6466 reflector.go:180] pkg/scheduler/factory/factory.go:184: Failed to list *api.Node: Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse: dial tcp 127...: connection refused
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031677    6466 reflector.go:180] pkg/scheduler/factory/factory.go:177: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%3D: dial tcp 127.0.0.1:8080:...: connection refused
Hint: Some lines were ellipsized, use -l to show in full.
[root@xt3 kubernetes]# 
[root@xt3 kubernetes]#

错误详情如下。

[root@xt3 kubernetes]# journalctl -xn
    -- Logs begin at Sat 2016-03-19 15:30:07 CST, end at Fri 2016-03-25 11:11:42 CST. --
    Mar 25 11:11:41 xt3 kube-controller-manager[6437]: E0325 11:11:41.958470    6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.ServiceAccount: Get http://127.0.0.1:8080/api/v1/serviceaccounts?fieldSelector=metadata.name%3Ddefault: d
    Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034315    6466 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%21%3D: dial tcp 127.0.0.1:8080: getsockopt:
    Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034325    6466 reflector.go:180] pkg/scheduler/factory/factory.go:184: Failed to list *api.Node: Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse: dial tcp 127.0.0.1:8080: getsockopt
    Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034324    6466 reflector.go:180] pkg/scheduler/factory/factory.go:189: Failed to list *api.Service: Get http://127.0.0.1:8080/api/v1/services: dial tcp 127.0.0.1:8080: getsockopt: connection refused
    Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034413    6466 reflector.go:180] pkg/scheduler/factory/factory.go:194: Failed to list *api.ReplicationController: Get http://127.0.0.1:8080/api/v1/replicationcontrollers: dial tcp 127.0.0.1:8080: getsockopt: conne
    Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034434    6466 reflector.go:180] pkg/scheduler/factory/factory.go:177: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%3D: dial tcp 127.0.0.1:8080: getsockopt: connection
    Mar 25 11:11:42 xt3 kube-apiserver[6487]: E0325 11:11:42.206743    6487 reflector.go:180] pkg/admission/namespace/lifecycle/admission.go:95: Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: getsockopt: connection refus
    Mar 25 11:11:42 xt3 kube-apiserver[6487]: E0325 11:11:42.206767    6487 reflector.go:180] pkg/admission/limitranger/admission.go:102: Failed to list *api.LimitRange: Get http://127.0.0.1:8080/api/v1/limitranges: dial tcp 127.0.0.1:8080: getsockopt: connection refused
    Mar 25 11:11:42 xt3 kube-apiserver[6487]: E0325 11:11:42.206816    6487 reflector.go:180] pkg/admission/namespace/exists/admission.go:89: Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: getsockopt: connection refused
    Mar 25 11:11:42 xt3 kube-apiserver[6487]: E0325 11:11:42.206831    6487 reflector.go:180] pkg/admission/resourcequota/admission.go:59: Failed to list *api.ResourceQuota: Get http://127.0.0.1:8080/api/v1/resourcequotas: dial tcp 127.0.0.1:8080: getsockopt: connection ref
    [root@xt3 kubernetes]#

配置如下：

[root@xt3 kubernetes]# pwd
    /etc/kubernetes
    [root@xt3 kubernetes]# cat config
    ###
    # kubernetes system config
    #
    # The following values are used to configure various aspects of all
    # kubernetes services, including
    #
    #   kube-apiserver.service
    #   kube-controller-manager.service
    #   kube-scheduler.service
    #   kubelet.service
    #   kube-proxy.service
    # logging to stderr means we get it in the systemd journal
    KUBE_LOGTOSTDERR="--logtostderr=true"

    # journal message level, 0 is debug
    KUBE_LOG_LEVEL="--v=0"

    # Should this cluster be allowed to run privileged docker containers
    KUBE_ALLOW_PRIV="--allow-privileged=false"

    # How the controller-manager, scheduler, and proxy find the apiserver
    KUBE_MASTER="--master=http://127.0.0.1:8080"
    [root@xt3 kubernetes]# 
    [root@xt3 kubernetes]# 
    [root@xt3 kubernetes]# cat apiserver
    ###
    # kubernetes system config
    #
    # The following values are used to configure the kube-apiserver
    #

    # The address on the local server to listen to.
    KUBE_API_ADDRESS="--insecure-bind-address=127.0.0.1"

    # The port on the local server to listen on.
    # KUBE_API_PORT="--port=8080"

    # Port minions listen on
    # KUBELET_PORT="--kubelet-port=10250"

    # Comma separated list of nodes in the etcd cluster
    KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379"

    # Address range to use for services
    KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"

    # default admission control policies
    KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

    # Add your own!
    KUBE_API_ARGS=""
    [root@xt3 kubernetes]# 
    [root@xt3 kubernetes]# 
    [root@xt3 kubernetes]# ls
    apiserver  apiserver.rpmsave  config  config.rpmsave  controller-manager  kubelet  proxy  scheduler
    [root@xt3 kubernetes]# cat controller-manager 
    ###
    # The following values are used to configure the kubernetes controller-manager

    # defaults from config and apiserver should be adequate

    # Add your own!
    KUBE_CONTROLLER_MANAGER_ARGS=""
    [root@xt3 kubernetes]# 
    [root@xt3 kubernetes]# cat kubelet 
    ###
    # kubernetes kubelet (minion) config

    # The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
    KUBELET_ADDRESS="--address=127.0.0.1"

    # The port for the info server to serve on
    # KUBELET_PORT="--port=10250"

    # You may leave this blank to use the actual hostname
    KUBELET_HOSTNAME="--hostname-override=127.0.0.1"

    # location of the api-server
    KUBELET_API_SERVER="--api-servers=http://127.0.0.1:8080"

    # Add your own!
    KUBELET_ARGS=""
    [root@xt3 kubernetes]# 
    [root@xt3 kubernetes]# cat proxy 
    ###
    # kubernetes proxy config

    # default config should be adequate

    # Add your own!
    KUBE_PROXY_ARGS=""
    [root@xt3 kubernetes]# 
    [root@xt3 kubernetes]# 
    [root@xt3 kubernetes]# 
    [root@xt3 kubernetes]# cat scheduler 
    ###
    # kubernetes scheduler config

    # default config should be adequate

    # Add your own!
    KUBE_SCHEDULER_ARGS=""

kubernetes 和 etcd 的版本：

[root@xt3 kubernetes]# rpm -qa | grep kuber
kubernetes-node-1.1.0-0.4.git2bfa9a1.el7.x86_64

我按照 kubernetes 网站的要求进行所有配置。(http://kubernetes.io/docs/getting-started-guides/centos/centos_manual_config/)

    kubernetes-client-1.1.0-0.4.git2bfa9a1.el7.x86_64
    kubernetes-1.1.0-0.4.git2bfa9a1.el7.x86_64
    kubernetes-master-1.1.0-0.4.git2bfa9a1.el7.x86_64
    [root@xt3 kubernetes]# rpm -qa | grep etcd
    etcd-2.0.9-1.el7.x86_64

我期待回复答案。请联系我。非常感谢。

【问题讨论】：

标签： kubernetes

【解决方案1】：

您提供的日志还不够。您可以通过使用查看所有详细信息

tail -n 1000 /var/log/messages

在我的情况下，我先停止 kube-apiserver 服务然后启动它，然后在 /var/log/messages 中搜索。我的错误是文件被删除，因为我重新启动机器。它可能没有是你的理由，但你可以使用我说的命令找到它。

祝你好运。

【讨论】：

【解决方案2】：

我有一个非常相似的问题，在我的情况下，我修复了它，将 KUBE_API_PORT= 更改为我系统上可用的另一个，例如 KUBE_API_PORT=9090

尝试curl 127.0.0.1:8080 确定该端口是否正在被其他服务使用

【讨论】：

【解决方案3】：

运行以下命令应该可以使其正常工作，因为 kubelet 可能会陷入崩溃循环：

sudo systemctl stop kubelet
sudo rm -rf /var/lib/kubelet/pods/*
sudo systemctl start kubelet

【讨论】：