【发布时间】：2016-11-16 05:38:34
【问题描述】：
我正在通过 Docker 在本地运行 Ubuntu（可信）上的 Kubernetes 集群。
由于我使用 Vagrant 创建 Ubuntu VM，我不得不稍微修改一下官方 Kubernetes 指南中的 docker run 命令：
docker run -d \
--volume=/:/rootfs:ro \
--volume=/sys:/sys:ro \
--volume=/var/lib/docker/:/var/lib/docker:rw \
--volume=/var/lib/kubelet/:/var/lib/kubelet:rw \
--volume=/var/run:/var/run:rw \
--net=host \
--privileged=true \
--pid=host \
gcr.io/google_containers/hyperkube:v1.3.0 \
/hyperkube kubelet \
--allow-privileged=true \
--api-servers=http://localhost:8080 \
--v=2 \
--address=0.0.0.0 \
--enable-server \
--hostname-override=192.168.10.30 \
--config=/etc/kubernetes/manifests-multi \
--containerized \
--cluster-dns=10.0.0.10 \
--cluster-domain=cluster.local
此外，运行反向代理允许我通过浏览器从 VM 外部访问集群的服务：
docker run -d --net=host --privileged gcr.io/google_containers/hyperkube:v1.3.0 \
/hyperkube proxy --master=http://127.0.0.1:8080 --v=2
这些步骤运行良好，最终我可以在浏览器中访问 Kubernetes UI。
vagrant@trusty-vm:~$ kubectl cluster-info
Kubernetes master is running at http://localhost:8080
KubeDNS is running at http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
现在我想在带有 InfluxDB 后端和 Grafana UI 的 Kubernetes 集群中运行 Heapster，正如 this guide 中所述。为此，我克隆了 Heapster 存储库并通过添加 type: NodePort 将 grafana-service.yaml 配置为使用外部 IP：
apiVersion: v1
kind: Service
metadata:
labels:
kubernetes.io/cluster-service: 'true'
kubernetes.io/name: monitoring-grafana
name: monitoring-grafana
namespace: kube-system
spec:
# In a production setup, we recommend accessing Grafana through an external Loadbalancer
# or through a public IP.
type: NodePort
ports:
- port: 80
targetPort: 3000
selector:
name: influxGrafana
创建服务、rcs 等：
vagrant@trusty-vm:~/heapster$ kubectl create -f deploy/kube-config/influxdb/
You have exposed your service on an external port on all nodes in your
cluster. If you want to expose this service to the external internet, you may
need to set up firewall rules for the service port(s) (tcp:30593) to serve traffic.
See http://releases.k8s.io/release-1.3/docs/user-guide/services-firewalls.md for more details.
service "monitoring-grafana" created
replicationcontroller "heapster" created
service "heapster" created
replicationcontroller "influxdb-grafana" created
service "monitoring-influxdb" created
vagrant@trusty-vm:~/heapster$ kubectl cluster-info
Kubernetes master is running at http://localhost:8080
Heapster is running at http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/heapster
KubeDNS is running at http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
monitoring-grafana is running at http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana
vagrant@trusty-vm:~/heapster$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system heapster-y2yci 1/1 Running 0 32m
kube-system influxdb-grafana-6udas 2/2 Running 0 32m
kube-system k8s-master-192.168.10.30 4/4 Running 0 58m
kube-system k8s-proxy-192.168.10.30 1/1 Running 0 58m
kube-system kube-addon-manager-192.168.10.30 2/2 Running 0 57m
kube-system kube-dns-v17-y4cwh 3/3 Running 0 58m
kube-system kubernetes-dashboard-v1.1.0-bnbnp 1/1 Running 0 58m
vagrant@trusty-vm:~/heapster$ kubectl get svc --all-namespaces
NAMESPACE NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes 10.0.0.1 <none> 443/TCP 18m
kube-system heapster 10.0.0.234 <none> 80/TCP 3s
kube-system kube-dns 10.0.0.10 <none> 53/UDP,53/TCP 18m
kube-system kubernetes-dashboard 10.0.0.58 <none> 80/TCP 18m
kube-system monitoring-grafana 10.0.0.132 <nodes> 80/TCP 3s
kube-system monitoring-influxdb 10.0.0.197 <none> 8083/TCP,8086/TCP 16m
如您所见，一切似乎运行顺利，我还可以通过浏览器访问 Grafana 的 UI，地址为 http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana/。
但是，大约 1 分钟后，Heapster 和 Grafana 端点都从 kubectl cluster-info 中消失了。
vagrant@trusty-vm:~/heapster$ kubectl cluster-info
Kubernetes master is running at http://localhost:8080
KubeDNS is running at http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns
kubernetes-dashboard is running at http://localhost:8080/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
浏览器输出：
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "endpoints \"monitoring-grafana\" not found",
"reason": "NotFound",
"details": {
"name": "monitoring-grafana",
"kind": "endpoints"
},
"code": 404
}
Pod 仍在运行中......
vagrant@trusty-vm:~/heapster$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system heapster-y2yci 1/1 Running 0 32m
kube-system influxdb-grafana-6udas 2/2 Running 0 32m
kube-system k8s-master-192.168.10.30 4/4 Running 0 58m
kube-system k8s-proxy-192.168.10.30 1/1 Running 0 58m
kube-system kube-addon-manager-192.168.10.30 2/2 Running 0 57m
kube-system kube-dns-v17-y4cwh 3/3 Running 0 58m
kube-system kubernetes-dashboard-v1.1.0-bnbnp 1/1 Running 0 58m
...但是 Heapster 和 Grafana 服务已经消失：
vagrant@trusty-vm:~/heapster$ kubectl get svc --all-namespaces
NAMESPACE NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes 10.0.0.1 <none> 443/TCP 19m
kube-system kube-dns 10.0.0.10 <none> 53/UDP,53/TCP 19m
kube-system kubernetes-dashboard 10.0.0.58 <none> 80/TCP 19m
kube-system monitoring-influxdb 10.0.0.197 <none> 8083/TCP,8086/TCP 17m
在检查kubectl cluster-info dump 的输出时，我发现了以下错误：
I0713 09:31:09.088567 1 proxier.go:427] Adding new service "kube-system/monitoring-grafana:" at 10.0.0.227:80/TCP
E0713 09:31:09.273385 1 proxier.go:887] can't open "nodePort for kube-system/monitoring-grafana:" (:30593/tcp), skipping this nodePort: listen tcp :30593: bind: address alread$
I0713 09:31:09.395280 1 proxier.go:427] Adding new service "kube-system/heapster:" at 10.0.0.111:80/TCP
E0713 09:31:09.466306 1 proxier.go:887] can't open "nodePort for kube-system/monitoring-grafana:" (:30593/tcp), skipping this nodePort: listen tcp :30593: bind: address alread$
I0713 09:31:09.480468 1 proxier.go:502] Setting endpoints for "kube-system/monitoring-grafana:" to [172.17.0.5:3000]
E0713 09:31:09.519698 1 proxier.go:887] can't open "nodePort for kube-system/monitoring-grafana:" (:30593/tcp), skipping this nodePort: listen tcp :30593: bind: address alread$
I0713 09:31:09.532026 1 proxier.go:502] Setting endpoints for "kube-system/heapster:" to [172.17.0.4:8082]
E0713 09:31:09.558527 1 proxier.go:887] can't open "nodePort for kube-system/monitoring-grafana:" (:30593/tcp), skipping this nodePort: listen tcp :30593: bind: address alread$
E0713 09:31:17.249001 1 server.go:294] Starting health server failed: listen tcp 127.0.0.1:10249: bind: address already in use
E0713 09:31:22.252280 1 server.go:294] Starting health server failed: listen tcp 127.0.0.1:10249: bind: address already in use
E0713 09:31:27.257895 1 server.go:294] Starting health server failed: listen tcp 127.0.0.1:10249: bind: address already in use
E0713 09:31:31.126035 1 proxier.go:887] can't open "nodePort for kube-system/monitoring-grafana:" (:30593/tcp), skipping this nodePort: listen tcp :30593: bind: address alread$
E0713 09:31:32.264430 1 server.go:294] Starting health server failed: E0709 09:32:01.153168 1 proxier.go:887] can't open "nodePort for kube-system/monitoring-grafana:" ($
E0713 09:31:37.265109 1 server.go:294] Starting health server failed: listen tcp 127.0.0.1:10249: bind: address already in use
E0713 09:31:42.269035 1 server.go:294] Starting health server failed: listen tcp 127.0.0.1:10249: bind: address already in use
E0713 09:31:47.270950 1 server.go:294] Starting health server failed: listen tcp 127.0.0.1:10249: bind: address already in use
E0713 09:31:52.272354 1 server.go:294] Starting health server failed: listen tcp 127.0.0.1:10249: bind: address already in use
E0713 09:31:57.273424 1 server.go:294] Starting health server failed: listen tcp 127.0.0.1:10249: bind: address already in use
E0713 09:32:01.153168 1 proxier.go:887] can't open "nodePort for kube-system/monitoring-grafana:" (:30593/tcp), skipping this nodePort: listen tcp :30593: bind: address alread$
E0713 09:32:02.276318 1 server.go:294] Starting health server failed: listen tcp 127.0.0.1:10249: bind: address already in use
I0713 09:32:06.105878 1 proxier.go:447] Removing service "kube-system/monitoring-grafana:"
I0713 09:32:07.175025 1 proxier.go:447] Removing service "kube-system/heapster:"
I0713 09:32:07.210270 1 proxier.go:517] Removing endpoints for "kube-system/monitoring-grafana:"
I0713 09:32:07.249824 1 proxier.go:517] Removing endpoints for "kube-system/heapster:"
显然，由于 nodePort 已在使用中，Heapster 和 Grafana 的服务和端点已被删除。我没有在grafana-service.yaml 中指定指定的nodePort，这意味着Kubernetes 可以选择一个尚未使用的——那么这怎么会是错误呢？另外，有没有办法解决这个问题？
操作系统：Ubuntu 14.04.4 LTS（值得信赖）| Kubernetes：v1.3.0 |码头工人：v1.11.2
【问题讨论】：
标签： docker kubernetes influxdb grafana heapster