prometheus operator alertmanager-main-0 挂起并显示答案

【问题标题】：promethues operator alertmanager-main-0 pending and displayprometheus operator alertmanager-main-0 挂起并显示
【发布时间】：2019-11-05 10:12:24
【问题描述】：

发生了什么？ Kubernetes 版本：1.12 普罗米修斯操作员：release-0.1 我按照自述文件：

$ kubectl create -f manifests/

# It can take a few seconds for the above 'create manifests' command to fully create the following resources, so verify the resources are ready before proceeding.
$ until kubectl get customresourcedefinitions servicemonitors.monitoring.coreos.com ; do date; sleep 1; echo ""; done
$ until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ""; done

$ kubectl apply -f manifests/ # This command sometimes may need to be done twice (to workaround a race condition).

然后我使用命令然后显示如下：

[root@VM_8_3_centos /data/hansenwu/kube-prometheus/manifests]# kubectl get pod -n monitoring
NAME                                  READY   STATUS    RESTARTS   AGE
alertmanager-main-0                   2/2     Running   0          66s
alertmanager-main-1                   1/2     Running   0          47s
grafana-54f84fdf45-kt2j9              1/1     Running   0          72s
kube-state-metrics-65b8dbf498-h7d8g   4/4     Running   0          57s
node-exporter-7mpjw                   2/2     Running   0          72s
node-exporter-crfgv                   2/2     Running   0          72s
node-exporter-l7s9g                   2/2     Running   0          72s
node-exporter-lqpns                   2/2     Running   0          72s
prometheus-adapter-5b6f856dbc-ndfwl   1/1     Running   0          72s
prometheus-k8s-0                      3/3     Running   1          59s
prometheus-k8s-1                      3/3     Running   1          59s
prometheus-operator-5c64c8969-lqvkb   1/1     Running   0          72s
[root@VM_8_3_centos /data/hansenwu/kube-prometheus/manifests]# kubectl get pod -n monitoring
NAME                                  READY   STATUS    RESTARTS   AGE
alertmanager-main-0                   0/2     Pending   0          0s
grafana-54f84fdf45-kt2j9              1/1     Running   0          75s
kube-state-metrics-65b8dbf498-h7d8g   4/4     Running   0          60s
node-exporter-7mpjw                   2/2     Running   0          75s
node-exporter-crfgv                   2/2     Running   0          75s
node-exporter-l7s9g                   2/2     Running   0          75s
node-exporter-lqpns                   2/2     Running   0          75s
prometheus-adapter-5b6f856dbc-ndfwl   1/1     Running   0          75s
prometheus-k8s-0                      3/3     Running   1          62s
prometheus-k8s-1                      3/3     Running   1          62s
prometheus-operator-5c64c8969-lqvkb   1/1     Running   0          75s

我不知道为什么 pod altertmanager-main-0 挂起并显示然后重新启动。我看到了这个事件，它显示为：

72s         Warning   FailedCreate             StatefulSet   create Pod alertmanager-main-0 in StatefulSet alertmanager-main failed error: The POST operation against Pod could not be completed at this time, please try again.
72s         Warning   FailedCreate             StatefulSet   create Pod alertmanager-main-0 in StatefulSet alertmanager-main failed error: The POST operation against Pod could not be completed at this time, please try again.
72s         Warning^Z   FailedCreate             StatefulSet
[10]+  Stopped                 kubectl get events -n monitoring

【问题讨论】：

标签： kubernetes prometheus prometheus-operator

【解决方案1】：

警报管理器很可能没有足够的时间正确启动。

看看这个答案：https://github.com/coreos/prometheus-operator/issues/965#issuecomment-460223268

您可以将 paused 字段设置为 true，然后修改 StatefulSet 以尝试扩展 liveness/readiness 是否可以解决您的问题。

【讨论】：