处于 CrashLoopBackOff 状态的 Kubernetes 复制控制器答案

【问题标题】：Kubernetes replication controller in CrashLoopBackOff state处于 CrashLoopBackOff 状态的 Kubernetes 复制控制器
【发布时间】：2016-04-17 23:12:54
【问题描述】：

我执行了以下步骤。

使用以下配置文件创建了复制控制器：

{
   "kind":"ReplicationController",
   "apiVersion":"v1",
   "metadata":{
      "name":"fsharp-service",
      "labels":{
         "app":"fsharp-service"
      }
   },
   "spec":{
      "replicas":1,
      "selector":{
         "app":"fsharp-service"
      },
      "template":{
         "metadata":{
            "labels":{
               "app":"fsharp-service"
            }
         },
         "spec":{
            "containers":[
               {
                  "name":"fsharp-service",
                  "image":"fsharp/fsharp:latest",
                  "ports":[
                     {
                        "name":"http-server",
                        "containerPort":3000
                     }
                  ]
               }
            ]
         }
      }
   }
}

运行命令：

kubectl create -f fsharp-controller.json

这是输出：

$ kubectl get rc
CONTROLLER       CONTAINER(S)     IMAGE(S)                             SELECTOR             REPLICAS
cassandra        cassandra        gcr.io/google-samples/cassandra:v8   app=cassandra        3
fsharp-service   fsharp-service   fsharp/fsharp:latest                 app=fsharp-service   1
$ kubectl get pods
NAME                   READY     REASON    RESTARTS   AGE
cassandra              1/1       Running   0          28m
cassandra-ch1br        1/1       Running   0          28m
cassandra-xog49        1/1       Running   0          27m
fsharp-service-7lrq8   0/1       Error     2          31s
$ kubectl logs fsharp-service-7lrq8

F# Interactive for F# 4.0 (Open Source Edition)
Freely distributed under the Apache 2.0 Open Source License

For help type #help;;

$ kubectl get pods
NAME                   READY     REASON             RESTARTS   AGE
cassandra              1/1       Running            0          28m
cassandra-ch1br        1/1       Running            0          28m
cassandra-xog49        1/1       Running            0          28m
fsharp-service-7lrq8   0/1       CrashLoopBackOff   3          1m
$ kubectl describe po fsharp-service-7lrq8
W0417 15:52:36.288492   11461 request.go:302] field selector: v1 - events - involvedObject.name - fsharp-service-7lrq8: need to check if this is versioned correctly.
W0417 15:52:36.289196   11461 request.go:302] field selector: v1 - events - involvedObject.namespace - default: need to check if this is versioned correctly.
W0417 15:52:36.289204   11461 request.go:302] field selector: v1 - events - involvedObject.uid - d4dab099-04ee-11e6-b7f9-0a11c670939b: need to check if this is versioned correctly.
Name:               fsharp-service-7lrq8
Image(s):           fsharp/fsharp:latest
Node:               ip-172-20-0-228.us-west-2.compute.internal/172.20.0.228
Labels:             app=fsharp-service
Status:             Running
Replication Controllers:    fsharp-service (1/1 replicas created)
Containers:
  fsharp-service:
    Image:      fsharp/fsharp:latest
    State:      Waiting
      Reason:       CrashLoopBackOff
    Ready:      False
    Restart Count:  3
Conditions:
  Type      Status
  Ready     False
Events:
  FirstSeen             LastSeen            Count   From                            SubobjectPath           Reason      Message
  Sun, 17 Apr 2016 15:50:50 -0700   Sun, 17 Apr 2016 15:50:50 -0700 1   {default-scheduler }                                    Scheduled   Successfully assigned fsharp-service-7lrq8 to ip-172-20-0-228.us-west-2.compute.internal
  Sun, 17 Apr 2016 15:50:51 -0700   Sun, 17 Apr 2016 15:50:51 -0700 1   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Created     Created container with docker id d44c288ea67b
  Sun, 17 Apr 2016 15:50:51 -0700   Sun, 17 Apr 2016 15:50:51 -0700 1   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Started     Started container with docker id d44c288ea67b
  Sun, 17 Apr 2016 15:50:55 -0700   Sun, 17 Apr 2016 15:50:55 -0700 1   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Started     Started container with docker id 688a3ed122d2
  Sun, 17 Apr 2016 15:50:55 -0700   Sun, 17 Apr 2016 15:50:55 -0700 1   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Created     Created container with docker id 688a3ed122d2
  Sun, 17 Apr 2016 15:50:58 -0700   Sun, 17 Apr 2016 15:50:58 -0700 1   {kubelet ip-172-20-0-228.us-west-2.compute.internal}                    FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "fsharp-service" with CrashLoopBackOff: "Back-off 10s restarting failed container=fsharp-service pod=fsharp-service-7lrq8_default(d4dab099-04ee-11e6-b7f9-0a11c670939b)"

  Sun, 17 Apr 2016 15:51:15 -0700   Sun, 17 Apr 2016 15:51:15 -0700 1   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Started     Started container with docker id c2e348e1722d
  Sun, 17 Apr 2016 15:51:15 -0700   Sun, 17 Apr 2016 15:51:15 -0700 1   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Created     Created container with docker id c2e348e1722d
  Sun, 17 Apr 2016 15:51:17 -0700   Sun, 17 Apr 2016 15:51:31 -0700 2   {kubelet ip-172-20-0-228.us-west-2.compute.internal}                    FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "fsharp-service" with CrashLoopBackOff: "Back-off 20s restarting failed container=fsharp-service pod=fsharp-service-7lrq8_default(d4dab099-04ee-11e6-b7f9-0a11c670939b)"

  Sun, 17 Apr 2016 15:50:50 -0700   Sun, 17 Apr 2016 15:51:44 -0700 4   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Pulling     pulling image "fsharp/fsharp:latest"
  Sun, 17 Apr 2016 15:51:45 -0700   Sun, 17 Apr 2016 15:51:45 -0700 1   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Created     Created container with docker id edaea97fb379
  Sun, 17 Apr 2016 15:50:51 -0700   Sun, 17 Apr 2016 15:51:45 -0700 4   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Pulled      Successfully pulled image "fsharp/fsharp:latest"
  Sun, 17 Apr 2016 15:51:46 -0700   Sun, 17 Apr 2016 15:51:46 -0700 1   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} Started     Started container with docker id edaea97fb379
  Sun, 17 Apr 2016 15:50:58 -0700   Sun, 17 Apr 2016 15:52:27 -0700 7   {kubelet ip-172-20-0-228.us-west-2.compute.internal}    spec.containers{fsharp-service} BackOff     Back-off restarting failed docker container
  Sun, 17 Apr 2016 15:51:48 -0700   Sun, 17 Apr 2016 15:52:27 -0700 4   {kubelet ip-172-20-0-228.us-west-2.compute.internal}                    FailedSync  Error syncing pod, skipping: failed to "StartContainer" for "fsharp-service" with CrashLoopBackOff: "Back-off 40s restarting failed container=fsharp-service pod=fsharp-service-7lrq8_default(d4dab099-04ee-11e6-b7f9-0a11c670939b)"

怎么了？

如何找出控制器无法正常启动的原因？

更新。

我尝试将简单的“fsharp/fsharp:latest”图像更改为另一个图像，其中会有服务监听端口，这就是我想要使用容器的方式。

图像名为“username/someservice:mytag”，并有一个服务监听 3000 端口。

我将服务运行为：

mono Service.exe

当我查看日志时，我看到：

$ kubectl logs -p fsharp-service-wjmpv
Running on http://127.0.0.1:3000
Press enter to exit

所以即使进程不应该退出，容器也处于相同的状态：

$ kubectl get pods
NAME                   READY     REASON             RESTARTS   AGE
fsharp-service-wjmpv   0/1       CrashLoopBackOff   9          25m

我还尝试使用 -i 标志从我的镜像运行容器，以使容器不退出，但 kubectl 似乎无法识别 -i 标志：\

有什么想法吗？

【问题讨论】：

标签： amazon-web-services docker kubernetes kubernetes-health-check

【解决方案1】：

您正在启动一个立即退出的容器。 kubelet 注意到，重新启动它，然后再次退出。在这种情况发生几次之后，kubelet 会减慢它尝试启动容器的速度（这是 CrashLoopBackOff 状态）。

fsharp documentation 表示使用-i 标志运行容器，这会提供交互式提示。如果你只是这样做

docker run fsharp/fsharp:latest

您会注意到容器立即退出并将您转储回本地 shell。这是您尝试调用集群中容器的方式，它同样会立即退出。

【讨论】：

我已经更新了我的问题。添加了我尝试使用另一个图像的结果（我运行侦听端口 3000 的服务的图像，基于 fsharp 图像）

【解决方案2】：

我会使用kubectl logs 来尝试找出您的容器发生了什么情况，如下所示：

kubectl logs -p fsharp-service-7lrq8

-p 标志可让您获取上次启动的日志，在这种情况下这是必要的，因为容器正在崩溃。

【讨论】：

谢谢，但是日志是空的，只是一个F#交互提示信息
你想要达到什么目的？从您添加的端口来看，您打算运行某种 Web 服务，但对我来说，容器中似乎不存在这样的 Web 服务，因为您只是在执行 vanilla F# 容器。
我对 F# 一无所知，所以请原谅我的无知。根据@RobertBaily 的说法，您的形象似乎只是启动交互式提示然后退出，除非以交互模式启动。为什么要启动一个在 Kubernetes 中什么都不做的容器？如果你真的这样做，我建议你以无限睡眠开始你的容器。例如。在您的容器定义中添加与此类似的内容：command: [ "bin/bash", "-c", "sleep", "10000d" ]
你说得对，这是我的 hello world 尝试，所以我只是尝试运行任何容器。在现实生活中，我将拥有一个基于 fsharp 图像和 Web 服务的自定义图像，我必须将必要的参数传递给该图像。这个特定的容器退出了，但我想检查它是否有效。我将以交互方式尝试，然后使用我的自定义图像
@TobiasGustafsson 要求您宣传您对答案的评论。这个帮助解决了我的问题。

【解决方案3】：

我在我的 F# 服务（Unix 特定代码）中添加了以下行，以确保进程不会退出：

let signals = [| new UnixSignal (Signum.SIGINT); 
                 new UnixSignal (Signum.SIGTERM); 
                 new UnixSignal (Signum.SIGQUIT)
              |]

let which = UnixSignal.WaitAny (signals, -1);

之后我的复制控制器运行正常。

【讨论】：