为什么在 Pod 中启动 daskdev/dask 会失败？答案

【问题标题】：Why does starting daskdev/dask into a Pod fail?为什么在 Pod 中启动 daskdev/dask 会失败？
【发布时间】：2021-04-01 16:13:14
【问题描述】：

为什么kubectl run dask --image daskdev/dask 会失败？

# starting the container with docker to make sure it basically works
➜  ~ docker run --rm -it --entrypoint bash daskdev/dask:latest
(base) root@5b34ce038eb3:/# python
Python 3.8.0 (default, Nov  6 2019, 21:49:08) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dask
>>> 
>>> exit()
(base) root@5b34ce038eb3:/# exit
exit

# now trying to fire up the container on a minikube cluster
➜  ~ kubectl run dask --image daskdev/dask  
pod/dask created

# let's see what's going on with the Pod
➜  ~ kubectl get pods -w
NAME                              READY   STATUS             RESTARTS   AGE
dask                              0/1     CrashLoopBackOff   1          13s
dask                              0/1     Completed          2          24s
dask                              0/1     CrashLoopBackOff   2          38s

# not sure why the logs look like something is missing
➜  ~ kubectl logs dask --tail=100
+ '[' '' ']'
+ '[' -e /opt/app/environment.yml ']'
+ echo 'no environment.yml'
+ '[' '' ']'
+ '[' '' ']'
+ exec
no environment.yml

【问题讨论】：

您似乎正试图在 kubernetes 中运行交互式应用程序，它（本机）不理解交互式应用程序。如果 Pod 成功运行，您希望 Pod 做什么？它会等待您的输入，它会运行您拥有的一些预定义工作吗？其他？
我最初的意图是在 K8s 环境中使用该容器 - 熟悉它。所以，我实际上不希望 Pod 做任何事情。下一步是登录并手动调用调度程序/工作人员。另一方面，我是 K8s 的新手，并认为这个问题也是一个学习机会——毕竟我还不明白为什么我的方法会失败。也许@Vitalli 的回答会启发我。
在这种情况下，您需要在 PodSpec 中设置 command: [sleep, infinity]（或使用您的 kubectl run），以便 pod 启动、配置自身，然后就永远坐在那里等着您到kubectl exec 进去；我看到有些人也使用tail -f /dev/null 来实现相同的阻塞过程目的

标签： kubernetes dask dask-kubernetes

【解决方案1】：

所以基本上，如果您检查kubectl describe pod dask 的结果，您会看到最后一个状态是Terminated，退出代码为 0，这实际上意味着您的容器已成功启动，是否正常工作并成功完成。您还期望 pod 会发生什么？此外，当您使用 kubectl run dask --image daskdev/dask 创建 pod 时，它默认使用 restartPolicy: Always 创建！！！！

总是意味着容器将被重新启动，即使它以零退出代码退出（即成功）。

    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 02 Apr 2021 15:06:00 +0000
      Finished:     Fri, 02 Apr 2021 15:06:00 +0000
    Ready:          False
    Restart Count:  3
    Environment:    <none>

您的容器中没有/opt/app/environment.yml。如果我没记错的话，你应该先用prepare.sh 配置它。请检查更多here - DASK 部分

#docker run --rm -it --entrypoint bash daskdev/dask:latest
(base) root@431d69bb9a80:/# ls -la /opt/app/
total 12
drwxr-xr-x 2 root root 4096 Mar 27 15:43 .
drwxr-xr-x 1 root root 4096 Mar 27 15:43 ..

不确定为什么日志看起来好像缺少某些东西➜ ~ kubectl logs dask --tail=100 ... 执行无环境.yml

已经准备好了helm DASK chart。用它。它工作正常：

helm repo add dask https://helm.dask.org/
helm repo update
helm install raffael-dask-release dask/dask

NAME: raffael-dask-release
LAST DEPLOYED: Fri Apr  2 15:43:38 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing DASK, released at name: raffael-dask-release.

This release includes a Dask scheduler, 3 Dask workers, and 1 Jupyter servers.

The Jupyter notebook server and Dask scheduler expose external services to
which you can connect to manage notebooks, or connect directly to the Dask
cluster. You can get these addresses by running the following:

  export DASK_SCHEDULER="127.0.0.1"
  export DASK_SCHEDULER_UI_IP="127.0.0.1"
  export DASK_SCHEDULER_PORT=8080
  export DASK_SCHEDULER_UI_PORT=8081
  kubectl port-forward --namespace default svc/raffael-dask-release-scheduler $DASK_SCHEDULER_PORT:8786 &
  kubectl port-forward --namespace default svc/raffael-dask-release-scheduler $DASK_SCHEDULER_UI_PORT:80 &

  export JUPYTER_NOTEBOOK_IP="127.0.0.1"
  export JUPYTER_NOTEBOOK_PORT=8082
  kubectl port-forward --namespace default svc/raffael-dask-release-jupyter $JUPYTER_NOTEBOOK_PORT:80 &

  echo tcp://$DASK_SCHEDULER:$DASK_SCHEDULER_PORT               -- Dask Client connection
  echo http://$DASK_SCHEDULER_UI_IP:$DASK_SCHEDULER_UI_PORT     -- Dask dashboard
  echo http://$JUPYTER_NOTEBOOK_IP:$JUPYTER_NOTEBOOK_PORT       -- Jupyter notebook

NOTE: It may take a few minutes for the LoadBalancer IP to be available. Until then, the commands above will not work for the LoadBalancer service type.
You can watch the status by running 'kubectl get svc --namespace default -w raffael-dask-release-scheduler'

NOTE: It may take a few minutes for the URLs above to be available if any EXTRA_PIP_PACKAGES or EXTRA_CONDA_PACKAGES were specified,
because they are installed before their respective services start.

NOTE: The default password to login to the notebook server is `dask`. To change this password, refer to the Jupyter password section in values.yaml, or in the README.md.

如果你还想手动创建pod，使用下面...主要思想设置restartPolicy: Never。

apiVersion: v1
kind: Pod
metadata:
  name: dask-tesssssst
  labels:
    foo: bar
spec:
  restartPolicy: Never
  containers:
  - image: daskdev/dask:latest
    imagePullPolicy: Always
    name: dask-tesssssst

请查看DASK KubeCluster official documentation for more examples. 我从那里拍摄的最后一张。

【讨论】：