【问题标题】:Notebooks not persistent for Jupyter in Docker container?Docker 容器中的 Jupyter 笔记本不持久?
【发布时间】:2019-04-11 14:08:21
【问题描述】:

在 docker 容器中运行 jupyter 对我来说是一个很好的解决方案,但我无法让笔记本文件按照文档 here 中的宣传保持不变。

文档说,在会话关闭和服务器关闭后,.ipynb(笔记本)文件应该保存在 ./work 目录中,但对我来说不是。我已经在根目录和 Jupyter 主页中显示的 /work 目录中创建了笔记本,但是在关闭后都找不到,如果我重新启动服务器,它们也不再在目录列表中。我尝试以两种方式启动容器——首先按照文档的建议(用 latest 代替 image 标签):

docker run -p 8888:8888 jupyter/scipy-notebook:latest

其次是创建一个 docker-compose.yml 文件,该文件允许我捕获命令文本选项并避免令牌安全性(我不需要),如下所示:

version: '3'

  services: # jupyter notebook
    jupyter_notebook:
    image: jupyter/scipy-notebook
    volumes:
      - ./work:/work
    ports:
      - "8888:8888"
    command: "start.sh jupyter notebook --NotebookApp.token=''"

我在 Ubuntu 18.04.1 LTS 下运行 docker 18.06.1-ce 我希望在主机系统的 ./work 文件夹中找到笔记本(至少是我在 /work 文件夹中创建的那个),该文件夹位于我启动 docker(或 docker-compose)的目录中,但那里什么也没有。

这是一个会话记录:

   s@VC66:ls -la
   -rw-r--r-- 1 steve steve  232 Nov  7 22:45 docker-compose.yml
   drwxr-xr-x 2 steve steve 4096 Nov  7 21:34 work

   s@VC66:~/sambashare/jupyter$ cat docker-compose.yml 

   version: '3'

     services:
       jupyter_notebook:
         image: jupyter/scipy-notebook
         volumes:
           - ./work:/work
         ports:
           - "8888:8888"
         command: "start.sh jupyter notebook --NotebookApp.token=''"

   s@VC66:~/sambashare/jupyter$ docker-compose up

   Creating network "jupyter_default" with the default driver
   Creating jupyter_jupyter_notebook_1 ... done
   Attaching to jupyter_jupyter_notebook_1
   jupyter_notebook_1  | Container must be run with group "root" to update passwd file
   jupyter_notebook_1  | Executing the command: jupyter notebook --NotebookApp.token=
   jupyter_notebook_1  | [I 16:08:40.454 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret
   jupyter_notebook_1  | [W 16:08:40.597 NotebookApp] All authentication is disabled.  Anyone who can connect to this server will be able to run code.
   jupyter_notebook_1  | [I 16:08:40.625 NotebookApp] JupyterLab extension loaded from /opt/conda/lib/python3.6/site-packages/jupyterlab
   jupyter_notebook_1  | [I 16:08:40.625 NotebookApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
   jupyter_notebook_1  | [I 16:08:40.631 NotebookApp] Serving notebooks from local directory: /home/jovyan
   jupyter_notebook_1  | [I 16:08:40.631 NotebookApp] The Jupyter Notebook is running at:
   jupyter_notebook_1  | [I 16:08:40.631 NotebookApp] http://(62b087792f87 or 127.0.0.1):8888/
   jupyter_notebook_1  | [I 16:08:40.631 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
   jupyter_notebook_1  | [I 16:08:58.820 NotebookApp] 302 GET / (172.21.0.1) 0.48ms
   jupyter_notebook_1  | [I 16:09:07.941 NotebookApp] Creating new file in /work
   jupyter_notebook_1  | [I 16:09:17.360 NotebookApp] Saving file at /work/untitled.txt
   jupyter_notebook_1  | [I 16:09:24.725 NotebookApp] Shutting down on /api/shutdown request.
   jupyter_notebook_1  | [I 16:09:24.727 NotebookApp] Shutting down 0 kernels
   jupyter_jupyter_notebook_1 exited with code 0
   s@VC666:~/sambashare/jupyter$ ls work
   s@VC66:~/sambashare/jupyter$ ls
   docker-compose.yml  work

如您所见,它说它在 /work 目录中保存了“untitled.txt”,但退出时那里什么都没有。

所以为了进一步细化这里的问题,我更改了 docker-compose.yml 文件以运行一个简单的 python 脚本来在 /work 目录中创建一个文件并查看它是否仍然存在。确实如此!

command: "python3 /work/test.py"  # rather than start.sh...

这是 python test.py 脚本:

   import os
   import pytz
   from datetime import datetime

   dir = "/work"
   if not os.path.isdir(dir):
      dir = "" # to test outside docker container...

   nyc_time = datetime.now( pytz.timezone("America/New_York"))
   fname = os.path.join(dir,"test.txt")
   f = open(fname, 'w')
   f.write(f"Test time is {nyc_time}\n")
   f.close()
   exit()

这一次,docker-compose up 后,工作文件夹包含“test.txt”,其中包含

测试时间为2018-11-09 11:55:28.472581-05:00

所以安装 /work 目录的 docker 容器看起来不错——问题可能是 jupyter 映像在关闭时正在做的事情?

【问题讨论】:

  • 主机系统“~/work 文件夹”是什么意思?那不是您从主机安装的文件夹...
  • 抱歉,我一定是在输入问题时无意中添加了一些额外的字符。上面的编辑版本现已修复,可以正确阅读。
  • 不过,~/work./work 还是有区别的,除非你把 Compose 文件放在你的主目录中
  • 所以 compose 文件位于 ~/sambashare/jupyter 中,并且有一个目录 ~/sambashare/jupyter/work ,我认为这是我在容器中安装为 /work 的目录。只是为了好玩,我还在我的主目录中创建了 ~/work 以查看我们是否正在安装它。在运行相同的测试后,它也仍然是空的。顺便说一句,谢谢你帮助我!
  • 如果你执行ls ~/sambashare/jupyter并且你看到docker-compose.yml,那么你cd ~/sambashare/jupyter && docker-compose up,那么~/sambashare/jupyter/work应该被挂载到容器的/work。更好的测试方法是 (1) 在主机上的工作文件夹中创建一个文件 (2) 启动容器,查看 Jupyter 是否确实显示了数据。这样,您就不会尝试调试相反的方向

标签: python docker jupyter-notebook


【解决方案1】:

我认为您的误解是关于使用 /work 的 docker 容器。 AFAIK 是 /home/jovyan/work

因此,您可以通过例如解决您的问题。此卷映射

mkdir -P /your-jupyter/work
docker run -p 8888:8888 -e JUPYTER_ENABLE_LAB=yes -v /your-jupyter/work:home/jovyan/work jupyter/scipy-notebook

HTH。

【讨论】:

  • 非常感谢!到目前为止,我已经为此苦苦挣扎了几个小时。对我来说,我正在使用:(github.com/NVAITC/ai-lab),我应该跑:docker run --rm -p 8888:8888 -v D:/quickstart-notebooks:/home/jovyan/work -e JUPYTER_ENABLE_LAB=no nvaitc/ai-lab:19.11
  • 谢谢,这个答案,我应该如何安装一个基于此的卷?我运行一个带有卷的 pyspark,它允许我访问我的本地文件,但是一旦我关闭。我丢失了所有 ly 输出文件并在 /word repo 中工作。
  • 挂载由您自己指定的-v 参数控制。您目前使用什么作为完整的 docker 命令行?
猜你喜欢
  • 1970-01-01
  • 2020-01-14
  • 1970-01-01
  • 2020-09-23
  • 1970-01-01
  • 2021-02-16
  • 2020-09-19
  • 2020-07-08
  • 1970-01-01
相关资源
最近更新 更多