【发布时间】:2018-10-08 16:08:36
【问题描述】:
我在 AWS EMR 上运行一个 pyspark 集群,并将 Jupyter 用于 pyspark 驱动程序。最近,脚本失败了。 Jupiter 无法启动服务器。我使用conda install jupyter 安装并从sudo initctl start jupyter 开始。集群运行良好。这是服务器的配置值。
# jupyter configs
mkdir -p ~/.jupyter
touch ls ~/.jupyter/jupyter_notebook_config.py
HASHED_PASSWORD=$(python -c "from notebook.auth import passwd;
print(passwd('$JUPYTER_PASSWORD'))")
echo "c.NotebookApp.password = u'$HASHED_PASSWORD'" >> ~/.jupyter/jupyter_notebook_config.py
echo "c.NotebookApp.open_browser = False" >> ~/.jupyter/jupyter_notebook_config.py
echo "c.NotebookApp.ip = '*'" >> ~/.jupyter/jupyter_notebook_config.py
echo "c.NotebookApp.notebook_dir = '/mnt/$BUCKET/$FOLDER'" >> ~/.jupyter/jupyter_notebook_config.py
echo "c.ContentsManager.checkpoints_kwargs = {'root_dir': '.checkpoints'}" >> ~/.jupyter/jupyter_notebook_config.py
echo "c.NotebookApp.port = 8080" >> ~/.jupyter/jupyter_notebook_config.py
【问题讨论】:
标签: python pyspark jupyter-notebook jupyter