【问题标题】:Running mlflow as a systemd service - gunicorn not found将 mlflow 作为系统服务运行 - 未找到 gunicorn
【发布时间】:2021-03-10 03:03:17
【问题描述】:

我正在尝试运行安装在 virtualenv 内的 mlflow 跟踪服务器作为 Ubuntu 20.04 上的 systemd 服务,但我收到一条错误消息,表明它无法找到 gunicorn。这是我的日记

nov 27 10:37:17 Atrium-Power mlflow[81375]: Traceback (most recent call last):
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/bin/mlflow", line 8, in <module>
nov 27 10:37:17 Atrium-Power mlflow[81375]:     sys.exit(cli())
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/site-packages/click/core.py", line 829, in __call__
nov 27 10:37:17 Atrium-Power mlflow[81375]:     return self.main(*args, **kwargs)
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/site-packages/click/core.py", line 782, in main
nov 27 10:37:17 Atrium-Power mlflow[81375]:     rv = self.invoke(ctx)
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
nov 27 10:37:17 Atrium-Power mlflow[81375]:     return _process_result(sub_ctx.command.invoke(sub_ctx))
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
nov 27 10:37:17 Atrium-Power mlflow[81375]:     return ctx.invoke(self.callback, **ctx.params)
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/site-packages/click/core.py", line 610, in invoke
nov 27 10:37:17 Atrium-Power mlflow[81375]:     return callback(*args, **kwargs)
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/site-packages/mlflow/cli.py", line 392, in server
nov 27 10:37:17 Atrium-Power mlflow[81375]:     _run_server(
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/site-packages/mlflow/server/__init__.py", line 138, in _run_server
nov 27 10:37:17 Atrium-Power mlflow[81375]:     exec_cmd(full_command, env=env_map, stream_output=True)
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/site-packages/mlflow/utils/process.py", line 34, in exec_cmd
nov 27 10:37:17 Atrium-Power mlflow[81375]:     child = subprocess.Popen(
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/subprocess.py", line 947, in __init__
nov 27 10:37:17 Atrium-Power mlflow[81375]:     self._execute_child(args, executable, preexec_fn, close_fds,
nov 27 10:37:17 Atrium-Power mlflow[81375]:   File "/home/praxasense/.miniconda3/envs/mlflow-server/lib/python3.9/subprocess.py", line 1819, in _execute_child
nov 27 10:37:17 Atrium-Power mlflow[81375]:     raise child_exception_type(errno_num, err_msg, err_filename)
nov 27 10:37:17 Atrium-Power mlflow[81375]: FileNotFoundError: [Errno 2] No such file or directory: 'gunicorn'

我的 systemd 是这样的:

[Unit]
StartLimitBurst=5
StartLimitIntervalSec=33

[Service]
User=praxasense
WorkingDirectory=/home/praxasense
Restart=always
RestartSec=5
ExecStart=/home/praxasense/.miniconda3/envs/mlflow-server/bin/mlflow server --port 3569 --backend-store-uri .mlruns

[Install]
WantedBy=multi-user.target

奇怪的是,如果我在终端中从ExecStart 运行命令,它在fish shell 中可以正常工作,但在bash 中却不行,但是如果我先执行conda activate mlflow-server 然后执行@ 987654325@它确实工作。据我了解,Python 解释器应该知道它是虚拟环境,因此它应该可以在我尝试时工作,但显然我遗漏了一些使其无法找到 gunicon 包的东西,这显然是存在的。

有什么想法吗?

【问题讨论】:

    标签: python systemd ubuntu-20.04 mlflow


    【解决方案1】:

    尝试将 venv 的 bin 路径添加到 systemd 运行的环境中:

    [Service]
    ...
    Environment="PATH=/home/praxasense/.miniconda3/envs/mlflow-server/bin"
    ...
    

    我还建议设置 KillMode=mixed,因为 MLFlow 将生成 gunicorn 实例,如果您以其他方式终止服务,这些实例不会终止。 mixed 表示子进程也将被终止。

    【讨论】:

    • 很好用,应该认为它与搜索路径有关,搜索路径是 virtualenv 工作的关键部分。也感谢您提供的提示!
    猜你喜欢
    • 2018-07-06
    • 2017-02-09
    • 1970-01-01
    • 2021-06-17
    • 2011-01-29
    • 1970-01-01
    • 2018-12-09
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多