Docker - Celery 作为守护进程 - 找不到 pidfiles

Question

我似乎在这里尝试了所有解决方案，但似乎都没有工作，我不确定我错过了什么。我试图通过我的 docker 容器将 celery 作为守护进程运行。

root@bae5de770400:/itapp/itapp# /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd down: no pidfiles found
root@bae5de770400:/itapp/itapp# /etc/init.d/celerybeat status
celery init v10.1.
Using configuration: /etc/default/celeryd
celerybeat is down: no pid file found
root@bae5de770400:/itapp/itapp#

我看过很多关于烫发的帖子，但我都试过了，但都无济于事。

这是我创建所有权限和文件夹的 docker 文件

FROM python:latest
ENV PYTHONUNBUFFERED 1

# add source for snmp
RUN sed -i "s#jessie main#jessie main contrib non-free#g" /etc/apt/sources.list
# install dependancies
RUN apt-get update -y \
    && apt-get install -y apt-utils python-software-properties libsasl2-dev python3-dev libldap2-dev libssl-dev libsnmp-dev snmp-mibs-downloader git vim

# copy and install requirements
RUN mkdir /config  
ADD /config/requirements.txt /config/  
RUN pip install -r /config/requirements.txt  
# create folders
RUN mkdir /itapp;
RUN mkdir /static;
# create celery user
RUN useradd -N -M --system -s /bin/false celery
RUN echo celery:"*****" | /usr/sbin/chpasswd
# celery perms
RUN groupadd grp_celery
RUN usermod -a -G grp_celery celery
RUN mkdir /var/run/celery/
RUN mkdir /var/log/celery/
RUN chown root:root /var/run/celery/
RUN chown root:root /var/log/celery/
# copy celery daemon files
ADD /config/celery/init_celeryd /etc/init.d/celeryd
RUN chmod +x /etc/init.d/celeryd
ADD /config/celery/celerybeat /etc/init.d/celerybeat
RUN chmod +x /etc/init.d/celerybeat
RUN chmod 755 /etc/init.d/celeryd
RUN chown root:root  /etc/init.d/celeryd
RUN chmod 755 /etc/init.d/celerybeat
RUN chown root:root /etc/init.d/celerybeat
# copy celery config
ADD /config/celery/default_celeryd /etc/default/celeryd
# RUN /etc/init.d/celeryd start
# set workign DIR for copying code
WORKDIR /itapp

如果我手动启动它就可以了

celery -A itapp worker -l info
/usr/local/lib/python3.6/site-packages/celery/platforms.py:795: RuntimeWarning: You're running the worker with superuser privileges: this is
absolutely not recommended!

Please specify a different user using the -u option.
...

[2017-09-25 17:29:51,707: INFO/MainProcess] Connected to amqp://it-app:**@rabbitmq:5672/it-app-vhost
[2017-09-25 17:29:51,730: INFO/MainProcess] mingle: searching for neighbors
[2017-09-25 17:29:52,764: INFO/MainProcess] mingle: all alone

init.d 文件是从 celery repo 复制的，如果有帮助，这是我的默认文件的内容

# Names of nodes to start
#   most people will only start one node:
CELERYD_NODES="worker1"
#   but you can also start multiple and configure settings
#   for each in CELERYD_OPTS
#CELERYD_NODES="worker1 worker2 worker3"
#   alternatively, you can specify the number of nodes to start:
#CELERYD_NODES=10

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/usr/local/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="itapp"
# or fully qualified:

# Where to chdir at start.
CELERYD_CHDIR="/itapp/itapp/"

# Extra command-line arguments to the worker
CELERYD_OPTS="flower --time-limit=300 --concurrency=8"
# Configure node-specific settings by appending node name to arguments:
#CELERYD_OPTS="--time-limit=300 -c 8 -c:worker2 4 -c:worker3 2 -Ofair:worker1"

# Set logging level to DEBUG
#CELERYD_LOG_LEVEL="DEBUG"

# %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"

# Workers should run as an unprivileged user.
#   You need to create this user manually (or you can choose
#   a user/group combination that already exists (e.g., nobody).
CELERYD_USER="celery"
CELERYD_GROUP="celery"

# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1

我认为这个文件中唯一可能出错的是 CELERY_BIN 值，我不确定在 docker 容器中也应该设置什么

谢谢

Answer 1

所以您的 Dockerfile 中几乎没有问题

• Celery 进程外壳设置为/bin/false，它不允许启动任何进程。
• 您需要将/var/run/celery 和/var/log/celery 的权限授予celery 用户
• /etc/default/celeryd应该是640权限
• Dockerfile 中的层也太多

所以我将 Dockerfile 更新到下面

FROM python:latest
ENV PYTHONUNBUFFERED 1

# add source for snmp
RUN sed -i "s#jessie main#jessie main contrib non-free#g" /etc/apt/sources.list
# install dependancies
RUN apt-get update -y \
    && apt-get install -y apt-utils python-software-properties libsasl2-dev python3-dev libldap2-dev libssl-dev libsnmp-dev git vim

# copy and install requirements
RUN mkdir /config
ADD /config/requirements.txt /config/
RUN pip install -r /config/requirements.txt
# create folders
RUN mkdir /itapp && mkdir /static;
# create celery user
RUN useradd -N -M --system -s /bin/bash celery && echo celery:"B1llyB0n3s" | /usr/sbin/chpasswd
# celery perms
RUN groupadd grp_celery && usermod -a -G grp_celery celery && mkdir -p /var/run/celery/ /var/log/celery/
RUN chown -R celery:grp_celery /var/run/celery/ /var/log/celery/
# copy celery daemon files
ADD /config/celery/init_celeryd /etc/init.d/celeryd
RUN chmod +x /etc/init.d/celeryd
ADD /config/celery/celerybeat /etc/init.d/celerybeat
RUN chmod 750 /etc/init.d/celeryd /etc/init.d/celerybeat
RUN chown root:root  /etc/init.d/celeryd /etc/init.d/celerybeat
# copy celery config
ADD /config/celery/default_celeryd /etc/default/celeryd
RUN chmod 640 /etc/default/celeryd
# set workign DIR for copying code
ADD /itapp/ /itapp/itapp
WORKDIR /itapp

然后进入web服务容器，一切正常

root@ab658c5d0c67:/itapp/itapp# /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd down: no pidfiles found
root@ab658c5d0c67:/itapp/itapp# /etc/init.d/celeryd start
celery init v10.1.
Using config script: /etc/default/celeryd
celery multi v4.1.0 (latentcall)
> Starting nodes...
    > worker1@ab658c5d0c67: OK
    > flower@ab658c5d0c67: OK
root@ab658c5d0c67:/itapp/itapp# /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd down: no pidfiles found
root@ab658c5d0c67:/itapp/itapp# /etc/init.d/celeryd status
celery init v10.1.
Using config script: /etc/default/celeryd
celeryd (node worker1) (pid 66) is up...
root@ab658c5d0c67:/itapp/itapp#