Uvicorn + Gunicorn + Starlette 服务时卡住，没有sigkill无法重启服务答案

【问题标题】：Uvicorn + Gunicorn + Starlette is getting stuck when serving, can't restart the service without sigkillUvicorn + Gunicorn + Starlette 服务时卡住，没有sigkill无法重启服务
【发布时间】：2021-03-12 03:41:45
【问题描述】：

我正在通过 gunicorn + uvicorn 在 VM 上提供模型。

由 supervisord 自动启动，运行api.sh。

api.sh 包含：

source /home/asd/.virtual_envs/myproject/bin/activate

/home/asd/.virtual_envs/myproject/bin/gunicorn --max-requests-jitter 30 -w 6 -b 0.0.0.0:4080 api:app -k uvicorn.workers.UvicornWorker

不用过多介绍api.py，它包含以下主要部分：

from starlette.applications import Starlette
from models import SomeModelClass


app = Starlette(debug=False)
model = SomeModelClass()


@app.route('/do_things', methods=['GET', 'POST', 'HEAD'])
async def add_styles(request):
    if request.method == 'GET':
        params = request.query_params
    elif request.method == 'POST':
        params = await request.json()
    elif request.method == 'HEAD':
        return UJSONResponse([])

    # Doing things
    result = model(params)
    return UJSONResponse(result)

在 api 运行几天后我开始收到这些错误：

[INFO] Starting gunicorn 20.0.3
[ERROR] Connection in use: ('0.0.0.0', 4080)
[ERROR] Retrying in 1 second.
[ERROR] Connection in use: ('0.0.0.0', 4080)
[ERROR] Retrying in 1 second.
[ERROR] Connection in use: ('0.0.0.0', 4080)
[ERROR] Retrying in 1 second.
[ERROR] Connection in use: ('0.0.0.0', 4080)
[ERROR] Retrying in 1 second.
...

在 supervisord 中重新启动 api 没有任何作用，我收到与上面相同的消息。我发现可行的唯一方法是：

在 supervisord 中停止 api
查看哪个 pid 在 4080 端口上运行（python3.8 进程）：sudo netstat -tulpn | grep LISTEN
杀死它运行kill -9 [PID]
重复步骤 2-3 1-2 次，直到没有任何东西占用 4080 端口
在 supervisord 中启动 api

你有什么办法解决这个问题吗？

【问题讨论】：

标签： python gunicorn supervisord uvicorn starlette

【解决方案1】：

代码实际使用了multiprocessing 中的Pool，这很可能是导致此问题的原因。

例子：

from starlette.applications import Starlette
from models import SomeModelClass
from multiprocessing import Pool
from utils import myfun


app = Starlette(debug=False)
model = SomeModelClass()


@app.route('/do_things', methods=['GET', 'POST', 'HEAD'])
async def add_styles(request):
    if request.method == 'GET':
        params = request.query_params
    elif request.method == 'POST':
        params = await request.json()
    elif request.method == 'HEAD':
        return UJSONResponse([])

    # Doing things
    result = model(params)
    # Start of the offending code
    pool = Pool(4)
    result = pool.map(myfun, result, chunksize=1)
    # End of the offending code
    return UJSONResponse(result)

解决方案是将multiprocessing替换为concurrency：

from starlette.applications import Starlette
from models import SomeModelClass
import concurrent.futures
from utils import myfun


app = Starlette(debug=False)
model = SomeModelClass()


@app.route('/do_things', methods=['GET', 'POST', 'HEAD'])
async def add_styles(request):
    if request.method == 'GET':
        params = request.query_params
    elif request.method == 'POST':
        params = await request.json()
    elif request.method == 'HEAD':
        return UJSONResponse([])

    # Doing things
    result = model(params)
    # Start of the fix
    with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
        result = executor.map(myfun, result)
    result = list(result)
    # End of the fix
    return UJSONResponse(result)

【讨论】：