如何将 Peewee 与多处理一起使用，而不会出现“命令不同步；您现在无法运行此命令”异常？

Question

我编写了一个大型多进程应用程序并使用 Peewee 作为 ORM，但是当多个进程尝试同时访问数据库时，我收到了同步错误。如何在不更改应用程序结构的情况下解决此问题？

导致问题的短代码：

from multiprocessing import Process
from time import sleep
from abc import ABC
from playhouse.shortcuts import *


class MySqlAutoReconnectDatabase(ReconnectMixin, MySQLDatabase, ABC):

    def __init__(self):
        super().__init__(
            database='test',
            autoconnect=True,
            **{
                'host': '127.0.0.1',
                'port': 3306,
                'charset': 'utf8',
                'sql_mode': 'PIPES_AS_CONCAT',
                'use_unicode': True,
                'user': 'root',
                'password': ''
            })


database = MySqlAutoReconnectDatabase()


class Entity(Model):
    info = CharField(null=True)

    class Meta:
        database = database
        table_name = 'entity'


def some_proc(id):
    while (1):
        entity = list(Entity.select().where(Entity.id == id))
        sleep(0.5)


entities = Entity.select()
for entity in entities:
    print(entity.info)
    process = Process(target=some_proc, args=(entity.id,))
    process.start()

我想避免的异常：

Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/root/PycharmProjects/test/test.py", line 37, in some_proc
    entity = list(Entity.select().where(Entity.id == id))
  File "/usr/local/lib/python3.7/dist-packages/peewee.py", line 1978, in __len__
    self._ensure_execution()
  File "/usr/local/lib/python3.7/dist-packages/peewee.py", line 1957, in _ensure_execution
    if not self._cursor_wrapper:
  File "/usr/local/lib/python3.7/dist-packages/peewee.py", line 4223, in __len__
    self.fill_cache()
  File "/usr/local/lib/python3.7/dist-packages/peewee.py", line 4264, in fill_cache
    iterator.next()
  File "/usr/local/lib/python3.7/dist-packages/peewee.py", line 4320, in next
    self.cursor_wrapper.iterate()
  File "/usr/local/lib/python3.7/dist-packages/peewee.py", line 4233, in iterate
    self.cursor.close()
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 84, in close
    while self.nextset():
  File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 172, in nextset
    nr = db.next_result()
_mysql_exceptions.ProgrammingError: (2014, "Commands out of sync; you can't run this command now")

数据库：

CREATE DATABASE IF NOT EXISTS `test` DEFAULT CHARACTER SET utf8 ;
USE `test` ;

CREATE TABLE IF NOT EXISTS `test`.`entity` (
  `id` INT NOT NULL AUTO_INCREMENT,
  `info` VARCHAR(2048) NULL,
  PRIMARY KEY (`id`))
ENGINE = InnoDB;

INSERT INTO `entity` (`id`, `info`) VALUES (NULL, 'test1');
INSERT INTO `entity` (`id`, `info`) VALUES (NULL, 'test2');
INSERT INTO `entity` (`id`, `info`) VALUES (NULL, 'test3');
INSERT INTO `entity` (`id`, `info`) VALUES (NULL, 'test4');
INSERT INTO `entity` (`id`, `info`) VALUES (NULL, 'test5');
INSERT INTO `entity` (`id`, `info`) VALUES (NULL, 'test6');

Answer 1

在分叉新进程之前，您正在创建数据库连接（带有关联的套接字/文件描述符）。这些被转移到子进程中，显然这不起作用。

很容易解决。只需等待打开数据库连接在分叉新进程之前。