具有相同外键的对象的类自动增量字段（Django 1.8，MySQL 5.5）

Question

我有一个 Django 模型（我们称之为 ObjectLog），其中许多模型通过外键与单个 Object 相关联。简要定义见下文：

class ObjectLog(models.Model):
    class Meta:
        ordering = ['-created','-N']
        unique_together = ("object","N")
    object = models.ForeignKey(Object, null=False)                                                                                                                                                              
    created = models.DateTimeField(auto_now_add=True)
    issuer = models.ForeignKey(User)
    N = models.IntegerField(null=False)

与单个 Object 相关的每个 ObjectLog 都应具有唯一的 N 值（如unique_together 要求所示）。另一种说法是，N 本质上应该是一个自动增量字段，但仅相对于为单个对象设置的 ObjectLog。使用自定义保存方法从逻辑上完成这并不是一件难事：

def save(self, *args, **kwargs):                                                                                                                                                                        
    with transaction.atomic():
        logs = self.object.objectlog_set.select_for_update().order_by('-N')
        if logs:
            self.N = logs[0].N + 1
        else:
            self.N = 1
        super(ObjectLog, self).save(*args, **kwargs)

但是，我在处理并发方面遇到了麻烦。当多个进程同时尝试为单个 Object 创建 ObjectLogs 时，它们通常以相同的 N 值结束，从而导致诸如“Duplicate entry \'249244-169\' for key”之类的错误。我试图通过在我的save 方法中使用transaction.atomic 和select_for_update 来解决这个问题，尽管我现在意识到原子性并不是真正可以帮助我的属性。看起来我需要某种方法来锁定与相关对象相关的 ObjectLog 表中的行，而代码确定 N 应该是什么并保存新行，但我不知道如何做到这一点。

我正在使用带有 MyISAM 引擎的 Django 1.8 和 MySQL 5.5。我还尝试修改有问题的表以使用 InnoDB。使用 InnoDB 时，我收到类似“1213，\'尝试获取锁定时发现死锁；尝试重新启动事务\'”之类的错误。所以似乎锁定在这种情况下是有效的，但也许我做得过火了？一个讨厌的解决方案是捕获这些错误并使用 while 循环来强制保存方法重试，但我真的不想这样做。

非常感谢任何建议！抱歉，如果我滥用了一些术语，我对这个场景还很陌生。如果我遗漏了一些重要的信息，我很乐意提供更多信息。

Answer 1

我最终通过构建和执行原始 SQL 查询以在单个数据库事务中执行自动增量和插入来解决问题。我花了很多时间研究 Django 源代码来了解他们的默认模型保存方法是如何工作的，以便我可以尽可能稳健地做到这一点。但是，我完全希望这需要针对非 MySQL 后端进行修改。

首先，我创建了一个抽象类，ObjectLog 现在将从中派生该抽象类，它具有这种新的保存方法：

class AutoIncrementModel(models.Model):
    """
    An abstract class used as a base for classes which need the
    autoincrementing save method described below.
    """
    class Meta:
        abstract = True

    def save(self, auto_field, auto_fk, *args, **kwargs):
        """
        Arguments:
            auto_field: name of field which acts as an autoincrement field.
            auto_fk:    name of ForeignKey to which the auto_field is relative.
        """

        # Do normal save if this is not an insert (i.e., the instance has a
        # primary key already).
        meta = self.__class__._meta
        pk_set = self._get_pk_val(meta) is not None
        if pk_set:
            super(ObjectLog, self).save(*args, **kwargs)
            return

        # Otherwise, we'll generate some raw SQL to do the
        # insert and auto-increment.

        # Get model fields, except for primary key field.
        fields = meta.local_concrete_fields
        if not pk_set:
            fields = [f for f in fields if not
                isinstance(f, models.fields.AutoField)]

        # Setup for generating base SQL query for doing an INSERT.
        query = models.sql.InsertQuery(self.__class__._base_manager.model)
        query.insert_values(fields, objs=[self])
        compiler = query.get_compiler(using=self.__class__._base_manager.db)
        compiler.return_id = meta.has_auto_field and not pk_set

        fk_name = meta.get_field(auto_fk).column
        with compiler.connection.cursor() as cursor:
            # Get base SQL query as string.
            for sql, params in compiler.as_sql():
                # compiler.as_sql() looks like:
                # INSERT INTO `table_objectlog` VALUES (%s,...,%s)
                # We modify this to do:
                # INSERT INTO `table_objectlog` SELECT %s,...,%s FROM
                # `table_objectlog` WHERE `object_id`=id
                # NOTE: it's unlikely that the following will generate
                # a functional database query for non-MySQL backends.

                # Replace VALUES (%s, %s, ..., %s) with
                # SELECT %s, %s, ..., %s
                sql = re.sub(r"VALUES \((.*)\)", r"SELECT \1", sql)

                # Add table to SELECT from and ForeignKey id corresponding to
                # our autoincrement field.
                sql += " FROM `{tbl_name}` WHERE `{fk_name}`={fk_id}".format(
                    tbl_name=meta.db_table,
                    fk_name=fk_name,
                    fk_id=getattr(self, fk_name)
                    )

                # Get index corresponding to auto_field.
                af_idx = [f.name for f in fields].index(auto_field)
                # Put this directly in the SQL. If we use parameter
                # substitution with cursor.execute, it gets quoted
                # as a literal, which causes the SQL command to fail.
                # We shouldn't have issues with SQL injection because
                # auto_field should never be a user-defined parameter.
                del params[af_idx]
                sql = re.sub(r"((%s, ){{{0}}})%s".format(af_idx),
                r"\1IFNULL(MAX({af}),0)+1", sql, 1).format(af=auto_field)

                # IFNULL(MAX({af}),0)+1 is the autoincrement SQL command,
                # {af} is substituted as the column name.

                # Execute SQL command.
                cursor.execute(sql, params)

            # Get primary key from database and set it in memory.
            if compiler.connection.features.can_return_id_from_insert:
                id = compiler.connection.ops.fetch_returned_insert_id(cursor)
            else:
                id = compiler.connection.ops.last_insert_id(cursor,
                    meta.db_table, meta.pk.column)
            self._set_pk_val(id)

            # Refresh object in memory in order to get auto_field value.
            self.refresh_from_db()

然后 ObjectLog 模型使用如下：

class ObjectLog(AutoIncrementModel):
    class Meta:
        ordering = ['-created','-N']
        unique_together = ("object","N")
    object = models.ForeignKey(Object, null=False)                                                                                                                                                              
    created = models.DateTimeField(auto_now_add=True)
    issuer = models.ForeignKey(User)
    N = models.IntegerField(null=False)

    def save(self, *args, **kwargs):
        # Set up to call save method of the base class (AutoIncrementModel)
        kwargs.update({'auto_field': 'N', 'auto_fk': 'event'})
        super(EventLog, self).save(*args, **kwargs)

这允许对 ObjectLog.save() 的调用仍按预期工作。