Wagtail 1.8.1 迁移问题答案

【问题标题】：Wagtail 1.8.1 migration issueWagtail 1.8.1 迁移问题
【发布时间】：2017-02-21 11:09:44
【问题描述】：

尝试将 Wagtail 从 1.7 升级到 1.8.1，但在运行迁移时出现以下错误。我们正在使用 Postgres 数据库。

Applying wagtailimages.0016_deprecate_rendition_filter_relation...Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 359, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 294, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 345, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 204, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 495, in alter_field
    old_db_params, new_db_params, strict)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/postgresql/schema.py", line 117, in _alter_field
    new_db_params, strict,
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 649, in _alter_field
    params,
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 112, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: could not create unique index "wagtailimages_rendition_image_id_742f4fe4119535f1_uniq"
DETAIL:  Key (image_id, filter_id, focal_point_key)=(507, 4, 2617-1240-3472x35) is duplicated.

这是 psql 命令的输出

Table "public.wagtailimages_rendition"
     Column      |          Type          |                              Modifiers
-----------------+------------------------+-------------------------- --------------------------------------------
 id              | integer                | not null default   nextval('wagtailimages_rendition_id_seq'::regclass)
 filter_id       | integer                |
 file            | character varying(100) | not null
 width           | integer                | not null
 height          | integer                | not null
 image_id        | integer                | not null
 focal_point_key | character varying(255) | not null
 filter_spec     | character varying(255) | not null
Indexes:
    "wagtailimages_rendition_pkey" PRIMARY KEY, btree (id)
    "wagtailimages_rendition_image_id_323c8fe0_uniq" UNIQUE CONSTRAINT, btree (image_id, filter_spec, focal_point_key)
    "wagtailimages_rendition_58c64917" btree (filter_spec)
    "wagtailimages_rendition_filter_id" btree (filter_id)
    "wagtailimages_rendition_filter_spec_1cba3201_like" btree (filter_spec varchar_pattern_ops)
    "wagtailimages_rendition_image_id" btree (image_id)
Foreign-key constraints:

sql 导入的错误是 psql:foxtail.sql:68823:

ERROR:  constraint    "wagtailcore__content_type_id_c28424df_fk_django_content_type_id" for  relation "wagtailcore_page" already exists
psql:foxtail.sql:68831: ERROR:  constraint   "wagtailcore_collection_id_5423575a_fk_wagtailcore_collection_id" for   relation "wagtailcore_groupcollectionpermission" already exists
ALTER TABLE
psql:foxtail.sql:68847: ERROR:  constraint   "wagtailcore_groupc_permission_id_1b626275_fk_auth_permission_id" for   relation "wagtailcore_groupcollectionpermission" already exists
psql:foxtail.sql:68855: ERROR:  constraint   "wagtailcore_groupcollectionp_group_id_05d61460_fk_auth_group_id" for   relation "wagtailcore_groupcollectionpermission" already exists
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
psql:foxtail.sql:68903: ERROR:  constraint  "wagtaildocs_collection_id_23881625_fk_wagtailcore_collection_id" for  relation "wagtaildocs_document" already exists
psql:foxtail.sql:68911: ERROR:  constraint  "wagtaildocs_docume_uploaded_by_user_id_17258b41_fk_auth_user_id" for   relation "wagtaildocs_document" already exists
ALTER TABLE
psql:foxtail.sql:68927: ERROR:  constraint   "wagtailimag_collection_id_c2f8af7e_fk_wagtailcore_collection_id" for    relation "wagtailimages_image" already exists

全新导入后但运行迁移前的输出

Table "public.wagtailimages_rendition"
Column      |          Type          |                                 Modifiers
-----------------+------------------------+--------------------------   --------------------------------------------
id              | integer                | not null default   nextval('wagtailimages_rendition_id_seq'::regclass)
filter_id       | integer                | not null
file            | character varying(100) | not null
width           | integer                | not null
height          | integer                | not null
image_id        | integer                | not null
focal_point_key | character varying(255) | not null
Indexes:
"wagtailimages_rendition_pkey" PRIMARY KEY, btree (id)
"wagtailimages_rendition_image_id_742f4fe4119535f1_uniq" UNIQUE  CONSTRAINT, btree (image_id, filter_id, focal_point_key)
"wagtailimages_rendition_filter_id" btree (filter_id)
"wagtailimages_rendition_image_id" btree (image_id)
Foreign-key constraints:
"filter_id_refs_id_6909da8c" FOREIGN KEY (filter_id) REFERENCES   wagtailimages_filter(id) DEFERRABLE INITIALLY DEFERRED
"image_id_refs_id_e221c01d" FOREIGN KEY (image_id) REFERENCES   wagtailimages_image(id) DEFERRA:

【问题讨论】：

我对那个错误有点困惑，因为有问题的迁移应该是删除有问题的唯一索引，而不是创建它...请你在 Postgresql 命令行上运行\d wagtailimages_rendition并将输出粘贴到这里，这样我就可以看到数据库表当前处于什么状态？
我们正在使用 docker 进行本地开发，当我在删除容器后再次尝试时，迁移运行正常。当我尝试导入一些 sql 以使用一些数据填充数据库时，我收到错误，我在管理员中拥有的只是默认欢迎您的 wagtail 网站页面。虽然它已经导入了用户等。似乎如果已经有数据迁移失败。
看起来您的 SQL 导入文件包含一些架构更改/创建命令，这些命令肯定不能很好地用于迁移，因为它依赖于从一个特定的初始数据库状态开始。我猜它打算从一个空数据库开始运行 - 这是否完成而没有错误？如果是这样，如果在此之后运行 Wagtail 1.8.1 迁移会发生什么？
从头开始导入没有错误，但随后发生迁移错误
请您在导入运行后但在运行迁移之前显示\d wagtailimages_rendition 的输出？（我假设上面显示的输出是在新数据库上成功运行迁移的结果，对吧？）

标签： django wagtail

【解决方案1】：

由于我无法理解上述情况是如何产生的，我只能建议“核选项”：通过运行删除 wagtailimages_rendition 表的内容

DELETE FROM wagtailimages_rendition;

在 Postgresql 命令行上。这应该允许剩余的迁移完成而不会出现错误。这不会导致持久的数据丢失，因为 Wagtail 会在需要时重新生成再现 - 尽管页面请求会在一段时间内变慢，因为它必须重新渲染缺少再现条目的每个图像。

更新：与其删除所有内容，更保守的做法是查找并删除重复记录：

DELETE FROM wagtailimages_rendition
WHERE image_id || '-' || filter_id || '-' || focal_point_key IN (
    SELECT image_id || '-' || filter_id || '-' || focal_point_key
    FROM (
        SELECT image_id, filter_id, focal_point_key, COUNT(*) AS count
        FROM wagtailimages_rendition GROUP BY image_id, filter_id, focal_point_key
    ) AS renditions
    WHERE count > 1
);

【讨论】：

在生产中，我们不确定是否对我们的渲染表进行核对，不仅仅是因为性能，还因为外部系统调用 wagtail 来生成在 django 模板之外使用的集合大小的图像，它们永远不会以与页面上使用的图像相同的方式再次调用。（如果这有意义吗？）我发现了 Karl 在此问题中发布的这段 SQL，github.com/wagtail/wagtail/issues/685，它删除了重复的再现。我已经在本地尝试过，它允许迁移继续进行。你认为这会是一个选择吗？谢谢乔斯
@joss 是的，应该可以。我认为您可能需要修改 Karl 的 SQL 以将 focal_point_key 包含在唯一性标准中，以使其与当前架构保持同步 - 我已更新我的答案以包含（未经测试的）固定 SQL。
我测试了更新后的查询，但是它没有带回要删除的结果。这是因为副本有一个 focus_point_key 值，在所有情况下似乎都是相同的，除了最后一个值比另一个值小一个，例如“4010-1722-4208x4230”和“4010-1722-4208x4231”
在错误中显示 DETAIL: Key (image_id, filter_id, focus_point_key)=(788, 4, 3231-1430-5472x28) is duplicated。当我带回图像 788 的所有再现时，这三个记录具有像 "3231-1430-5472x2874" 这样的焦点，所以它切断了最后两个字符，并认为我们有重复
github.com/wagtail/wagtail/blob/… 此迁移似乎为 focus_point_key 设置了 max_length=16，这与最后两个被忽略有关

【解决方案2】：

我可以确认乔斯在这边看到了什么。

我猜出于某种原因，我们的 focus_point_key 值比其他人长，否则每个人在迁移时都会看到这个问题。

这可能是因为您过去为我们完成了一些定制工作吗？

不知道现在的解决方案是什么，任何帮助，非常感谢。

【讨论】：