Heroku postgres postgis - django 发布失败：关系 \"spatial_ref_sys\" 不存在

Question

Heroku 于 2022 年 8 月 1 日更改了他们的 PostgreSQL 扩展架构管理。(https://devcenter.heroku.com/changelog-items/2446)

从那时起，我们现有的 django 4.0 应用程序的每次部署到 Heroku 都在发布阶段失败，构建成功。

有没有人遇到过同样的问题？ 除了重新安装 postgis 扩展之外，是否有将新版本推送到 Heroku 的解决方法？

如果我正确理解更改，Heroku 为新创建的扩展添加了一个名为“heroku_ext”的模式。由于在我们的案例中存在扩展，因此它不应该受到影响。

所有当前安装的扩展将继续按预期工作。

通过 git push 跟踪发布的完整日志：

git push staging develop:master
Gesamt 0 (Delta 0), Wiederverwendet 0 (Delta 0), Pack wiederverwendet 0
remote: Compressing source files... done.
remote: Building source:
remote: 
remote: -----> Building on the Heroku-20 stack
remote: -----> Using buildpacks:
remote:        1. https://github.com/heroku/heroku-geo-buildpack.git
remote:        2. heroku/python
remote: -----> Geo Packages (GDAL/GEOS/PROJ) app detected
remote: -----> Installing GDAL-2.4.0
remote: -----> Installing GEOS-3.7.2
remote: -----> Installing PROJ-5.2.0
remote: -----> Python app detected
remote: -----> Using Python version specified in runtime.txt
remote: -----> No change in requirements detected, installing from cache
remote: -----> Using cached install of python-3.9.13
remote: -----> Installing pip 22.1.2, setuptools 60.10.0 and wheel 0.37.1
remote: -----> Installing SQLite3
remote: -----> Installing requirements with pip
remote: -----> Skipping Django collectstatic since the env var DISABLE_COLLECTSTATIC is set.
remote: -----> Discovering process types
remote:        Procfile declares types -> release, web, worker
remote: 
remote: -----> Compressing...
remote:        Done: 156.1M
remote: -----> Launching...
remote:  !     Release command declared: this new release will not be available until the command succeeds.
remote:        Released v123
remote:        https://myherokuapp.herokuapp.com/ deployed to Heroku
remote: 
remote: This app is using the Heroku-20 stack, however a newer stack is available.
remote: To upgrade to Heroku-22, see:
remote: https://devcenter.heroku.com/articles/upgrading-to-the-latest-stack
remote: 
remote: Verifying deploy... done.
remote: Running release command...
remote: 
remote: Traceback (most recent call last):
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/utils.py", line 87, in _execute
remote:     return self.cursor.execute(sql)
remote: psycopg2.errors.UndefinedTable: relation "spatial_ref_sys" does not exist
remote: 
remote: 
remote: The above exception was the direct cause of the following exception:
remote: 
remote: Traceback (most recent call last):
remote:   File "/app/manage.py", line 22, in <module>
remote:     main()
remote:   File "/app/manage.py", line 18, in main
remote:     execute_from_command_line(sys.argv)
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
remote:     utility.execute()
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/core/management/__init__.py", line 440, in execute
remote:     self.fetch_command(subcommand).run_from_argv(self.argv)
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/core/management/base.py", line 414, in run_from_argv
remote:     self.execute(*args, **cmd_options)
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/core/management/base.py", line 460, in execute
remote:     output = self.handle(*args, **options)
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/core/management/base.py", line 98, in wrapped
remote:     res = handle_func(*args, **kwargs)
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/core/management/commands/migrate.py", line 106, in handle
remote:     connection.prepare_database()
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 26, in prepare_database
remote:     cursor.execute("CREATE EXTENSION IF NOT EXISTS postgis")
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/sentry_sdk/integrations/django/__init__.py", line 544, in execute
remote:     return real_execute(self, sql, params)
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/utils.py", line 67, in execute
remote:     return self._execute_with_wrappers(
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
remote:     return executor(sql, params, many, context)
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
remote:     return self.cursor.execute(sql, params)
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/db/utils.py", line 91, in __exit__
remote:     raise dj_exc_value.with_traceback(traceback) from exc_value
remote:   File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/utils.py", line 87, in _execute
remote:     return self.cursor.execute(sql)
remote: django.db.utils.ProgrammingError: relation "spatial_ref_sys" does not exist
remote: 
remote: Sentry is attempting to send 2 pending error messages
remote: Waiting up to 2 seconds
remote: Press Ctrl-C to quit
remote: Waiting for release.... failed.
To https://git.heroku.com/myherokuapp

Answer 1

这是我通过pg:backups:restore 为我们使用数据库备份的审查应用程序提出的解决方法（如果您正在操作生产数据库，可能需要启用维护）：

1. 在本地复制您的评论应用程序数据库（最近通过pg:backups:restore 恢复的数据库，以便您获得所有数据）：heroku pg:pull [Database URL] localdb -a [app-name]

2. 将您的应用程序数据库配置设置为使用 localdb 然后连接到 psql 并执行：ALTER EXTENSION "hstore" SET SCHEMA heroku_ext;。为所有现有的扩展运行此命令。

   要列出您已提取的所有可用扩展，请运行 dx。您不必更改 plpgsql 它是 PostgreSQL 原生的

   这将起作用，因为您在本地拥有所有权限。

   3. 将此版本推送回您的评论应用程序：heroku pg:push mylocaldb [Database URL] -a [app-name] ==> 您的数据库需要为空才能执行此操作。您可以在数据为空的新评论应用上尝试此操作。这样它就可以成为你所有评论应用的新基地

   4. 确保一切按预期工作（数据正确恢复）。然后你可以做的是通过pg:backups:capture 生成一个新的数据库转储，并将其用作所有新评论应用程序的新数据库备份。

   来源：https://devcenter.heroku.com/articles/managing-heroku-postgres-using-cli

   我也必须这样做，因为例如删除 hstore 扩展并重新启用它对于我们的案例来说不是一个可行的选择。

Answer 2

我已经通过覆盖 postgis/base.py 引擎来解决它，我将以下内容放在我的应用程序中 db/base.py

from django.contrib.gis.db.backends.postgis.base import (
     DatabaseWrapper as PostGISDatabaseWrapper,
)

class DatabaseWrapper(PostGISDatabaseWrapper):
    def prepare_database(self):
        # This is the overwrite - we don't want to call the
        # super() because of a faulty extension creation
     pass

然后在我的设置中，我刚刚指出了DATABASES["engine"] = "app.db"

它对备份没有帮助，但至少我可以再次发布。

Answer 3

我们在尝试使用 heroku pg:backups:restore 和 heroku pg:copy 时也遇到了这个问题 - 如果现有备份快照包含已安装的扩展，则无法恢复它们。

Answer 4

我的团队也遇到了这种情况。等不及 Heroku 修复了，所以我们做了一些粗略的手术。不会适用于所有人，但由于我们对 PostGIS 的使用非常少，所以还不错。

这是我对每个受影响数据库的清单：

• 备份数据库
• 开启维护模式
• 在所有使用geography 类型存储原始纬度/经度数据的表上创建新的原始float 列
• 将值从geography 字段写入新列
• 删除 postgis 扩展：DROP EXTENSION postgis CASCADE;
• 重新创建扩展：CREATE EXTENSION IF NOT EXISTS postgis;
• 重新创建我们的geography 字段并从新列中填充它们
• 删除新的临时列
• 重新创建适当的索引

到目前为止，一切都很好。

Answer 5

我们遵循@chedli https://stackoverflow.com/a/73219273/840568 提供的解决方法

但是在我们的例子中，postgis 在尝试ALTER EXTENSION postgis SET SCHEMA heroku_ext 解决方法时给出了不允许重定位模式的错误，所以我们最终不得不执行这个额外的步骤。

UPDATE pg_extension
  SET extrelocatable = true
    WHERE extname = 'postgis';

ALTER EXTENSION "postgis" SET SCHEMA "heroku_ext";

UPDATE pg_extension
  SET extrelocatable = false
    WHERE extname = 'postgis';

Answer 6

我正在跟踪一个非常相似的问题。我一直与heroku保持联系。我之前的错误提到了 pgaudit 扩展，现在我看到了和你一样的错误。如果我知道解决方案，我会在这里更新。

System check identified no issues (5 silenced).
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.10/site-packages/django/db/backends/utils.py", line 87, in _execute
return self.cursor.execute(sql)
psycopg2.errors.UndefinedTable: relation "spatial_ref_sys" does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/manage.py", line 40, in
main()
File "/app/manage.py", line 36, in main
execute_from_command_line(sys.argv)
File "/app/.heroku/python/lib/python3.10/site-packages/django/core/management/init.py", line 446, in execute_from_command_line
utility.execute()
File "/app/.heroku/python/lib/python3.10/site-packages/django/core/management/init.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/app/.heroku/python/lib/python3.10/site-packages/django/core/management/base.py", line 414, in run_from_argv
self.execute(*args, **cmd_options)
File "/app/.heroku/python/lib/python3.10/site-packages/django/core/management/base.py", line 460, in execute
output = self.handle(*args, **options)
File "/app/.heroku/python/lib/python3.10/site-packages/django/core/management/base.py", line 98, in wrapped
res = handle_func(*args, **kwargs)
File "/app/.heroku/python/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 106, in handle
connection.prepare_database()
File "/app/.heroku/python/lib/python3.10/site-packages/psqlextra/backend/base.py", line 32, in prepare_database
super().prepare_database()
File "/app/.heroku/python/lib/python3.10/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 26, in prepare_database
cursor.execute("CREATE EXTENSION IF NOT EXISTS postgis")
File "/app/.heroku/python/lib/python3.10/site-packages/sentry_sdk/integrations/django/init.py", line 544, in execute
return real_execute(self, sql, params)
File "/app/.heroku/python/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/app/.heroku/python/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/app/.heroku/python/lib/python3.10/site-packages/django_read_only/init.py", line 74, in blocker
return execute(sql, params, many, context)
File "/app/.heroku/python/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
with self.db.wrap_database_errors:
File "/app/.heroku/python/lib/python3.10/site-packages/django/db/utils.py", line 91, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/app/.heroku/python/lib/python3.10/site-packages/django/db/backends/utils.py", line 87, in _execute
return self.cursor.execute(sql)
django.db.utils.ProgrammingError: relation "spatial_ref_sys" does not exist
Sentry is attempting to send 2 pending error messages
Waiting up to 2 seconds
Press Ctrl-C to quit

Answer 7

在此之前我们也遇到了一个错误，上面写着pgaudit stack is not empty，它要求我手动将我们的数据库置于维护模式以便轮换它们。 https://devcenter.heroku.com/articles/data-maintenance-cli-commands#heroku-data-maintenances-schedule 这篇文章帮助我解决了第一个问题，但现在我在同一条船上出现了 spatial_ref_sys 错误。

Answer 8

我有一个不需要更改代码库的解决方案，可以完全通过 Heroku CLI 完成

1. 使用 Heroku 数据存储持久性工具在 源数据库或heroku pg:backups:capture -a <SOURCE_APP>。
2. 确定数据库使用哪些 pg 扩展（可以从 psql 中检查） 与 dx)
3. 创建一个逗号分隔的扩展字符串（例如：fuzzystrmatch,pg_stat_statements,pg_trgm,pgcrypto,plpgsql,unaccent,uuid-ossp'）
4. 确保您的 Heroku CLI 至少更新到版本 7.63.0（使用 heroku update更新）
5. 运行这个：

   heroku pg:backups:restore $(heroku pg:backups public-url -a <SOURCE_APP>) DATABASE_URL --extensions '<EXTENSIONS>' -a <TARGET_APP>
   

6. 在 TARGET_APP 上重置测功机

Answer 9

Heroku 团队终于解决了这个问题 :)