pg_restore 不恢复某个表？答案

【问题标题】：pg_restore not restoring a certain table?pg_restore 不恢复某个表？
【发布时间】：2018-12-21 07:47:31
【问题描述】：

在一个 Django 项目中，我在 lucy_web 应用程序中有一个模型 Question，但对应的 lucy_web_question 表不存在，从数据库 shell 中的 \dt 命令可以看出：

(lucy-web-CVxkrCFK) bash-3.2$ python manage.py dbshell
psql (10.4)
Type "help" for help.

lucy=> \dt
                    List of relations
 Schema |             Name             | Type  |  Owner  
--------+------------------------------+-------+---------
 public | auditlog_logentry            | table | lucyapp
 public | auth_group                   | table | lucyapp
 public | auth_group_permissions       | table | lucyapp
 public | auth_permission              | table | lucyapp
 public | auth_user                    | table | lucyapp
 public | auth_user_groups             | table | lucyapp
 public | auth_user_user_permissions   | table | lucyapp
 public | defender_accessattempt       | table | lucyapp
 public | django_admin_log             | table | lucyapp
 public | django_content_type          | table | lucyapp
 public | django_migrations            | table | lucyapp
 public | django_session               | table | lucyapp
 public | lucy_web_checkin             | table | lucyapp
 public | lucy_web_checkintype         | table | lucyapp
 public | lucy_web_company             | table | lucyapp
 public | lucy_web_expert              | table | lucyapp
 public | lucy_web_expertsessiontype   | table | lucyapp
 public | lucy_web_family              | table | lucyapp
 public | lucy_web_lucyguide           | table | lucyapp
 public | lucy_web_notification        | table | lucyapp
 public | lucy_web_package             | table | lucyapp
 public | lucy_web_packagesessiontype  | table | lucyapp
 public | lucy_web_preactivationfamily | table | lucyapp
 public | lucy_web_profile             | table | lucyapp
 public | lucy_web_questionanswer      | table | lucyapp
 public | lucy_web_questioncategory    | table | lucyapp
 public | lucy_web_session             | table | lucyapp
 public | lucy_web_sessioncategory     | table | lucyapp
 public | lucy_web_sessiontype         | table | lucyapp
 public | lucy_web_userapn             | table | lucyapp
 public | oauth2_provider_accesstoken  | table | lucyapp
 public | oauth2_provider_application  | table | lucyapp
 public | oauth2_provider_grant        | table | lucyapp
 public | oauth2_provider_refreshtoken | table | lucyapp
 public | otp_static_staticdevice      | table | lucyapp
 public | otp_static_statictoken       | table | lucyapp
 public | otp_totp_totpdevice          | table | lucyapp
 public | two_factor_phonedevice       | table | lucyapp
(38 rows)

我们还有一个部署在 Aptible 上的临时环境，其中似乎确实有这些表。使用Aptible CLI to create a database tunnel，如果我psql <connection_url> 和dt 我确实看到lucy_web_question 表：

db=# \dt
                    List of relations
 Schema |             Name             | Type  |  Owner  
--------+------------------------------+-------+---------
 public | auditlog_logentry            | table | aptible
 public | auth_group                   | table | aptible
 public | auth_group_permissions       | table | aptible
 public | auth_permission              | table | aptible
 public | auth_user                    | table | aptible
 public | auth_user_groups             | table | aptible
 public | auth_user_user_permissions   | table | aptible
 public | defender_accessattempt       | table | aptible
 public | django_admin_log             | table | aptible
 public | django_content_type          | table | aptible
 public | django_migrations            | table | aptible
 public | django_session               | table | aptible
 public | lucy_web_checkin             | table | aptible
 public | lucy_web_checkintype         | table | aptible
 public | lucy_web_company             | table | aptible
 public | lucy_web_expert              | table | aptible
 public | lucy_web_expertsessiontype   | table | aptible
 public | lucy_web_family              | table | aptible
 public | lucy_web_lucyguide           | table | aptible
 public | lucy_web_notification        | table | aptible
 public | lucy_web_package             | table | aptible
 public | lucy_web_packagesessiontype  | table | aptible
 public | lucy_web_preactivationfamily | table | aptible
 public | lucy_web_profile             | table | aptible
 public | lucy_web_question            | table | aptible
 public | lucy_web_questionanswer      | table | aptible
 public | lucy_web_questioncategory    | table | aptible
 public | lucy_web_questionprompt      | table | aptible
 public | lucy_web_session             | table | aptible
 public | lucy_web_sessioncategory     | table | aptible
 public | lucy_web_sessiontype         | table | aptible
 public | lucy_web_userapn             | table | aptible
 public | oauth2_provider_accesstoken  | table | aptible
 public | oauth2_provider_application  | table | aptible
 public | oauth2_provider_grant        | table | aptible
 public | oauth2_provider_refreshtoken | table | aptible
 public | otp_static_staticdevice      | table | aptible
 public | otp_static_statictoken       | table | aptible
 public | otp_totp_totpdevice          | table | aptible
 public | two_factor_phonedevice       | table | aptible
(40 rows)

因为这些测试环境上的数据并不重要，我想pg_dump Aptible 数据库和pg_restore 在我的本地机器上。

我的本地DATABASE_URL 是postgres://lucyapp:<my_password>@localhost/lucy，所以首先，我用--format=custom 做了一个pg_dump 并指定一个--file，如下所示：

Kurts-MacBook-Pro-2:lucy2 kurtpeek$ touch staging_db_12_July.dump
Kurts-MacBook-Pro-2:lucy2 kurtpeek$ pg_dump postgresql://aptible:<some_aptible_hash>@localhost.aptible.in:62288/db --format=custom --file=staging_db_12_July.dump
Kurts-MacBook-Pro-2:lucy2 kurtpeek$ ls -lhtr | tail -1
-rw-r--r--   1 kurtpeek  staff   1.5M Jul 12 18:09 staging_db_12_July.dump

这会产生一个 1.5Mb 的 .dump 文件，我尝试通过使用 pg_restore 和 --no-owner 选项和 --role=lucyapp 来恢复该文件（以便将所有者从 aptible 更改为 lucyapp）。但是，这会产生大量“已经存在”的错误，如下图所示：

Kurts-MacBook-Pro-2:lucy2 kurtpeek$ pg_restore staging_db_12_July.dump --dbname=lucy --no-owner --role=lucyapp
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 3522; 0 0 COMMENT EXTENSION plpgsql 
pg_restore: [archiver (db)] could not execute query: ERROR:  must be owner of extension plpgsql
    Command was: COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language';



pg_restore: [archiver (db)] Error from TOC entry 2; 3079 16392 EXTENSION hstore 
pg_restore: [archiver (db)] could not execute query: ERROR:  permission denied to create extension "hstore"
HINT:  Must be superuser to create this extension.
    Command was: CREATE EXTENSION IF NOT EXISTS hstore WITH SCHEMA public;



pg_restore: [archiver (db)] Error from TOC entry 3523; 0 0 COMMENT EXTENSION hstore 
pg_restore: [archiver (db)] could not execute query: ERROR:  extension "hstore" does not exist
    Command was: COMMENT ON EXTENSION hstore IS 'data type for storing sets of (key, value) pairs';



pg_restore: [archiver (db)] Error from TOC entry 197; 1259 16515 TABLE auditlog_logentry aptible
pg_restore: [archiver (db)] could not execute query: ERROR:  relation "auditlog_logentry" already exists
    Command was: CREATE TABLE public.auditlog_logentry (
    id integer NOT NULL,
    object_pk character varying(255) NOT NULL,
    object_id bigint,
    object_repr text NOT NULL,
    action smallint NOT NULL,
    changes text NOT NULL,
    "timestamp" timestamp with time zone NOT NULL,
    actor_id integer,
    content_type_id integer NOT NULL,
    remote_addr inet,
    additional_data jsonb,
    CONSTRAINT auditlog_logentry_action_check CHECK ((action >= 0))
);



WARNING: errors ignored on restore: 294

问题是，如果我在python manage.py dbshell 中再次\dt，我仍然看不到lucy_web_question 表。

我遇到过这个解决方案，Django : Table doesn't exist，就我的情况而言，但在我的情况下，Question 模型被导入并在很多地方用作外键，我认为恢复起来会更容易一个数据库。为什么它不恢复lucy_web_question 表呢？

【问题讨论】：

标签： django database postgresql psql pg-restore

【解决方案1】：

问题似乎在于lucyapp 用户没有足够的权限来创建表。我基本上必须确保\dn+ 命令产生了这个结果：

lucy=# \dn+
                          List of schemas
  Name  |  Owner   |  Access privileges   |      Description       
--------+----------+----------------------+------------------------
 public | postgres | postgres=UC/postgres+| standard public schema
        |          | =UC/postgres        +| 
        |          | lucyapp=UC/postgres  | 
(1 row)

其中lucyapp 同时拥有USAGE (U) 和CREATE (C) 权限。跟随https://www.postgresql.org/docs/9.0/static/sql-grant.html，这可以通过命令来实现

GRANT USAGE ON SCHEMA public TO lucyapp;
GRANT CREATE ON SCHEMA public TO lucyapp;

在运行这些命令之前，我还创建了 lucyapp 和 superuser，尽管不建议在生产环境中这样做。

【讨论】：