【问题标题】:Postgres Optimiser using parallel sequential scan instead of index scanPostgres 优化器使用并行顺序扫描而不是索引扫描
【发布时间】:2022-01-15 02:53:21
【问题描述】:

我有两个主表和一个子表。所以对于这个问题,我将把master称为m,将child table称为c。

主表属性:

Records: 50 million
primary key (index): m_id 
btree index: modified_id

子表属性(1-n 关系):

Records: 400 million
primary key (index): c_id
foreign key (btree index): m_id

查询计划

Gather  (cost=9159.80..6768939.59 rows=18940 width=107) (actual time=137160.885..297009.782 rows=25 loops=1)
  Output: m.<date_column>, m.m_id, m.other_unique_id_1, m.modified_id, c.<date_column>, c.c_id, c.m_id, c.<other_column_1>, c.<other_column_2>, c.<other_column_3>, c.<other_column_4>, c.<other_column_5>, c.<other_column_6>, c.<other_column_7>, c.<other_column_8>
  Workers Planned: 2
  Workers Launched: 2
  Buffers: shared hit=11266 read=4680890
  ->  Hash Join  (cost=8159.80..6766045.59 rows=7892 width=107) (actual time=87154.268..297003.331 rows=8 loops=3)
        Output: c.<date_column>, c.m_id, c.other_unique_id_1, c.modified_id, c.<date_column>, c.c_id, c.m_id, c.<other_column_1>, c.<other_column_2>, c.<other_column_3>, c.<other_column_4>, c.<other_column_5>, c.<other_column_6>, c.<other_column_7>, c.<other_column_8>
        Inner Unique: true
        Hash Cond: (c.m_id = c.m_id)
        Buffers: shared hit=11266 read=4680890
        Worker 0: actual time=82162.730..297002.014 rows=13 loops=1
          Buffers: shared hit=3789 read=1555017
        Worker 1: actual time=42139.478..297002.514 rows=8 loops=1
          Buffers: shared hit=3634 read=1569261
        ->  Parallel Seq Scan on child c  (cost=0.00..6328357.20 rows=163629920 width=87) (actual time=0.638..279084.058 rows=130858840 loops=3)
              Output: c.<date_column>, c.c_id, c.m_id, c.<other_column_1>, c.<other_column_2>, c.<other_column_3>, c.<other_column_4>, c.<other_column_5>, c.<other_column_6>, c.<other_column_7>, c.<other_column_8>
              Buffers: shared hit=11171 read=4680887
              Worker 0: actual time=0.814..279170.920 rows=130423418 loops=1
                Buffers: shared hit=3746 read=1555017
              Worker 1: actual time=1.095..278955.836 rows=131603000 loops=1
                Buffers: shared hit=3592 read=1569260
        ->  Hash  (cost=8132.89..8132.89 rows=2153 width=20) (actual time=0.046..0.047 rows=8 loops=3)
              Output: c.<date_column>, c.m_id, c.other_unique_id_1, c.modified_id
              Buckets: 4096  Batches: 1  Memory Usage: 33kB
              Buffers: shared hit=35 read=3
              Worker 0: actual time=0.026..0.027 rows=8 loops=1
                Buffers: shared hit=13
              Worker 1: actual time=0.059..0.060 rows=8 loops=1
                Buffers: shared hit=12 read=1
              ->  Bitmap Heap Scan on master c  (cost=41.25..8132.89 rows=2153 width=20) (actual time=0.031..0.041 rows=8 loops=3)
                    Output: c.<date_column>, c.m_id, c.other_unique_id_1, c.modified_id
                    Recheck Cond: (c.modified_id = 561869)
                    Heap Blocks: exact=8
                    Buffers: shared hit=35 read=3
                    Worker 0: actual time=0.018..0.024 rows=8 loops=1
                      Buffers: shared hit=13
                    Worker 1: actual time=0.043..0.055 rows=8 loops=1
                      Buffers: shared hit=12 read=1
                    ->  Bitmap Index Scan on ix_master_modified_id  (cost=0.00..40.71 rows=2153 width=0) (actual time=0.027..0.027 rows=8 loops=3)
                          Index Cond: (c.modified_id = 561869)
                          Buffers: shared hit=11 read=3
                          Worker 0: actual time=0.015..0.015 rows=8 loops=1
                            Buffers: shared hit=5
                          Worker 1: actual time=0.038..0.038 rows=8 loops=1
                            Buffers: shared hit=4 read=1
Planning time: 0.354 ms
Execution time: 297009.825 ms

查询

select
    *
from
    master m
inner join child c on
    m.m_id = c.m_id
where
    m.modified_id = <xyz>

主表的 DDL 语句

CREATE TABLE master (
    <date_column> timestamp NULL DEFAULT CURRENT_DATE,
    m_id serial4 NOT NULL,
    <other_unique_id_1> int4 NULL,
    modified_id int4 NULL,
    CONSTRAINT master_pkey PRIMARY KEY (m_id),
    CONSTRAINT <foreign key> FOREIGN KEY (<other_unique_id_1>) REFERENCES <other table>(<other_unique_id_1>)
);
CREATE INDEX ix_master_modified_id ON master USING btree (modified_id);
CREATE UNIQUE INDEX ix_master_other_unique_id_1 ON master USING btree (other_unique_id_1);

子表的 DDL

CREATE TABLE child (
    <date column> timestamp NULL DEFAULT now(),
    c_id serial4 NOT NULL,
    m_id int4 NULL,
    <other_column_1> varchar(50) NULL,
    <other_column_2> varchar(50) NULL,
    <other_column_3> bool NULL,
    <other_column_4> varchar(50) NULL,
    <other_column_5> varchar(2) NULL,
    <other_column_6> varchar(50) NULL,
    <other_column_7> varchar(10) NULL,
    <other_column_8> text NULL,
    CONSTRAINT child_pkey PRIMARY KEY (c_id),
    CONSTRAINT child_master_fkey FOREIGN KEY (m_id) REFERENCES master(m_id)
);
CREATE INDEX ix_child_m_id ON child USING btree (m_id);

【问题讨论】:

  • 能否同时添加查询和创建表/索引语句?
  • edit您的问题并添加您正在使用的查询。 explain (analyze, verbose, buffers, timing) 的输出也会很有趣。
  • 让我也这样做。而且我不能在解释中添加分析,因为如果你想要它,这将花费很多时间@a_horse_with_no_name,然后我可以解雇它
  • @AnandTripathi 你是对的......我没注意到这是一个 PK :)
  • @AnandTripathi 该计划建议child 节点返回超过 130m 行,这是表的重要部分。可能这就是计划者决定跳过索引的原因。

标签: postgresql indexing


【解决方案1】:

我认为问题的根源在于这种严重的错误估计。

->  Bitmap Index Scan on ix_master_modified_id  (cost=0.00..40.71 rows=2153 width=0) (actual time=0.027..0.027 rows=8 loops=3)

您说您已经对表格进行了 VACUUM ANALYZE。在这种情况下,您可能需要增加统计数据大小,然后再次分析,以获得更好的估计。您可以全局更改 default_statistics_target,也可以将这一列作为目标:

alter table master alter modified_id set statistics 10000;
analyze master;

您可能不需要将它一直增加到 10000(允许的最大值),但如果您只为一列这样做,我认为没有理由尝试对其进行微调。直接打出大炮就行了。如果它有效,那么您可以担心对其进行微调。

【讨论】:

  • 感谢我增加了子 m_id 统计信息,帮助我在解释计划中获得索引
猜你喜欢
  • 2021-08-12
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2021-06-23
  • 2020-02-14
  • 2021-03-19
  • 1970-01-01
相关资源
最近更新 更多