对 int + tstzrange 列的简单连接在大约 100 万行上非常缓慢答案

【问题标题】：Simple join on int + tstzrange column painstakingly slow on just ~1 million rows对 int + tstzrange 列的简单连接在大约 100 万行上非常缓慢
【发布时间】：2018-01-17 10:07:54
【问题描述】：

我正在努力处理涉及int-column 和tstzrange 上的“简单”左连接的查询：

SELECT
      table_1.id_col
    , table_1.time_range
    , table_1.other_col_1
    , table_2.other_col_2
FROM table_1
LEFT JOIN table_2
ON table_1.id_col = table_2.id_col
AND table_1.time_range = table_2.time_range

此查询需要 ~80-100 秒运行，最终结果集 ~100 万行（table_1 和 table_2 顺序相同)

此查询是更复杂的 CTE 查询的一部分（实际上选择了这些 1 million 行的一小部分），但我已经解除了存在瓶颈的部分。

我已经添加（我认为）是这些列组合的适当索引 (GIST-index)，但从解释来看，我猜当我基本上加入几乎所有行时，它会被丢弃。

有什么方法可以提高性能吗？

喜欢对行进行物理预排序以进行顺序扫描？

我的桌子：

CREATE TABLE data.table_1 (
    table_1_id SERIAL NOT NULL, 
    id_col INTEGER NOT NULL, 
    time_range TSTZRANGE NOT NULL, 
    other_col_1 INTEGER, 
    PRIMARY KEY (table_1_id), 
);

CREATE INDEX idx_table_1_id_col ON data.table_1 (id_col);
CREATE INDEX idx_table_1_time_range ON data.table_1 USING gist (time_range);
CREATE INDEX idx_table_1_id_col_time_range ON data.table_1 USING gist (id_col, time_range);

CREATE TABLE data.table_2 (
    table_2_id SERIAL NOT NULL, 
    id_col INTEGER NOT NULL, 
    time_range TSTZRANGE NOT NULL, 
    other_col_2 DOUBLE PRECISION, 
    PRIMARY KEY (table_2_id), 
);

CREATE INDEX idx_table_2_id_col ON data.table_2 (id_col);
CREATE INDEX idx_table_2_time_range ON data.table_2 USING gist (time_range);
CREATE INDEX idx_table_2_id_col_time_range ON data.table_2 USING gist (id_col, time_range);

这里是详细的解释：

[
  {
    "Plan": {
      "Node Type": "Hash Join",
      "Join Type": "Left",
      "Startup Cost": 198185.10,
      "Total Cost": 4163704.54,
      "Plan Rows": 73508636,
      "Plan Width": 20,
      "Actual Startup Time": 31055.086,
      "Actual Total Time": 89488.540,
      "Actual Rows": 1015568,
      "Actual Loops": 1,
      "Output": ["table_1.id_col", "table_1.other_col_1", "table_2.other_col_2"],
      "Hash Cond": "((table_1.id_col = table_2.id_col) AND (table_1.time_range = table_2.time_range))",
      "Shared Hit Blocks": 165149,
      "Shared Read Blocks": 632793,
      "Shared Dirtied Blocks": 0,
      "Shared Written Blocks": 0,
      "Local Hit Blocks": 0,
      "Local Read Blocks": 0,
      "Local Dirtied Blocks": 0,
      "Local Written Blocks": 0,
      "Temp Read Blocks": 38220,
      "Temp Written Blocks": 37966,
      "I/O Read Time": 0.000,
      "I/O Write Time": 0.000,
      "Plans": [
        {
          "Node Type": "Seq Scan",
          "Parent Relationship": "Outer",
          "Relation Name": "table_1",
          "Schema": "data",
          "Alias": "table_1",
          "Startup Cost": 0.00,
          "Total Cost": 1492907.36,
          "Plan Rows": 73508636,
          "Plan Width": 34,
          "Actual Startup Time": 24827.453,
          "Actual Total Time": 77143.930,
          "Actual Rows": 904431,
          "Actual Loops": 1,
          "Output": ["table_1.id_col", "table_1.other_col_1", "table_1.time_range"],
          "Shared Hit Blocks": 165147,
          "Shared Read Blocks": 592674,
          "Shared Dirtied Blocks": 0,
          "Shared Written Blocks": 0,
          "Local Hit Blocks": 0,
          "Local Read Blocks": 0,
          "Local Dirtied Blocks": 0,
          "Local Written Blocks": 0,
          "Temp Read Blocks": 0,
          "Temp Written Blocks": 0,
          "I/O Read Time": 0.000,
          "I/O Write Time": 0.000
        },
        {
          "Node Type": "Hash",
          "Parent Relationship": "Inner",
          "Startup Cost": 88292.64,
          "Total Cost": 88292.64,
          "Plan Rows": 4817164,
          "Plan Width": 34,
          "Actual Startup Time": 6204.927,
          "Actual Total Time": 6204.927,
          "Actual Rows": 4817085,
          "Actual Loops": 1,
          "Output": ["table_2.other_col_2", "table_2.id_col", "table_2.time_range"],
          "Hash Buckets": 65536,
          "Original Hash Buckets": 65536,
          "Hash Batches": 128,
          "Original Hash Batches": 128,
          "Peak Memory Usage": 2930,
          "Shared Hit Blocks": 2,
          "Shared Read Blocks": 40119,
          "Shared Dirtied Blocks": 0,
          "Shared Written Blocks": 0,
          "Local Hit Blocks": 0,
          "Local Read Blocks": 0,
          "Local Dirtied Blocks": 0,
          "Local Written Blocks": 0,
          "Temp Read Blocks": 0,
          "Temp Written Blocks": 31422,
          "I/O Read Time": 0.000,
          "I/O Write Time": 0.000,
          "Plans": [
            {
              "Node Type": "Seq Scan",
              "Parent Relationship": "Outer",
              "Relation Name": "table_2",
              "Schema": "data",
              "Alias": "table_2",
              "Startup Cost": 0.00,
              "Total Cost": 88292.64,
              "Plan Rows": 4817164,
              "Plan Width": 34,
              "Actual Startup Time": 0.650,
              "Actual Total Time": 3769.157,
              "Actual Rows": 4817085,
              "Actual Loops": 1,
              "Output": ["table_2.other_col_2", "table_2.id_col", "table_2.time_range"],
              "Shared Hit Blocks": 2,
              "Shared Read Blocks": 40119,
              "Shared Dirtied Blocks": 0,
              "Shared Written Blocks": 0,
              "Local Hit Blocks": 0,
              "Local Read Blocks": 0,
              "Local Dirtied Blocks": 0,
              "Local Written Blocks": 0,
              "Temp Read Blocks": 0,
              "Temp Written Blocks": 0,
              "I/O Read Time": 0.000,
              "I/O Write Time": 0.000
            }
          ]
        }
      ]
    },
    "Planning Time": 0.350,
    "Triggers": [
    ],
    "Execution Time": 89689.809
  }
]

【问题讨论】：

你不能直接在这个查询中加入一些 where 条件（我猜你以后会过滤这些结果）吗？
@LorenzoCatalano，是的，但它是通过 CTE 产生的条件间接完成的。我基本上还有其他一些表，其中加入了上述子集。（如果有意义的话）
看起来像一个普通的连接，我无法准确说出普通话的意思，但我看到像“Plan Rows”：73508636，这是什么意思？

标签： postgresql date-range

【解决方案1】：

使用CLUSTER 对数据进行物理排序将查询时间缩短到~5 秒，这没关系，考虑到我将进一步选择行的子集：

CLUSTER table_1 USING idx_table_1_id_col_time_range;
CLUSTER table_2 USING idx_table_2_id_col_time_range;

【讨论】：