【问题标题】:Query in Postgres takes more time for later added data than earlier added dataPostgres 中的查询对于后来添加的数据比之前添加的数据需要更多时间
【发布时间】:2019-12-21 12:00:09
【问题描述】:

我有一个应用程序,我必须过滤一个表并聚合各种值。现在,如果过滤的数据范围是较早添加的行的一部分,则查询执行速度很快,而如果过滤的数据范围是后来添加的行的一部分,则查询需要更长的时间。

这是第一个查询,过滤 date_int 介于 43719 和 43726 之间的数据。此查询仅需 669 毫秒

SELECT COUNT(*) FROM (SELECT DISTINCT "content_song"."title" AS Col1 FROM "tags_playlistentryviewership" INNER JOIN "content_playlistentry" ON ("tags_playlistentryviewership"."playlist_entry_id" = "content_playlistentry"."id") 左外连接 "content_song" ON ("content_playlistentry"."song_id" = "content_song"."id") WHERE (("tags_playlistentryviewership"."channel_id" = '1ddc96bc-2f77-4293-b774-0018bc541044'::uuid 或 "tags_playlistentryviewership"."channel_id" = '25260324-e79d-4e0b-b08b-f7ac9419024d'::uuid 或 "tags_playlistentryviewership"."channel_id" = '25c29a3b-635d-4afc-b0ac-5373132f7ed8'::uuid 或 "tags_playlistentryviewership"."channel_id" = '3af60df1-7d19-48eb-971e-162504e33b24'::uuid 或 "tags_playlistentryviewership"."channel_id" = '783dbae0-cca5-4272-b101-7e10d4ce1f17'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'b4f627c3-727b-40b6-bbe0-2501a8a30079'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'c5933372-af36-406e-89e3-49d3e30bfa61'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'c9a26012-f070-4637-a80f-947024856887'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'f53781f4-7248-4cdc-a0c8-30fe691b9248'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'fa2dfe7c-8218-4236-a70a-f76729e35f5e'::uuid) 和 "tags_playlistentryviewership"."target_group_id" = '23f70629-ebb7-4f13-af5c-2ed26e0d0e31'::uuid 和 "tags_playlistentryviewership"."region_id" IN (SELECT U0."id" AS Col1 FROM "content_region" U0 INNER JOIN "content_region_portal_map" U1 ON (U0."id" = U1."to_region_id") U1."from_region_id" = 'c8fd4db2-790a-4fa2-9a54-8f6cdf76da2f'::uuid) 和 "tags_playlistentryviewership"."date_int" >= 43719 AND "tags_playlistentryviewership"."date_int"

虽然所有其他过滤器保持不变,但现在查询 date_int 介于 43806 和 43812 之间的数据如下所示。此查询需要 11100 毫秒。

SELECT COUNT(*) FROM (SELECT DISTINCT "content_song"."title" AS Col1 FROM "tags_playlistentryviewership" INNER JOIN "content_playlistentry" ON ("tags_playlistentryviewership"."playlist_entry_id" = "content_playlistentry"."id") 左外连接 "content_song" ON ("content_playlistentry"."song_id" = "content_song"."id") WHERE (("tags_playlistentryviewership"."channel_id" = '1ddc96bc-2f77-4293-b774-0018bc541044'::uuid 或 "tags_playlistentryviewership"."channel_id" = '25260324-e79d-4e0b-b08b-f7ac9419024d'::uuid 或 "tags_playlistentryviewership"."channel_id" = '25c29a3b-635d-4afc-b0ac-5373132f7ed8'::uuid 或 "tags_playlistentryviewership"."channel_id" = '3af60df1-7d19-48eb-971e-162504e33b24'::uuid 或 "tags_playlistentryviewership"."channel_id" = '783dbae0-cca5-4272-b101-7e10d4ce1f17'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'b4f627c3-727b-40b6-bbe0-2501a8a30079'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'c5933372-af36-406e-89e3-49d3e30bfa61'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'c9a26012-f070-4637-a80f-947024856887'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'f53781f4-7248-4cdc-a0c8-30fe691b9248'::uuid 或 "tags_playlistentryviewership"."channel_id" = 'fa2dfe7c-8218-4236-a70a-f76729e35f5e'::uuid) 和 "tags_playlistentryviewership"."target_group_id" = '23f70629-ebb7-4f13-af5c-2ed26e0d0e31'::uuid 和 "tags_playlistentryviewership"."region_id" IN (SELECT U0."id" AS Col1 FROM "content_region" U0 INNER JOIN "content_region_portal_map" U1 ON (U0."id" = U1."to_region_id") U1."from_region_id" = 'c8fd4db2-790a-4fa2-9a54-8f6cdf76da2f'::uuid) 和 "tags_playlistentryviewership"."date_int" >= 43806 AND "tags_playlistentryviewership"."date_int"

我不明白为什么查询响应时间如此不同以及如何改进。

【问题讨论】:

  • 在添加最近的数据后是否对表进行了清理?
  • 不,我没有运行真空
  • 如果 VACUUM 无法修复它,请为每个版本的查询发布 EXPLAIN (ANALYZE, BUFFERS)。如果可以的话,最好先打开 track_io_timing。您需要成为超级用户才能执行此操作。
  • 在我用完 Vacuum 之后。现在查询时间几乎相似。

标签: sql django database postgresql


【解决方案1】:

尝试对两个查询执行explain analyze,看看它们的执行计划是否不同。
PostgreSQL 包含了相当不错的查询计划系统,但它有时会选择无效的路径。
有了这些信息,您就可以决定下一步该做什么。

另外,我建议检查您在 where 和 join 语句中使用的所有列是否都有索引。

如果您可以更改这些查询,那么您可以删除未使用的子查询:

select count(distinct column) from ...

【讨论】:

    猜你喜欢
    • 2019-05-08
    • 2021-03-21
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2022-01-14
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多