MySQL ORDER BY 子句将查询速度降低 4 倍

【问题标题】：MySQL ORDER BY clause slowing query by 4xMySQL ORDER BY 子句将查询速度降低 4 倍
【发布时间】：2021-11-17 23:36:58
【问题描述】：

更新：我已经大大简化了下面的调用并对问题进行了调整。

我正在想办法让这个查询更快。

我的查询检索特定活动（在本例中为 4）中个人的个人/通信信息。我加入了最近的通信并按通信 date_created 值排序。下面是我们使用的 SQL。

请注意，我之所以使用 LEFT JOIN，是因为只有少数人可能会进行与他们相关的对话；我仍然想根据核心选择返回匹配的人（例如，如果 5 人进行了对话，但有 30 人，我仍然想显示全部 30 人，但按有对话的人以及这些对话的日期排序）。

以下是执行时间仅为 350 毫秒的 FAST 调用；问题是由于 AND 子句，它相当于 INNER JOIN，因此不会 LEFT JOIN 个人主记录。

SELECT *
    FROM person_main pm
    LEFT JOIN cms_conversation_member ccmem ON ccmem.member_object_record_id = pm.ID   
        AND ccmem.member_type_c = get_gcs('cms_conversation_member', 'member_type_c', 'internal')
        AND ccmem.member_object_type_sc = get_gss('OBJECT_BASE','PERSON')
        AND ccmem.member_object_subtype_sc = get_gss('OBJECT_PERSON','MAIN')
    LEFT JOIN cms_conversation_main convm ON convm.ID = ccmem.cms_conversation_main_ref_id
    LEFT JOIN cms_communication_main ccomm ON ccomm.cms_conversation_main_ref_id = convm.ID
        AND ccomm.comm_direction_c = get_gcs('cms_communication_main', 'comm_direction_c', 'incoming')
    WHERE 1 = 1       
    GROUP BY pm.ID
    ORDER BY ccomm.date_created
    LIMIT 0,30;

这是调用的 SQL 解释

[
    {
        "id": "1",
        "select_type": "PRIMARY",
        "table": "ci",
        "partitions": "null",
        "type": "ref",
        "possible_keys": "UK_campaign_item",
        "key": "UK_campaign_item",
        "key_len": "6",
        "ref": "const,const,const",
        "rows": "1000",
        "filtered": "100.0",
        "Extra": "Using index; Using temporary; Using filesort"
    },
    {
        "id": "1",
        "select_type": "PRIMARY",
        "table": "pm",
        "partitions": "null",
        "type": "eq_ref",
        "possible_keys": "PRIMARY,ID_UNIQUE,IDX_person_main_name_first,IDX_person_main_name_middle,IDX_person_main_name_last,IDX_person_main_name_nick",
        "key": "PRIMARY",
        "key_len": "4",
        "ref": "xxx.ci.item_object_ref_id",
        "rows": "1",
        "filtered": "100.0",
        "Extra": "null"
    },
    {
        "id": "1",
        "select_type": "PRIMARY",
        "table": "ccomm",
        "partitions": "null",
        "type": "eq_ref",
        "possible_keys": "PRIMARY,UK_cms_communication_main_ID",
        "key": "PRIMARY",
        "key_len": "4",
        "ref": "func",
        "rows": "1",
        "filtered": "100.0",
        "Extra": "Using where"
    },
    {
        "id": "2",
        "select_type": "DEPENDENT SUBQUERY",
        "table": "ccomm2",
        "partitions": "null",
        "type": "index",
        "possible_keys": "FK_cms_communication_main_cms_conversation_main_ref_id,IDX_cms_communication_main_comm_direction_c",
        "key": "IDX_cms_communication_main_date_created",
        "key_len": "5",
        "ref": "null",
        "rows": "2",
        "filtered": "40.0",
        "Extra": "Using where; Backward index scan"
    },
    {
        "id": "2",
        "select_type": "DEPENDENT SUBQUERY",
        "table": "<subquery3>",
        "partitions": "null",
        "type": "eq_ref",
        "possible_keys": "<auto_distinct_key>",
        "key": "<auto_distinct_key>",
        "key_len": "6",
        "ref": "xxx.ccomm2.cms_conversation_main_ref_id,xxx.pm.ID",
        "rows": "1",
        "filtered": "100.0",
        "Extra": "Using where"
    },
    {
        "id": "3",
        "select_type": "MATERIALIZED",
        "table": "ccmem",
        "partitions": "null",
        "type": "ref",
        "possible_keys": "IDX_cms_conversation_member,IDX_cms_conversation_member_member_type_c,FK_cms_conversation_member_cms_conversation_main_ref_id",
        "key": "IDX_cms_conversation_member",
        "key_len": "6",
        "ref": "const,const,xxx.pm.ID",
        "rows": "3",
        "filtered": "100.0",
        "Extra": "Using where"
    }
]

这是返回所有记录的调用（即使是没有通信的人），但需要 2.4 秒（比上述调用慢 6 倍）。 但这是我需要的结果数据。

    SELECT *
    FROM person_main pm
    LEFT JOIN cms_conversation_member ccmem ON ccmem.member_object_record_id = pm.ID   
        AND ccmem.member_type_c = get_gcs('cms_conversation_member', 'member_type_c', 'internal')
        AND ccmem.member_object_type_sc = get_gss('OBJECT_BASE','PERSON')
        AND ccmem.member_object_subtype_sc = get_gss('OBJECT_PERSON','MAIN')
    LEFT JOIN cms_conversation_main convm ON convm.ID = ccmem.cms_conversation_main_ref_id
    LEFT JOIN cms_communication_main ccomm ON ccomm.cms_conversation_main_ref_id = convm.ID
        AND ccomm.comm_direction_c = get_gcs('cms_communication_main', 'comm_direction_c', 'incoming')
    WHERE 1 = 1       
    GROUP BY pm.ID
    ORDER BY ccomm.date_created
    LIMIT 0,30;

这是上述调用的解释：

[
    {
        "id": "1",
        "select_type": "PRIMARY",
        "table": "ci",
        "partitions": "null",
        "type": "ref",
        "possible_keys": "UK_campaign_item",
        "key": "UK_campaign_item",
        "key_len": "6",
        "ref": "const,const,const",
        "rows": "1000",
        "filtered": "100.0",
        "Extra": "Using index; Using temporary; Using filesort"
    },
    {
        "id": "1",
        "select_type": "PRIMARY",
        "table": "pm",
        "partitions": "null",
        "type": "eq_ref",
        "possible_keys": "PRIMARY,ID_UNIQUE,IDX_person_main_name_first,IDX_person_main_name_middle,IDX_person_main_name_last,IDX_person_main_name_nick",
        "key": "PRIMARY",
        "key_len": "4",
        "ref": "xxx.ci.item_object_ref_id",
        "rows": "1",
        "filtered": "100.0",
        "Extra": "null"
    },
    {
        "id": "1",
        "select_type": "PRIMARY",
        "table": "ccomm",
        "partitions": "null",
        "type": "eq_ref",
        "possible_keys": "PRIMARY,UK_cms_communication_main_ID",
        "key": "PRIMARY",
        "key_len": "4",
        "ref": "func",
        "rows": "1",
        "filtered": "100.0",
        "Extra": "Using where"
    },
    {
        "id": "2",
        "select_type": "DEPENDENT SUBQUERY",
        "table": "ccomm2",
        "partitions": "null",
        "type": "index",
        "possible_keys": "FK_cms_communication_main_cms_conversation_main_ref_id,IDX_cms_communication_main_comm_direction_c",
        "key": "IDX_cms_communication_main_date_created",
        "key_len": "5",
        "ref": "null",
        "rows": "2",
        "filtered": "40.0",
        "Extra": "Using where; Backward index scan"
    },
    {
        "id": "2",
        "select_type": "DEPENDENT SUBQUERY",
        "table": "<subquery3>",
        "partitions": "null",
        "type": "eq_ref",
        "possible_keys": "<auto_distinct_key>",
        "key": "<auto_distinct_key>",
        "key_len": "6",
        "ref": "xxx.ccomm2.cms_conversation_main_ref_id,xxx.pm.ID",
        "rows": "1",
        "filtered": "100.0",
        "Extra": "Using where"
    },
    {
        "id": "3",
        "select_type": "MATERIALIZED",
        "table": "ccmem",
        "partitions": "null",
        "type": "ref",
        "possible_keys": "IDX_cms_conversation_member,IDX_cms_conversation_member_member_type_c,FK_cms_conversation_member_cms_conversation_main_ref_id",
        "key": "IDX_cms_conversation_member",
        "key_len": "6",
        "ref": "const,const,xxx.pm.ID",
        "rows": "3",
        "filtered": "100.0",
        "Extra": "Using where"
    }
]

这里是表创建脚本。

cms_communication_main

SET NAMES 'utf8mb4';

--
-- Create table `cms_communication_main`
--
CREATE TABLE cms_communication_main (
    ID int NOT NULL AUTO_INCREMENT,
    date_created datetime NOT NULL,
    comm_id_external varchar(1024) NOT NULL,
    comm_platform_c smallint NOT NULL,
    comm_direction_c tinyint NOT NULL,
    comm_state_sc smallint NOT NULL,
    comm_timestamp datetime DEFAULT NULL,
    comm_details json DEFAULT NULL,
    cms_conversation_main_ref_id int NOT NULL,
    PRIMARY KEY (ID),
    UNIQUE INDEX UK_cms_communication_main_ID (ID)
)
ENGINE = INNODB,
AUTO_INCREMENT = 6,
AVG_ROW_LENGTH = 3276,
CHARACTER SET utf8mb4,
COLLATE utf8mb4_unicode_ci;

--
-- Create index `IDX_cms_communication_main_date_created` on table `cms_communication_main`
--
ALTER TABLE cms_communication_main
ADD INDEX IDX_cms_communication_main_date_created (date_created);

--
-- Create index `IDX_cms_communication_main_comm_platform_c` on table `cms_communication_main`
--
ALTER TABLE cms_communication_main
ADD INDEX IDX_cms_communication_main_comm_platform_c (comm_platform_c);

--
-- Create index `IDX_cms_communication_main_comm_direction_c` on table `cms_communication_main`
--
ALTER TABLE cms_communication_main
ADD INDEX IDX_cms_communication_main_comm_direction_c (comm_direction_c);

--
-- Create index `IDX_cms_communication_main_comm_id_external` on table `cms_communication_main`
--
ALTER TABLE cms_communication_main
ADD FULLTEXT INDEX IDX_cms_communication_main_comm_id_external (comm_id_external);

--
-- Create foreign key
--
ALTER TABLE cms_communication_main
ADD CONSTRAINT FK_cms_communication_main_cms_conversation_main_ref_id FOREIGN KEY (cms_conversation_main_ref_id)
REFERENCES cms_conversation_main (ID);

cms_conversation_main

SET NAMES 'utf8mb4';

CREATE TABLE cms_conversation_main (
    ID int NOT NULL AUTO_INCREMENT,
    date_created datetime NOT NULL,
    conversation_type_c smallint NOT NULL,
    conversation_id_external varchar(995) DEFAULT '',
    owner_account_main_ref_id int NOT NULL,
    conversation_state_sc smallint NOT NULL,
    PRIMARY KEY (ID),
    UNIQUE INDEX UK_cms_conversation_main_ID (ID)
)
ENGINE = INNODB,
AUTO_INCREMENT = 6,
AVG_ROW_LENGTH = 47,
CHARACTER SET utf8mb4,
COLLATE utf8mb4_unicode_ci;

--
-- Create index `IDX_cms_conversation_main_date_created` on table `cms_conversation_main`
--
ALTER TABLE cms_conversation_main
ADD INDEX IDX_cms_conversation_main_date_created (date_created);

--
-- Create foreign key
--
ALTER TABLE cms_conversation_main
ADD CONSTRAINT FK_cms_conversation_main_owner_account_main_ref_id FOREIGN KEY (owner_account_main_ref_id)
REFERENCES account_main (ID);

cms_conversation_member

SET NAMES 'utf8mb4';

CREATE TABLE cms_conversation_member (
    ID int NOT NULL AUTO_INCREMENT,
    date_joined datetime NOT NULL,
    member_type_c smallint NOT NULL,
    member_originator tinyint(1) NOT NULL DEFAULT 0,
    member_object_type_sc smallint NOT NULL,
    member_object_subtype_sc smallint NOT NULL,
    member_object_record_id smallint NOT NULL,
    member_custom_id varchar(995) DEFAULT NULL,
    member_communication_key varchar(255) DEFAULT NULL,
    cms_conversation_main_ref_id int NOT NULL,
    PRIMARY KEY (ID),
    UNIQUE INDEX UK_cms_conversation_member_ID (ID)
)
ENGINE = INNODB,
AUTO_INCREMENT = 11,
AVG_ROW_LENGTH = 50,
CHARACTER SET utf8mb4,
COLLATE utf8mb4_unicode_ci;

--
-- Create index `IDX_cms_conversation_member_date_joined` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member_date_joined (date_joined);

--
-- Create index `IDX_cms_conversation_member` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member (member_object_type_sc, member_object_subtype_sc, member_object_record_id);

--
-- Create index `IDX_cms_conversation_member_member_type_c` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member_member_type_c (member_type_c);

--
-- Create index `IDX_cms_conversation_member_member_originator` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member_member_originator (member_originator);

--
-- Create index `IDX_cms_conversation_member_member_communication_key` on table `cms_conversation_member`
--
ALTER TABLE cms_conversation_member
ADD INDEX IDX_cms_conversation_member_member_communication_key (member_communication_key);

--
-- Create foreign key
--
ALTER TABLE cms_conversation_member
ADD CONSTRAINT FK_cms_conversation_member_cms_conversation_main_ref_id FOREIGN KEY (cms_conversation_main_ref_id)
REFERENCES cms_conversation_main (ID);

知道为什么第二次调用的时间比第一次长 5-6 倍吗？为什么 MySQL 在与来自 person_main 表的通信不匹配的左连接上“滞后”这么多？

【问题讨论】：

  • 请编辑您的问题以添加（作为文本，而不是图像）show create table yourtablename 的查询和输出中的所有表的explain SELECT rest-of-your-statement 的输出，无论是否按顺序
  • 你说的没有命令的速度是有限制还是没有限制？如果有，没有限制会有多少行，删除限制也会使其变慢吗？
  • @ysth 我添加了 SQL 解释。我不想用表格创建来膨胀它；但我可以说所有引用的字段都是索引的。
  • 为了帮助您解决query-optimization 的问题，我们需要更多信息。请read this。我们确实需要您的表定义。
  • @O.Jones 理解并更新了。

标签： mysql sql-order-by query-optimization where-clause


【解决方案1】：

  • get_gcs() 是什么？

  • 如果是存储函数，是DETERMINISTIC吗？

  • 尝试使用JOIN 而不是IN ( SELECT ... )。 （好像IN (... campaign_item ci ...)已经变成了JOIN（见EXPLAIN的第一行）。

  • 暂定索引：

      ccomm:  INDEX(ID,  date_created)
  cms_conversation_member:  INDEX(member_type_c, member_object_type_sc,
     member_object_subtype_sc, member_object_record_id,  cms_conversation_main_ref_id)
  cms_communication_main:  INDEX(cms_conversation_main_ref_id,
     comm_direction_c, date_created,  ID)
  ci:  INDEX(campaign_main_ref_id, item_object_type_sc,
     item_object_subtype_sc, item_object_ref_id)

  • 您确定要LEFT JOIN，而不仅仅是INNER JOIN？毕竟，查询似乎期望这个值：ORDER BY ccomm.date_created。

  • 您可能不需要GROUP BY。如果你这样做了，那么你可能仍然可以通过将查询从里到外来摆脱它。

  • 为子查询计时（为pm.ID 填写一些值）。目标是确定其中之一是否是主要的性能瓶颈。

  • PRIMARY,ID_UNIQUE -- 这似乎暗示你有PRIMARY KEY(ID), UNIQUE(ID)。如果是这样，请注意 PK 是唯一键。因此UNIQUE(ID) 是多余的。

【讨论】：

  • 是的，似乎几乎每个条件都包含在一个函数中，这意味着他将绕过几乎所有这些索引，除非我弄错了。
  • @ChrisStricland - 有迹象表明get_gss('constant', 'constant') 是不变的。也就是说，它似乎是预先评估的。因此，column = get_gss(...)可以在column上建立索引。
  • 实际上，我想只要您不将该函数应用于列本身，就可以了。我刚刚在函数调用中看到（看起来像）一些列名并误读了它。再看一遍，我想只要你在做WHERE get_gss(col, arg) = condition 就可以了。
  • @ChrisStricland - 第一部分是正确的。 fcn(col) 不是“sargable”，因此索引没有用处。最后一部分是不可分割的。
  • 感谢 Rick 的评论，但不幸的是它对我没有帮助。注意我已经更新了核心示例代码以优化和简化问题。
猜你喜欢
  • 2017-10-13
  • 1970-01-01
  • 1970-01-01
  • 2014-06-05
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2012-11-30
  • 2012-04-09
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode