【问题标题】:Optimize not in sub-query MYSQL优化不在子查询MYSQL
【发布时间】:2014-05-15 23:09:10
【问题描述】:

我在 MySQL db 中有 >20 000 000 行的表,下面的查询在少量行上执行得很好,但如果有更多行则需要 2-3 秒。我该如何优化它以使其至少运行

SELECT sql_no_cache users.id AS uid,
  name,
  avatar,
  avatar_date,
  driver,
  msg,
  DATE,
  messages.removed,
  from_id = 528798 AS outbox ,
  !(0    IN
  (SELECT read_state
  FROM messages AS msgs FORCE KEY(user_id_2)
  WHERE (msgs.from_id = messages.from_id
  OR msgs.from_id = messages.user_id)
  AND msgs.user_id = 528798
  AND removed = 0
  )) AS read_state
FROM dialog,
  messages,
  users
WHERE messages.id = mid
AND ((uid1 = 528798
AND users.id = uid2)
OR (uid2 = 528798
AND users.id = uid1))
ORDER BY DATE DESC;

显示消息中的索引;

+----------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table    | Non_unique | Key_name    | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| messages |          0 | PRIMARY     |            1 | id          | A         |    27531939 |     NULL | NULL   |      | BTREE      |         |               |
| messages |          1 | to_number   |            1 | to_number   | A         |          22 |     NULL | NULL   |      | BTREE      |         |               |
| messages |          1 | from_id     |            1 | from_id     | A         |      529460 |     NULL | NULL   |      | BTREE      |         |               |
| messages |          1 | from_id     |            2 | to_number   | A         |      529460 |     NULL | NULL   |      | BTREE      |         |               |
| messages |          1 | user_id_2   |            1 | user_id     | A         |      655522 |     NULL | NULL   |      | BTREE      |         |               |
| messages |          1 | user_id_2   |            2 | read_state  | A         |      917731 |     NULL | NULL   |      | BTREE      |         |               |
| messages |          1 | user_id_2   |            3 | removed     | A         |      949377 |     NULL | NULL   |      | BTREE      |         |               |
| messages |          1 | idx_user_id |            1 | user_id     | A         |      809762 |     NULL | NULL   |      | BTREE      |         |               |
| messages |          1 | idx_from_id |            1 | from_id     | A         |      302548 |     NULL | NULL   |      | BTREE      |         |               |
+----------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

描述消息;

+------------+-------------+------+-----+---------+----------------+
| Field      | Type        | Null | Key | Default | Extra          |
+------------+-------------+------+-----+---------+----------------+
| id         | int(11)     | NO   | PRI | NULL    | auto_increment |
| from_id    | int(11)     | NO   | MUL | NULL    |                |
| user_id    | int(11)     | NO   | MUL | NULL    |                |
| group_id   | int(11)     | NO   |     | NULL    |                |
| to_number  | varchar(30) | NO   | MUL | NULL    |                |
| msg        | text        | NO   |     | NULL    |                |
| image      | varchar(20) | NO   |     | NULL    |                |
| date       | bigint(20)  | NO   |     | NULL    |                |
| read_state | tinyint(1)  | NO   |     | 0       |                |
| removed    | tinyint(1)  | NO   |     | NULL    |                |
+------------+-------------+------+-----+---------+----------------+

解释扩展:

+----+--------------------+----------+-------------+---------------+-----------+---------+--------------------+--------+----------+---------------------------------------------------------------------------+
| id | select_type        | table    | type        | possible_keys | key       | key_len | ref                | rows   | filtered | Extra                                                                     |
+----+--------------------+----------+-------------+---------------+-----------+---------+--------------------+--------+----------+---------------------------------------------------------------------------+
|  1 | PRIMARY            | dialog   | index_merge | uid1,uid2     | uid1,uid2 | 4,4     | NULL               |   1707 |   100.00 | Using sort_union(uid1,uid2); Using where; Using temporary; Using filesort |
|  1 | PRIMARY            | users    | ALL         | PRIMARY       | NULL      | NULL    | NULL               | 608993 |   100.00 | Range checked for each record (index map: 0x1)                            |
|  1 | PRIMARY            | messages | eq_ref      | PRIMARY       | PRIMARY   | 4       | numbers.dialog.mid |      1 |   100.00 |                                                                           |
|  2 | DEPENDENT SUBQUERY | msgs     | ref         | user_id_2     | user_id_2 | 6       | const,const,const  |   2607 |   100.00 | Using where                                                               |
+----+--------------------+----------+-------------+---------------+-----------+---------+--------------------+--------+----------+---------------------------------------------------------------------------+

【问题讨论】:

  • 您应该能够将该子查询作为 JOIN 放入。然后,您可以将其作为非相关子查询来执行,这应该更有效。但是,混合连接语法(在隐式连接和显式连接之间)有点混乱,并且您没有提供表声明来整理连接(即,您加入messages.id = mid,但没有说明来自哪个表中间)。如果您可以添加声明,它将有所帮助。
  • 从你的解释我猜这个问题更多的是用户表而不是消息表?你也有关于那张桌子的信息吗?
  • 只是一个边缘信息:在我看来,您可以删除索引 idx_user_id(将使用 user_id_2)和 idx_from_id(将使用 from_id)以节省硬盘空间并加快数据操作操作(插入、更新、删除)。
  • 您能否快速返回查询并进行编辑,以便所有列都表示为 table.column(甚至是 alias.column),以便我们知道各个表的列的来源。
  • 当然你不应该从一开始就使用隐式语法,非常糟糕的编程技术和 SQL 反模式。

标签: mysql sql


【解决方案1】:

做一些猜测,这样的事情可能更有效:-

SELECT DISTINCT users.id AS uid,
  name,
  avatar,
  avatar_date,
  driver,
  msg,
  `DATE`,
  messages.removed,
  from_id = 528798 AS outbox ,
  CASE WHEN msgs.read_state IS NULL THEN 1 ELSE 0 END AS read_state
FROM messages
INNER JOIN dialog ON messages.id = dialog.mid
INNER JOIN users ON (dialog.uid1 = 528798 AND users.id = dialog.uid2) OR (dialog.uid2 = 528798 AND users.id = dialog.uid1)
LEFT OUTER JOIN messages msgs ON msgs.read_state = 0 AND msgs.user_id = 528798 AND removed = 0 AND (msgs.from_id = messages.from_id OR msgs.from_id = messages.user_id)
ORDER BY `DATE` DESC;

这是对消息再次执行一个额外的连接作为左连接,然后使用大小写将结果转换为 0 或 1。

当 LEFT JOIN 可以带回多个匹配行时,DISTINCT 应​​该可以应对(如果这不可能,那么您可以消除 DISTINCT)

怀疑连接到用户中的 OR 子句不会那么有效。用 2 个 LEFT OUTER JOIN 替换针对用户的 INNER JOIN 可能会更好。像这样的东西:-

SELECT DISTINCT COALESCE(users1.id, users2.id) AS uid,
  COALESCE(users1.name, users2.name),
  COALESCE(users1.avatar, users2.avatar),
  COALESCE(users1.avatar_date, users2.avatar_date),
  COALESCE(users1.driver, users2.driver),
  msg,
  `DATE`,
  messages.removed,
  from_id = 528798 AS outbox ,
  CASE WHEN msgs.read_state IS NULL THEN 1 ELSE 0 END AS read_state
FROM messages
INNER JOIN dialog ON messages.id = dialog.mid
LEFT OUTER JOIN users users1 ON (dialog.uid1 = 528798 AND users1.id = dialog.uid2)
LEFT OUTER JOIN users users2 ON (dialog.uid2 = 528798 AND users2.id = dialog.uid1)
LEFT OUTER JOIN messages msgs ON msgs.read_state = 0 AND msgs.user_id = 528798 AND removed = 0 AND (msgs.from_id = messages.from_id OR msgs.from_id = messages.user_id)
WHERE users1.id IS NOT NULL
OR users2.id IS NOT NULL
ORDER BY `DATE` DESC;

【讨论】:

  • 奇妙:错误查询 - 21 行集合(1.46 秒)好一个(你的) - 29 行集合(0.01 秒)谢谢很多。
  • 抱歉,遇到问题 - distinct 不适用于具有多列的 uid。使用 GROUP BY uid 测试的 sql - 比原来的要多)。很高兴看到像 1-st 具有唯一 uid 的示例。无论如何,感谢您的帮助支持您的回答。
  • GROUP BY 可能会导致奇怪的问题。您是否有一些示例数据可以证明该问题?不过,我刚刚发现了一个错误(我也应该在 uid 上使用 COALESCE)。
  • 这是第一个 sql 示例数据结果:610253 0 0 Роо 1400157327052 0 0 1 610253 0 0 Рррррр 1400157320994 0 0 1 610253 0 0 Пьо 1400157309489 0 0 1 610253 0 0 rwerwfew 1400157293064 0 1 1 610253 0 0 Asd 1400157276348 0 1 1 610253 0 0 1400154840172 0 0 1 610253 0 0 1400153603744 0 0 1
  • 这些行都不是重复的(DISTINCT 消除重复的行,而不是具有特定列重复的行)因此不会被消除,并且 GROUP BY uid 将消除除 1 之外的所有行(以及一个是未定义的)。你想如何决定 uid 保留哪一行?我在您的原始查询中看不到任何可以消除这些重复项的内容。我可能可以整理出一个解决方案,但我需要一些示例数据(没有这些数据并且表声明我无法测试任何东西)。
猜你喜欢
  • 2011-11-27
  • 2021-04-15
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2014-07-31
  • 1970-01-01
相关资源
最近更新 更多