【发布时间】:2021-11-13 17:29:32
【问题描述】:
我有一个简单的查询:
select count(*)
from ror
where rorsts not in ( 'RECD', 'CANC', 'CCNS', 'SDNY' )
and ( ( rorbcd = '00009310022487'
and rorfid = 'VDR' )
or ( rorfid = 'VDR'
and rorbcd in (
SELECT pplbcd
from ppl
where pplfid = 'VDR'
and pplscb6 = '00009310022487'
and pplsflg = 'Y'
and pplsku = '0332690-008'
and ppldoc = '73'
and pplsca9 = ''
and pplven = '10112' ) )
)
在 MySql 5.6 机器上,它运行得非常快:
+----------+
| count(*) |
+----------+
| 2 |
+----------+
1 row in set (0.01 sec)
但在我的 MySql 8 服务器上,它的运行速度要慢得多:
+----------+
| count(*) |
+----------+
| 2 |
+----------+
1 row in set (0.90 sec)
我主要使用两个服务器的默认参数(嗯,Amazon RDS 默认值)。什么参数会导致这种减速?什么配置可以帮助它在 MySql 8 中以与 MySql 5.6 大致相同的速度运行。
根据网上许多其他地方,这是一个糟糕的 SQL。我相信我可以很容易地将它重写为exists,这应该会更有效率。我的问题是我们正在从 5.6 迁移到 8.0,很有可能我会错过一个查询,或者由于我的一些奇怪的要求而无法修复其中一个查询。我敢打赌,我可以在 MySql 8 配置中调整一些参数,以使其以与 5.6 大致相当的方式运行。这就是我要找的。p>
更多信息:
- 在硬件方面相同的 Amazon RDS 服务器设置
- 我们刚刚将 5.6 升级到 5.7,然后将 5.7 升级到 8.0。我尽可能使用默认的 MySql 参数。我只是故意更改了
lower_case_table_names = 1和max_connections=300,以匹配 5.6 和我们当前的要求。 - PPL 和 ROR 表非常复杂。 PPL 137 列,ROR 144 列。
对于 MySql 5.6 的解释,更简单(而且更快)的查询,以及一些索引信息:
mysql> select version();
+------------+
| version() |
+------------+
| 5.6.51-log |
+------------+
1 row in set (0.01 sec)
mysql> explain extended select count(*) from ror where rorsts not in ( 'RECD', 'CANC', 'CCNS', 'SDNY' ) and ( ( rorbcd = '00009310022487' and rorfid = 'VDR' ) or ( rorfid = 'VDR' and rorbcd in ( select pplbcd from ppl where pplfid = 'VDR' and pplscb6 = '00009310022487' and pplsflg = 'Y' and pplsku = '0332690-008' and ppldoc = '73' and pplsca9 = '' and pplven = '10112' ) ) );
+----+--------------------+-------+-----------------+----------------------------------------------------------------+--------------+---------+------------+------+----------+------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------------+-------+-----------------+----------------------------------------------------------------+--------------+---------+------------+------+----------+------------------------------------+
| 1 | PRIMARY | ror | ref | ROR_RORBCDK,ROR_RORSTSK,ROR_RORBSTSK,ROR_RORSDT3K,ROR_RORGMOQK | ROR_RORGMOQK | 5 | const | 1 | 100.00 | Using index condition; Using where |
| 2 | DEPENDENT SUBQUERY | ppl | unique_subquery | PPL_PPLBCDC,PPL_PPLDOCK,PPL_PPLVENC,PPL_PPLSKUC,PPL_PPLFID | PPL_PPLBCDC | 29 | func,const | 1 | 100.00 | Using where |
+----+--------------------+-------+-----------------+----------------------------------------------------------------+--------------+---------+------------+------+----------+------------------------------------+
2 rows in set, 1 warning (0.00 sec)
mysql> show warnings; +-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note | 1003 | /* select#1 */ select count(0) AS `count(*)` from `celltreat`.`ror` where ((`celltreat`.`ror`.`RORSTS` not in ('RECD','CANC','CCNS','SDNY')) and (((`celltreat`.`ror`.`RORFID` = 'VDR') and (`celltreat`.`ror`.`RORBCD` = '00009310022487')) or ((`celltreat`.`ror`.`RORFID` = 'VDR') and <in_optimizer>(`celltreat`.`ror`.`RORBCD`,<exists>(<primary_index_lookup>(<cache>(`celltreat`.`ror`.`RORBCD`) in ppl on PPL_PPLBCDC where ((`celltreat`.`ppl`.`PPLVEN` = '10112') and (`celltreat`.`ppl`.`PPLSCA9` = '') and (`celltreat`.`ppl`.`PPLDOC` = '73') and (`celltreat`.`ppl`.`PPLSKU` = '0332690-008') and (`celltreat`.`ppl`.`PPLSFLG` = 'Y') and (`celltreat`.`ppl`.`PPLSCB6` = '00009310022487') and (`celltreat`.`ppl`.`PPLFID` = 'VDR') and (<cache>(`celltreat`.`ror`.`RORBCD`) = `celltreat`.`ppl`.`PPLBCD`)))))))) |
+-------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql>
mysql> select count(*) from ppl;
+----------+
| count(*) |
+----------+
| 95108 |
+----------+
1 row in set (0.01 sec)
mysql> explain extended select count(*) from ppl;
+----+-------------+-------+-------+---------------+------------+---------+------+-------+----------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+-------+---------------+------------+---------+------+-------+----------+-------------+
| 1 | SIMPLE | ppl | index | NULL | PPL_PPLFID | 5 | NULL | 11194 | 100.00 | Using index |
+----+-------------+-------+-------+---------------+------------+---------+------+-------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
mysql> show warnings;
+-------+------+---------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+---------------------------------------------------------------------+
| Note | 1003 | /* select#1 */ select count(0) AS `count(*)` from `celltreat`.`ppl` |
+-------+------+---------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql>
mysql> SELECT TABLE_NAME, index_name, COUNT(1) column_count FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA = 'veeder' and table_name in ( 'ppl', 'ror' ) group by table_name, index_name order by table_name collate utf8_general_ci, index_name collate utf8_general_ci;
+------------+--------------+--------------+
| TABLE_NAME | index_name | column_count |
+------------+--------------+--------------+
| ppl | PPLKEY | 1 |
| ppl | PPL_PPLBCDC | 2 |
| ppl | PPL_PPLDOCK | 3 |
| ppl | PPL_PPLFID | 1 |
| ppl | PPL_PPLPONOC | 4 |
| ppl | PPL_PPLSKUC | 4 |
| ppl | PPL_PPLVENC | 5 |
| ror | RORKEY | 1 |
| ror | ROR_RORBCDK | 5 |
| ror | ROR_RORBSTSK | 4 |
| ror | ROR_RORCNSK | 3 |
| ror | ROR_RORCRDTK | 4 |
| ror | ROR_RORCUIDK | 5 |
| ror | ROR_RORDOCK | 5 |
| ror | ROR_RORGMOQK | 4 |
| ror | ROR_RORPONOK | 6 |
| ror | ROR_RORRORK | 2 |
| ror | ROR_RORSDT3K | 4 |
| ror | ROR_RORSKUK | 5 |
| ror | ROR_RORSTSK | 5 |
| ror | ROR_RORUSTSK | 4 |
| ror | ROR_RORVENK | 7 |
| ror | ROR_RORVSTSK | 4 |
+------------+--------------+--------------+
23 rows in set (0.00 sec)
现在是 MySQL 8 的解释,带有更简单(但更慢)的查询,以及一些索引信息:
mysql> select version();
+-----------+
| version() |
+-----------+
| 8.0.23 |
+-----------+
1 row in set (0.00 sec)
mysql> explain select count(*) from ror where rorsts not in ( 'RECD', 'CANC', 'CCNS', 'SDNY' ) and ( ( rorbcd = '00009310022487' and rorfid = 'VDR' ) or ( rorfid = 'VDR' and rorbcd in ( select pplbcd from ppl where pplfid = 'VDR' and pplscb6 = '00009310022487' and pplsflg = 'Y' and pplsku = '0332690-008' and ppldoc = '73' and pplsca9 = '' and pplven = '10112' ) ) );
+----+-------------+-------+------------+------+----------------------------------------------------------------+--------------+---------+-------------------------+--------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+----------------------------------------------------------------+--------------+---------+-------------------------+--------+----------+-------------+
| 1 | PRIMARY | ror | NULL | ref | ROR_RORBCDK,ROR_RORSTSK,ROR_RORBSTSK,ROR_RORSDT3K,ROR_RORGMOQK | ROR_RORGMOQK | 5 | const | 185179 | 50.03 | Using where |
| 2 | SUBQUERY | ppl | NULL | ref | PPL_PPLBCDC,PPL_PPLDOCK,PPL_PPLVENC,PPL_PPLSKUC,PPL_PPLFID | PPL_PPLVENC | 53 | const,const,const,const | 2 | 2.50 | Using where |
+----+-------------+-------+------------+------+----------------------------------------------------------------+--------------+---------+-------------------------+--------+----------+-------------+
2 rows in set, 1 warning (0.02 sec)
mysql> show warnings;
+-------+------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note | 1003 | /* select#1 */ select count(0) AS `count(*)` from `veeder`.`ror` where ((`veeder`.`ror`.`RORSTS` not in ('RECD','CANC','CCNS','SDNY')) and (((`veeder`.`ror`.`RORFID` = 'VDR') and (`veeder`.`ror`.`RORBCD` = '00009310022487')) or ((`veeder`.`ror`.`RORFID` = 'VDR') and <in_optimizer>(`veeder`.`ror`.`RORBCD`,`veeder`.`ror`.`RORBCD` in ( <materialize> (/* select#2 */ select `veeder`.`ppl`.`PPLBCD` from `veeder`.`ppl` where ((`veeder`.`ppl`.`PPLVEN` = '10112') and (`veeder`.`ppl`.`PPLSCA9` = '') and (`veeder`.`ppl`.`PPLDOC` = '73') and (`veeder`.`ppl`.`PPLSKU` = '0332690-008') and (`veeder`.`ppl`.`PPLSFLG` = 'Y') and (`veeder`.`ppl`.`PPLSCB6` = '00009310022487') and (`veeder`.`ppl`.`PPLFID` = 'VDR')) ), <primary_index_lookup>(`veeder`.`ror`.`RORBCD` in <temporary table> on <auto_distinct_key> where ((`veeder`.`ror`.`RORBCD` = `<materialized_subquery>`.`pplbcd`)))))))) |
+-------+------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql>
mysql> select count(*) from ppl;
+----------+
| count(*) |
+----------+
| 94657 |
+----------+
1 row in set (0.49 sec)
mysql> explain select count(*) from ppl;
+----+-------------+-------+------------+-------+---------------+------------+---------+------+-------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+------------+---------+------+-------+----------+-------------+
| 1 | SIMPLE | ppl | NULL | index | NULL | PPL_PPLFID | 5 | NULL | 89284 | 100.00 | Using index |
+----+-------------+-------+------------+-------+---------------+------------+---------+------+-------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
mysql> show warnings;
+-------+------+------------------------------------------------------------------+
| Level | Code | Message |
+-------+------+------------------------------------------------------------------+
| Note | 1003 | /* select#1 */ select count(0) AS `count(*)` from `veeder`.`ppl` |
+-------+------+------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> SELECT TABLE_NAME, index_name, COUNT(1) column_count FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA = 'veeder' and table_name in ( 'ppl', 'ror' ) group by table_name, index_name order by table_name collate utf8_general_ci, index_name collate utf8_general_ci;
+------------+--------------+--------------+
| TABLE_NAME | INDEX_NAME | column_count |
+------------+--------------+--------------+
| ppl | PPLKEY | 1 |
| ppl | PPL_PPLBCDC | 2 |
| ppl | PPL_PPLDOCK | 3 |
| ppl | PPL_PPLFID | 1 |
| ppl | PPL_PPLPONOC | 4 |
| ppl | PPL_PPLSKUC | 4 |
| ppl | PPL_PPLVENC | 5 |
| ror | RORKEY | 1 |
| ror | ROR_RORBCDK | 5 |
| ror | ROR_RORBSTSK | 4 |
| ror | ROR_RORCNSK | 3 |
| ror | ROR_RORCRDTK | 4 |
| ror | ROR_RORCUIDK | 5 |
| ror | ROR_RORDOCK | 5 |
| ror | ROR_RORGMOQK | 4 |
| ror | ROR_RORPONOK | 6 |
| ror | ROR_RORRORK | 2 |
| ror | ROR_RORSDT3K | 4 |
| ror | ROR_RORSKUK | 5 |
| ror | ROR_RORSTSK | 5 |
| ror | ROR_RORUSTSK | 4 |
| ror | ROR_RORVENK | 7 |
| ror | ROR_RORVSTSK | 4 |
+------------+--------------+--------------+
23 rows in set (0.01 sec)
我不明白 MySql 8 给出的警告,但也许这是找到正确服务器设置的关键。
表定义和索引有点长,特别是对于两个数据库,但我确实有它们的副本,它们看起来几乎相同。最大的区别似乎是在 5.6 中我将一些整数列定义为“int(11)”,而在 8.0 中,它们被定义为“int”。在“show create table”输出中,索引是相同的。显然,“AUTO_INCREMENT”值是不同的,但这是有道理的,因为 MySql 8 版本已经过时了一周。希望这是足够的信息。当我在两个数据库服务器上执行“describe ppl”、“show index from ppl”、“describe ror”和“show index from ror”时,它们实际上是相同的,除了以下几点:
- 如上,
int(11)改为int。 - “基数”列发生变化。 MySQL 8 版本较少,可能是因为它已经过时了
- 显然,8.0“显示索引”命令显示更多列。 “可见”列始终为“YES”,“表达式”列始终为“NULL”。顺便说一下,INDEX_TYPE 对于所有索引都是“BTREE”。
因为这些表格有点大,show create table 和 describe <table>;show indexes from <table> 产生的结果对于帖子的正文来说太长了。因此,如果您需要更多信息,我需要知道您不需要我目前提供的内容。谢谢。
编辑:
在应用@o-jones 的建议后进行编辑:
下面的结果是我运行ANALYZE TABLE ror, ppl; 之后的结果,但它们与我运行ANALYZE TABLE ror, ppl; 之前发生的情况相匹配。
mysql> select version();
+-----------+
| version() |
+-----------+
| 8.0.23 |
+-----------+
1 row in set (0.00 sec)
mysql> explain select count(*) from ror where rorsts not in ( 'RECD', 'CANC', 'CCNS', 'SDNY' ) and ( ( rorbcd = '00009310022487' and rorfid = 'VDR' ) or ( rorfid = 'VDR' and rorbcd in ( select pplbcd from ppl where pplfid = 'VDR' and '00009310022487' and pplsflg = 'Y' and pplsku = '0332690-008' and ppldoc = '73' and pplsca9 = '' and pplven = '10112' ) ) );
+----+-------------+-------+------------+------+----------------------------------------------------------------+--------------+---------+-------------------------+--------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+----------------------------------------------------------------+--------------+---------+-------------------------+--------+----------+-------------+
| 1 | PRIMARY | ror | NULL | ref | ROR_RORBCDK,ROR_RORSTSK,ROR_RORBSTSK,ROR_RORSDT3K,ROR_RORGMOQK | ROR_RORGMOQK | 5 | const | 185179 | 50.03 | Using where |
| 2 | SUBQUERY | ppl | NULL | ref | PPL_PPLBCDC,PPL_PPLDOCK,PPL_PPLVENC,PPL_PPLSKUC,PPL_PPLFID | PPL_PPLVENC | 53 | const,const,const,const | 2 | 2.50 | Using where |
+----+-------------+-------+------------+------+----------------------------------------------------------------+--------------+---------+-------------------------+--------+----------+-------------+
2 rows in set, 1 warning (0.01 sec)
mysql> select count(*) from ror where rorsts not in ( 'RECD', 'CANC', 'CCNS', 'SDNY' ) and ( ( rorbcd = '00009310022487' and rorfid = 'VDR' ) or ( rorfid = 'VDR' and rorbcd in ( select pplbcd from ppl where pplfid = 'VDR' and pplscb610022487' and pplsflg = 'Y' and pplsku = '0332690-008' and ppldoc = '73' and pplsca9 = '' and pplven = '10112' ) ) );
+----------+
| count(*) |
+----------+
| 2 |
+----------+
1 row in set (0.80 sec)
mysql> explain select count(*) from ror USE INDEX(ROR_RORSTSK) where rorsts not in ( 'RECD', 'CANC', 'CCNS', 'SDNY' ) and ( ( rorbcd = '00009310022487' and rorfid = 'VDR' ) or ( rorfid = 'VDR' and rorbcd in ( select pplbcd from ppl wd = 'VDR' and pplscb6 = '00009310022487' and pplsflg = 'Y' and pplsku = '0332690-008' and ppldoc = '73' and pplsca9 = '' and pplven = '10112' ) ) );
+----+-------------+-------+------------+------+------------------------------------------------------------+-------------+---------+-------------------------+--------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+------------------------------------------------------------+-------------+---------+-------------------------+--------+----------+-------------+
| 1 | PRIMARY | ror | NULL | ALL | ROR_RORSTSK | NULL | NULL | NULL | 370358 | 50.03 | Using where |
| 2 | SUBQUERY | ppl | NULL | ref | PPL_PPLBCDC,PPL_PPLDOCK,PPL_PPLVENC,PPL_PPLSKUC,PPL_PPLFID | PPL_PPLVENC | 53 | const,const,const,const | 2 | 2.50 | Using where |
+----+-------------+-------+------------+------+------------------------------------------------------------+-------------+---------+-------------------------+--------+----------+-------------+
2 rows in set, 1 warning (0.00 sec)
mysql> select count(*) from ror USE INDEX(ROR_RORSTSK) where rorsts not in ( 'RECD', 'CANC', 'CCNS', 'SDNY' ) and ( ( rorbcd = '00009310022487' and rorfid = 'VDR' ) or ( rorfid = 'VDR' and rorbcd in ( select pplbcd from ppl where ppl' and pplscb6 = '00009310022487' and pplsflg = 'Y' and pplsku = '0332690-008' and ppldoc = '73' and pplsca9 = '' and pplven = '10112' ) ) );
+----------+
| count(*) |
+----------+
| 2 |
+----------+
1 row in set (0.25 sec)
mysql> explain select count(*) from ror where rorsts not in ( 'RECD', 'CANC', 'CCNS', 'SDNY' ) and ( ( rorbcd = '00009310022487' and rorfid = 'VDR' ) );+----+-------------+-------+------------+-------+----------------------------------------------------------------+--------------+---------+------+------+----------+--------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+----------------------------------------------------------------+--------------+---------+------+------+----------+--------------------------+
| 1 | SIMPLE | ror | NULL | range | ROR_RORBCDK,ROR_RORSTSK,ROR_RORBSTSK,ROR_RORSDT3K,ROR_RORGMOQK | ROR_RORBSTSK | 35 | NULL | 37 | 100.00 | Using where; Using index |
+----+-------------+-------+------------+-------+----------------------------------------------------------------+--------------+---------+------+------+----------+--------------------------+
1 row in set, 1 warning (0.00 sec)
mysql> select count(*) from ror where rorsts not in ( 'RECD', 'CANC', 'CCNS', 'SDNY' ) and ( ( rorbcd = '00009310022487' and rorfid = 'VDR' ) );
+----------+
| count(*) |
+----------+
| 2 |
+----------+
1 row in set (0.00 sec)
顺便说一句,在这种情况下,使用索引提示不是一个可接受的解决方案,因为这只是一个示例 SQL。我一直都知道我可以让这个特定的 SQL 更高效。我的问题是,对于相同的 SQL,MySql 8 应该更快,但速度要慢得多,因此试图找出原因。我怀疑在 MySql 5.7 中也会发生同样的事情,但我还没有尝试过,因为我听说 MySql 8 是要走的路。
【问题讨论】:
-
我只是在问为什么完全相同的 SQL 工作方式不同。我知道我可以通过多种方式改进这个特定的 SQL。有没有影响这一点的配置?
-
而且我们不能告诉你为什么相同的 sql 语句在两个 mysql 版本中运行不同,而没有看到你的数据结构和两个版本中的解释输出。相信我,mysql v8 中没有神奇的设置会突然导致查询运行速度变慢或变快!
-
v8 查询计划器与 v5.6 中的查询计划器完全不同。此外,您可能已经更改了存储引擎,甚至可能您的索引已更改。如果您的问题的答案是“将 framis 变量设置为 42”,那就太好了,但事情并不是那么简单,很遗憾。
-
请为每张桌子提供
SHOW CREATE TABLE。每个版本都有EXPLAIN SELECT ...。
标签: mysql query-optimization amazon-rds mysql-8.0 mysql-5.6