为什么我的 UPDATE ... WHERE ... ORDER BY .. LIMIT 1 语句需要这么长时间？ [复制]

Question

我正在尝试改进我的查询，以便它不会花费这么长时间。有什么我可以尝试的吗？

我正在使用 InnoDB。

我的桌子：

mysql> describe hunted_place_review_external_urls;
+--------------+--------------+------+-----+---------+----------------+
| Field        | Type         | Null | Key | Default | Extra          |
+--------------+--------------+------+-----+---------+----------------+
| id           | bigint(20)   | NO   | PRI | NULL    | auto_increment |
| worker_id    | varchar(255) | YES  | MUL | NULL    |                |
| queued_at    | bigint(20)   | YES  | MUL | NULL    |                |
| external_url | varchar(255) | NO   |     | NULL    |                |
| place_id     | varchar(63)  | NO   | MUL | NULL    |                |
| source_id    | varchar(63)  | NO   |     | NULL    |                |
| successful   | tinyint(1)   | NO   |     | 0       |                |
+--------------+--------------+------+-----+---------+----------------+

mysql> show index from hunted_place_review_external_urls;
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table                             | Non_unique | Key_name                                   | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| hunted_place_review_external_urls |          0 | PRIMARY                                    |            1 | id           | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | worker_id                                  |            1 | worker_id    | A         |     5118685 |     NULL | NULL   | YES  | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | queued_at                                  |            1 | queued_at    | A         |     5118685 |     NULL | NULL   | YES  | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | worker_id_and_queued_at                    |            1 | worker_id    | A         |     5118685 |     NULL | NULL   | YES  | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | worker_id_and_queued_at                    |            2 | queued_at    | A         |     5118685 |     NULL | NULL   | YES  | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | place_id_source_id_external_url_successful |            1 | place_id     | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | place_id_source_id_external_url_successful |            2 | source_id    | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | place_id_source_id_external_url_successful |            3 | external_url | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | place_id_source_id_external_url_successful |            4 | successful   | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

我的查询：

mysql> select count(*) from hunted_place_review_external_urls;
+----------+
| count(*) |
+----------+
|  4217356 |
+----------+
1 row in set (0.96 sec)

mysql> select count(*) from hunted_place_review_external_urls where worker_id is null;
+----------+
| count(*) |
+----------+
|   772626 |
+----------+
1 row in set (0.27 sec)

mysql> update hunted_place_review_external_urls set worker_id = "123" where worker_id is null order by queued_at asc limit 1;
Query OK, 1 row affected (4.80 sec)
Rows matched: 1  Changed: 1  Warnings: 0

为什么即使我在queued_at 和worker_id 上都有单索引和复合索引，更新查询仍需要 4 秒？当 worker_id = null 的行数少得多时，这种情况以前从未发生过。使用约 200k 行而不是 780k 行，只需几毫秒。

请注意，使用 SELECT 而不是 UPDATE 的等效查询非常快：

mysql> select * from hunted_place_review_external_urls where worker_id is null order by  queued_at asc limit 1;
1 row in set (0.00 sec)

我的queued_at 值是以毫秒数表示的时间戳，例如1398210069531

我已经尝试在 worker_id 和 queued_at 上删除我的单个索引，但问题仍然存在：

mysql> drop index queued_at on hunted_place_review_external_urls;
Query OK, 0 rows affected (3.75 sec)

mysql> drop index worker_id on hunted_place_review_external_urls;
Query OK, 0 rows affected (3.75 sec)

mysql> show index from hunted_place_review_external_urls;
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table                             | Non_unique | Key_name                                   | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| hunted_place_review_external_urls |          0 | PRIMARY                                    |            1 | id           | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | worker_id_and_queued_at                    |            1 | worker_id    | A         |     5118685 |     NULL | NULL   | YES  | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | worker_id_and_queued_at                    |            2 | queued_at    | A         |     5118685 |     NULL | NULL   | YES  | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | place_id_source_id_external_url_successful |            1 | place_id     | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | place_id_source_id_external_url_successful |            2 | source_id    | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | place_id_source_id_external_url_successful |            3 | external_url | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
| hunted_place_review_external_urls |          1 | place_id_source_id_external_url_successful |            4 | successful   | A         |     5118685 |     NULL | NULL   |      | BTREE      |         |               |
+-----------------------------------+------------+--------------------------------------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

这是我的EXPLAIN SELECT 声明。我使用的是不支持EXPLAIN UPDATE的旧版MYSQL：

mysql> explain select * from hunted_place_review_external_urls where worker_id is null order by queued_at asc limit 1;
+----+-------------+-----------------------------------+------+-------------------------+-------------------------+---------+-------+---------+-------------+
| id | select_type | table                             | type | possible_keys           | key                     | key_len | ref   | rows    | Extra       |
+----+-------------+-----------------------------------+------+-------------------------+-------------------------+---------+-------+---------+-------------+
|  1 | SIMPLE      | hunted_place_review_external_urls | ref  | worker_id_and_queued_at | worker_id_and_queued_at | 768     | const | 1587282 | Using where |
+----+-------------+-----------------------------------+------+-------------------------+-------------------------+---------+-------+---------+-------------+
1 row in set (0.00 sec)

Answer 1

比较Drop Index、Create Index 和update 的时间。 您可能会注意到相关性。

• 当您简单地执行SELECT 查询时，索引是有用的，因为它们会加快速度。

• 当您执行 UPDATE 或 DELETE 语句时 - 索引不好，因为它们会减慢速度！每当您更改索引列的值时，MySQL 都需要为任何后续行重建索引。 （假设您总是获取最旧的条目 - 这意味着：重新索引 ALL reamaining 772625 Rows。）

尝试删除worker_id 上的索引并查看更新性能。如果 worker_id 没有被索引，更新会更快。 （查找条目 to 更新仍然和以前一样快，因为它主要取决于对索引列 queued_at 和 @987654330 上的一小部分未索引的 null 值执行的排序@，匹配所需的queued_at 值）

---

我刚刚创建了一个虚拟数据库并测试了您的设置： 有 1.000.000 行和两者 - worker_id 上的单个索引和 worker_id|queued at 上的复合索引，选择看起来像：

SELECT * FROM `tbl` WHERE ISNULL( worker_ID ) ORDER BY queued_at ASC LIMIT 1 

和性能：

Query took 0.3360 sec

尝试以您的方式修改worker_id，结果：

UPDATE `tbl` SET worker_id=1 WHERE ISNULL(worker_ID) ORDER BY queued_at ASC LIMIT 1

性能：

1 row affected. (Query took 7.9592 sec)

删除worker_id 上的两个索引（单个和复合），然后相同的查询结果：

1 row affected. (Query took 1.4364 sec) 

（每次插入生成 50.000 行，因此它们具有相同的日期，因此索引对于搜索来说并不“完美”，因此“真实”数据的性能可能会更好。）

Answer 2

这是您的查询：

update hunted_place_review_external_urls
    set worker_id = "123"
    where worker_id is null
    order by queued_at asc
    limit 1;

它首先必须找到用于更新的行，这需要应用where 子句和order by 子句。它要么完成所有工作（扫描表然后排序），要么使用索引。正确的索引是hunted_place_review_external_urls(worker_id, queued_at)。您可以在最后添加更多列，但这些必须是前两列，并按此顺序。

编辑：

鉴于select 速度很快，试试这个版本：

update hunted_place_review_external_urls toupdate join
       (select
        from hunted_place_review_external_urls
        where worker_id is null
        order by queued_at asc
        limit 1
       ) l
       on toupdate.id = l.id
    set toupdate.worker_id = '123';

我不确定为什么索引会在此处正确使用，但不会在 update 中使用，但希望这会起作用。