为什么并行执行查询时 MySQL 的性能会下降？答案

【问题标题】：Why does MySQL's performance decrease when queries are executed in parallel?为什么并行执行查询时 MySQL 的性能会下降？
【发布时间】：2012-06-25 00:11:10
【问题描述】：

问题：为什么在并行执行连接几乎空表的查询时 MySQL 性能会下降？

下面是对我所面临问题的更详细说明。我在 MySQL 中有两个表

CREATE TABLE first (
    num int(10) NOT NULL,
    UNIQUE KEY key_num (num)
) ENGINE=InnoDB

CREATE TABLE second (
    num int(10) NOT NULL,
    num2 int(10) NOT NULL,
    UNIQUE KEY key_num (num, num2)
) ENGINE=InnoDB

第一个包含大约一千条记录。第二个是空的或包含很少的记录。它还包含与问题有关的双索引：单索引问题就消失了。现在我正在尝试对这些表并行进行许多相同的查询。每个查询如下所示：

SELECT first.num
FROM first
LEFT JOIN second AS second_1 ON second_1.num = -1 # non-existent key
LEFT JOIN second AS second_2 ON second_2.num = -2 # non-existent key
LEFT JOIN second AS second_3 ON second_3.num = -3 # non-existent key
LEFT JOIN second AS second_4 ON second_4.num = -4 # non-existent key
LEFT JOIN second AS second_5 ON second_5.num = -5 # non-existent key
LEFT JOIN second AS second_6 ON second_6.num = -6 # non-existent key
WHERE second_1.num IS NULL
  AND second_2.num IS NULL
  AND second_3.num IS NULL
  AND second_4.num IS NULL
  AND second_5.num IS NULL
  AND second_6.num IS NULL

我遇到的问题是，在 8 核机器上，性能几乎没有线性提升，而是实际上有所下降。即有一个进程，我每秒的典型请求数约为 200。有两个进程而不是预期的每秒增加 300 - 400 个查询，我实际上下降到 150 个。对于 10 个进程，我只有 70 个查询每秒。我用于测试的 Perl 代码如下所示：

#!/usr/bin/perl

use strict;
use warnings;

use DBI;
use Parallel::Benchmark;
use SQL::Abstract;
use SQL::Abstract::Plugin::InsertMulti;

my $children_dbh;

foreach my $second_table_row_count (0, 1, 1000) {
    print '#' x 80, "\nsecond_table_row_count = $second_table_row_count\n";
    create_and_fill_tables(1000, $second_table_row_count);
    foreach my $concurrency (1, 2, 3, 4, 6, 8, 10, 20) {
        my $bm = Parallel::Benchmark->new(
            'benchmark' => sub {
                _run_sql();
                return 1;
            },
            'concurrency' => $concurrency,
            'time' => 3,
        );
        my $result = $bm->run();
    }
}

sub create_and_fill_tables {
    my ($first_table_row_count, $second_table_row_count) = @_;
    my $dbh = dbi_connect();
    {
        $dbh->do(q{DROP TABLE IF EXISTS first});
        $dbh->do(q{
            CREATE TABLE first (
                num int(10) NOT NULL,
                UNIQUE KEY key_num (num)
            ) ENGINE=InnoDB
        });
        if ($first_table_row_count) {
            my ($stmt, @bind) = SQL::Abstract->new()->insert_multi(
                'first',
                ['num'],
                [map {[$_]} 1 .. $first_table_row_count],
            );
            $dbh->do($stmt, undef, @bind);
        }
    }
    {
        $dbh->do(q{DROP TABLE IF EXISTS second});
        $dbh->do(q{
            CREATE TABLE second (
                num int(10) NOT NULL,
                num2 int(10) NOT NULL,
                UNIQUE KEY key_num (num, num2)
            ) ENGINE=InnoDB
        });
        if ($second_table_row_count) {
            my ($stmt, @bind) = SQL::Abstract->new()->insert_multi(
                'second',
                ['num'],
                [map {[$_]} 1 .. $second_table_row_count],
            );
            $dbh->do($stmt, undef, @bind);
        }
    }
}

sub _run_sql {
    $children_dbh ||= dbi_connect();
    $children_dbh->selectall_arrayref(q{
        SELECT first.num
        FROM first
        LEFT JOIN second AS second_1 ON second_1.num = -1
        LEFT JOIN second AS second_2 ON second_2.num = -2
        LEFT JOIN second AS second_3 ON second_3.num = -3
        LEFT JOIN second AS second_4 ON second_4.num = -4
        LEFT JOIN second AS second_5 ON second_5.num = -5
        LEFT JOIN second AS second_6 ON second_6.num = -6
        WHERE second_1.num IS NULL
          AND second_2.num IS NULL
          AND second_3.num IS NULL
          AND second_4.num IS NULL
          AND second_5.num IS NULL
          AND second_6.num IS NULL
    });
}

sub dbi_connect {
    return DBI->connect(
        'dbi:mysql:'
            . 'database=tmp'
            . ';host=localhost'
            . ';port=3306',
        'root',
        '',
    );
}

对于这样的比较查询，并发执行并提高性能：

SELECT first.num
FROM first
LEFT JOIN second AS second_1 ON second_1.num = 1 # existent key
LEFT JOIN second AS second_2 ON second_2.num = 2 # existent key
LEFT JOIN second AS second_3 ON second_3.num = 3 # existent key
LEFT JOIN second AS second_4 ON second_4.num = 4 # existent key
LEFT JOIN second AS second_5 ON second_5.num = 5 # existent key
LEFT JOIN second AS second_6 ON second_6.num = 6 # existent key
WHERE second_1.num IS NOT NULL
  AND second_2.num IS NOT NULL
  AND second_3.num IS NOT NULL
  AND second_4.num IS NOT NULL
  AND second_5.num IS NOT NULL
  AND second_6.num IS NOT NULL

测试结果、cpu 和磁盘使用测量在这里：

* 表 `first` 有 1000 行 * 表 `second` 有 6 行：`[1,1],[2,2],..[6,6]` 查询： SELECT first.num 从第一 LEFT JOIN second AS second_1 ON second_1.num = -1 # 不存在的键 LEFT JOIN second AS second_2 ON second_2.num = -2 # 不存在的键 LEFT JOIN second AS second_3 ON second_3.num = -3 # 不存在的键 LEFT JOIN second AS second_4 ON second_4.num = -4 # 不存在的键 LEFT JOIN second AS second_5 ON second_5.num = -5 # 不存在的键 LEFT JOIN second AS second_6 ON second_6.num = -6 # 不存在的键在哪里 second_1.num 为空 AND second_2.num 为空 AND second_3.num 为空 AND second_4.num 为空 AND second_5.num 为空 AND second_6.num 为空结果：并发：1，速度：162.910/秒并发：2，速度：137.818/秒并发：3，速度：130.728/秒并发：4，速度：107.387/秒并发：6，速度：90.513/秒并发：8，速度：80.445/秒并发：10，速度：80.381/秒并发：20，速度：84.069/秒在 6 个进程中运行查询的最后 60 分钟后的系统使用情况： $ iostat -cdkx 60 平均 CPU：%user %nice %system %iowait %steal %idle 74.82 0.00 0.08 0.00 0.08 25.02 设备：rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util sda1 0.00 0.00 0.00 0.12 0.00 0.80 13.71 0.00 1.43 1.43 0.02 sdf10 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 sdf4 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 30.00 15.00 0.05 sdm 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf8 0.00 0.00 0.00 0.37 0.00 1.24 6.77 0.00 5.00 3.18 0.12 sdf6 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 sdf9 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 0.00 0.00 0.00 自卫队 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf3 0.00 0.00 0.00 0.08 0.00 1.33 32.00 0.00 4.00 4.00 0.03 sdf2 0.00 0.00 0.00 0.17 0.00 1.37 16.50 0.00 3.00 3.00 0.05 sdf15 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf14 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf1 0.00 0.00 0.00 0.05 0.00 0.40 16.00 0.00 0.00 0.00 0.00 sdf13 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 sdf5 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 50.00 25.00 0.08 sdm2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdm1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf12 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 sdf11 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 sdf7 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 md0 0.00 0.00 0.00 0.97 0.00 13.95 28.86 0.00 0.00 0.00 0.00 ################################################# ############################## 查询： SELECT first.num 从第一 LEFT JOIN second AS second_1 ON second_1.num = 1 # 存在键 LEFT JOIN second AS second_2 ON second_2.num = 2 # 存在键 LEFT JOIN second AS second_3 ON second_3.num = 3 # 存在键 LEFT JOIN second AS second_4 ON second_4.num = 4 # 存在键 LEFT JOIN second AS second_5 ON second_5.num = 5 # 存在键 LEFT JOIN second AS second_6 ON second_6.num = 6 # 存在键在哪里 second_1.num 不为空 AND second_2.num 不为空 AND second_3.num 不为空并且 second_4.num 不为空并且 second_5.num 不为空并且 second_6.num 不为空结果：并发：1，速度：875.973/秒并发：2，速度：944.986/秒并发：3，速度：1256.072/秒并发：4，速度：1401.657/秒并发：6，速度：1354.351/秒并发：8，速度：1110.100/秒并发：10，速度：1145.251/秒并发：20，速度：1142.514/秒在 6 个进程中运行查询的最后 60 分钟后的系统使用情况： $ iostat -cdkx 60 平均 CPU：%user %nice %system %iowait %steal %idle 74.40 0.00 0.53 0.00 0.06 25.01 设备：rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util sda1 0.00 0.00 0.00 0.02 0.00 0.13 16.00 0.00 0.00 0.00 0.00 sdf10 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 sdf4 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 sdm 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf8 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 sdf6 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 0.00 0.00 0.00 sdf9 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 自卫队 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf3 0.00 0.00 0.00 0.13 0.00 2.67 40.00 0.00 3.75 2.50 0.03 sdf2 0.00 0.00 0.00 0.23 0.00 2.72 23.29 0.00 2.14 1.43 0.03 sdf15 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf14 0.00 0.00 0.00 0.98 0.00 0.54 1.10 0.00 2.71 2.71 0.27 sdf1 0.00 0.00 0.00 0.08 0.00 1.47 35.20 0.00 8.00 6.00 0.05 sdf13 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf5 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 sdm2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdm1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf12 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdf11 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 0.00 0.00 0.00 sdf7 0.00 0.00 0.00 0.03 0.00 1.07 64.00 0.00 10.00 5.00 0.02 md0 0.00 0.00 0.00 1.70 0.00 15.92 18.74 0.00 0.00 0.00 0.00 ################################################# ############################## 这个服务器有很多可用内存。顶部示例： top - 19:02:59 up 4:23, 4 users, load average: 4.43, 3.03, 2.01 任务：共 218 个，运行 1 个，睡眠 217 个，停止 0 个，僵尸 0 个 CPU：72.8%us、0.7%sy、0.0%ni、26.3%id、0.0%wa、0.0%hi、0.0%si、0.1%st 内存：总计 71701416k，已使用 22183980k，可用 49517436k，284k 缓冲区交换：总共 0k，使用 0k，免费 0k，缓存 1282768k PID 用户 PR NI VIRT RES SHR S %CPU %MEM TIME+ 命令 2506 mysql 20 0 51.7g 17g 5920 S 590 25.8 213:15.12 mysqld 9348 topadver 20 0 72256 11m 1428 S 2 0.0 0:01.45 perl 9349 topadver 20 0 72256 11m 1428 S 2 0.0 0:01.44 perl 9350 topadver 20 0 72256 11m 1428 S 2 0.0 0:01.45 perl 9351 topadver 20 0 72256 11m 1428 S 1 0.0 0:01.44 perl 9352 topadver 20 0 72256 11m 1428 S 1 0.0 0:01.44 perl 9353 topadver 20 0 72256 11m 1428 S 1 0.0 0:01.44 perl 9346 顶部 20 0 19340 1504 1064 R 0 0.0 0:01.89 顶部

有人知道为什么使用不存在的键进行查询会降低性能吗？

【问题讨论】：

如果您的创建表已经具有 NOT NULL 条件，为什么还要使用 'where num.... is null'？
@jcho360 左连接会创建这样的空值。这似乎是配置。 Awayka，你能提供一些关于你的 MYSQL 服务器的信息吗？它有多个处理器吗？
@Twelfth mysql Ver 14.14 Distrib 5.1.59，适用于在具有 8 个内核的 m2.4xlarge EC2 实例上使用 readline 5.1 的 debian-linux-gnu (x86_64)
我建议查看不运行此查询与运行一个查询与并行运行多个查询时系统上非 CPU 资源的负载情况。您的查询可能正在使用大量内存，导致您在多个查询上命中交换。或者您的磁盘/磁盘总线/内存总线可能是瓶颈。如果你不能足够快地提供 CPU 的数据，并行查询将无济于事。
@Ven'Tatsu 我可以从我的测量中得出结论，这些资源不存在瓶颈。内存有很大的余量——这是肯定的。类似但略有不同的查询按预期执行，我在上面添加了此信息。我将案例的磁盘使用情况与“好”查询和“坏”查询进行了比较，发现没有区别。因此，如果可以，请阅读我的结果，也许您可以找到一些东西，或者给我一些建议以及我可以测量的方法。

标签： mysql performance perl

【解决方案1】：

写得很好，这表明了一些研究。

出于好奇，我尝试了 MySQL 5.6，看看那里的工具对这些查询有什么看法。

首先，请注意查询是不同的：

将存在/不存在的值从“1”更改为“-1” 钥匙包是一回事
将“second_1.num IS NOT NULL”更改为 WHERE 子句中的“second_1.num IS NULL”是另一个。

使用 EXPLAIN 给出不同的计划：

EXPLAIN SELECT `first`.num
FROM `first`
LEFT JOIN `second` AS second_1 ON second_1.num = -1 # non-existent key
LEFT JOIN `second` AS second_2 ON second_2.num = -2 # non-existent key
LEFT JOIN `second` AS second_3 ON second_3.num = -3 # non-existent key
LEFT JOIN `second` AS second_4 ON second_4.num = -4 # non-existent key
LEFT JOIN `second` AS second_5 ON second_5.num = -5 # non-existent key
LEFT JOIN `second` AS second_6 ON second_6.num = -6 # non-existent key
WHERE second_1.num IS NULL
AND second_2.num IS NULL
AND second_3.num IS NULL
AND second_4.num IS NULL
AND second_5.num IS NULL
AND second_6.num IS NULL
;
id      select_type     table   type    possible_keys   key     key_len ref     rows    Extra
1       SIMPLE  first   index   NULL    key_num 4       NULL    1000    Using index
1       SIMPLE  second_1        ref     key_num key_num 4       const   1       Using where; Not exists; Using index
1       SIMPLE  second_2        ref     key_num key_num 4       const   1       Using where; Not exists; Using index
1       SIMPLE  second_3        ref     key_num key_num 4       const   1       Using where; Not exists; Using index
1       SIMPLE  second_4        ref     key_num key_num 4       const   1       Using where; Not exists; Using index
1       SIMPLE  second_5        ref     key_num key_num 4       const   1       Using where; Not exists; Using index
1       SIMPLE  second_6        ref     key_num key_num 4       const   1       Using where; Not exists; Using index

相对

EXPLAIN SELECT `first`.num
FROM `first`
LEFT JOIN `second` AS second_1 ON second_1.num = 1 # existent key
LEFT JOIN `second` AS second_2 ON second_2.num = 2 # existent key
LEFT JOIN `second` AS second_3 ON second_3.num = 3 # existent key
LEFT JOIN `second` AS second_4 ON second_4.num = 4 # existent key
LEFT JOIN `second` AS second_5 ON second_5.num = 5 # existent key
LEFT JOIN `second` AS second_6 ON second_6.num = 6 # existent key
WHERE second_1.num IS NOT NULL
AND second_2.num IS NOT NULL
AND second_3.num IS NOT NULL
AND second_4.num IS NOT NULL
AND second_5.num IS NOT NULL
AND second_6.num IS NOT NULL
;
id      select_type     table   type    possible_keys   key     key_len ref     rows    Extra
1       SIMPLE  second_1        ref     key_num key_num 4       const   1       Using index
1       SIMPLE  second_2        ref     key_num key_num 4       const   1       Using index
1       SIMPLE  second_3        ref     key_num key_num 4       const   1       Using index
1       SIMPLE  second_4        ref     key_num key_num 4       const   1       Using index
1       SIMPLE  second_5        ref     key_num key_num 4       const   1       Using index
1       SIMPLE  second_6        ref     key_num key_num 4       const   1       Using index
1       SIMPLE  first   index   NULL    key_num 4       NULL    1000    Using index; Using join buffer (Block Nested Loop)

使用 JSON 格式，我们有：

EXPLAIN FORMAT=JSON SELECT `first`.num
FROM `first`
LEFT JOIN `second` AS second_1 ON second_1.num = -1 # non-existent key
LEFT JOIN `second` AS second_2 ON second_2.num = -2 # non-existent key
LEFT JOIN `second` AS second_3 ON second_3.num = -3 # non-existent key
LEFT JOIN `second` AS second_4 ON second_4.num = -4 # non-existent key
LEFT JOIN `second` AS second_5 ON second_5.num = -5 # non-existent key
LEFT JOIN `second` AS second_6 ON second_6.num = -6 # non-existent key
WHERE second_1.num IS NULL
AND second_2.num IS NULL
AND second_3.num IS NULL
AND second_4.num IS NULL
AND second_5.num IS NULL
AND second_6.num IS NULL
;
EXPLAIN
{
  "query_block": {
    "select_id": 1,
    "nested_loop": [
      {
        "table": {
          "table_name": "first",
          "access_type": "index",
          "key": "key_num",
          "key_length": "4",
          "rows": 1000,
          "filtered": 100,
          "using_index": true
        }
      },
      {
        "table": {
          "table_name": "second_1",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "not_exists": true,
          "using_index": true,
          "attached_condition": "<if>(found_match(second_1), isnull(`test`.`second_1`.`num`), true)"
        }
      },
      {
        "table": {
          "table_name": "second_2",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "not_exists": true,
          "using_index": true,
          "attached_condition": "<if>(found_match(second_2), isnull(`test`.`second_2`.`num`), true)"
        }
      },
      {
        "table": {
          "table_name": "second_3",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "not_exists": true,
          "using_index": true,
          "attached_condition": "<if>(found_match(second_3), isnull(`test`.`second_3`.`num`), true)"
        }
      },
      {
        "table": {
          "table_name": "second_4",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "not_exists": true,
          "using_index": true,
          "attached_condition": "<if>(found_match(second_4), isnull(`test`.`second_4`.`num`), true)"
        }
      },
      {
        "table": {
          "table_name": "second_5",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "not_exists": true,
          "using_index": true,
          "attached_condition": "<if>(found_match(second_5), isnull(`test`.`second_5`.`num`), true)"
        }
      },
      {
        "table": {
          "table_name": "second_6",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "not_exists": true,
          "using_index": true,
          "attached_condition": "<if>(found_match(second_6), isnull(`test`.`second_6`.`num`), true)"
        }
      }
    ]
  }
}

相对

EXPLAIN FORMAT=JSON SELECT `first`.num
FROM `first`
LEFT JOIN `second` AS second_1 ON second_1.num = 1 # existent key
LEFT JOIN `second` AS second_2 ON second_2.num = 2 # existent key
LEFT JOIN `second` AS second_3 ON second_3.num = 3 # existent key
LEFT JOIN `second` AS second_4 ON second_4.num = 4 # existent key
LEFT JOIN `second` AS second_5 ON second_5.num = 5 # existent key
LEFT JOIN `second` AS second_6 ON second_6.num = 6 # existent key
WHERE second_1.num IS NOT NULL
AND second_2.num IS NOT NULL
AND second_3.num IS NOT NULL
AND second_4.num IS NOT NULL
AND second_5.num IS NOT NULL
AND second_6.num IS NOT NULL
;
EXPLAIN
{
  "query_block": {
    "select_id": 1,
    "nested_loop": [
      {
        "table": {
          "table_name": "second_1",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "using_index": true
        }
      },
      {
        "table": {
          "table_name": "second_2",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "using_index": true
        }
      },
      {
        "table": {
          "table_name": "second_3",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "using_index": true
        }
      },
      {
        "table": {
          "table_name": "second_4",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "using_index": true
        }
      },
      {
        "table": {
          "table_name": "second_5",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "using_index": true
        }
      },
      {
        "table": {
          "table_name": "second_6",
          "access_type": "ref",
          "possible_keys": [
            "key_num"
          ],
          "key": "key_num",
          "key_length": "4",
          "ref": [
            "const"
          ],
          "rows": 1,
          "filtered": 100,
          "using_index": true
        }
      },
      {
        "table": {
          "table_name": "first",
          "access_type": "index",
          "key": "key_num",
          "key_length": "4",
          "rows": 1000,
          "filtered": 100,
          "using_index": true,
          "using_join_buffer": "Block Nested Loop"
        }
      }
    ]
  }
}

查看运行时性能模式检测的表 io，我们有：

truncate table performance_schema.objects_summary_global_by_type;
select * from performance_schema.objects_summary_global_by_type
where OBJECT_NAME in ("first", "second");
OBJECT_TYPE OBJECT_SCHEMA   OBJECT_NAME COUNT_STAR  SUM_TIMER_WAIT  MIN_TIMER_WAIT  AVG_TIMER_WAIT  MAX_TIMER_WAIT
TABLE   test    first   0   0   0   0   0
TABLE   test    second  0   0   0   0   0
SELECT `first`.num
FROM `first`
LEFT JOIN `second` AS second_1 ON second_1.num = -1 # non-existent key
LEFT JOIN `second` AS second_2 ON second_2.num = -2 # non-existent key
LEFT JOIN `second` AS second_3 ON second_3.num = -3 # non-existent key
LEFT JOIN `second` AS second_4 ON second_4.num = -4 # non-existent key
LEFT JOIN `second` AS second_5 ON second_5.num = -5 # non-existent key
LEFT JOIN `second` AS second_6 ON second_6.num = -6 # non-existent key
WHERE second_1.num IS NULL
AND second_2.num IS NULL
AND second_3.num IS NULL
AND second_4.num IS NULL
AND second_5.num IS NULL
AND second_6.num IS NULL
;
(...)
select * from performance_schema.objects_summary_global_by_type
where OBJECT_NAME in ("first", "second");
OBJECT_TYPE OBJECT_SCHEMA   OBJECT_NAME COUNT_STAR  SUM_TIMER_WAIT  MIN_TIMER_WAIT  AVG_TIMER_WAIT  MAX_TIMER_WAIT
TABLE   test    first   1003    5705014442  1026171 5687889 87356557
TABLE   test    second  6012    271786533972    537266  45207298    1123939292

相对于：

select * from performance_schema.objects_summary_global_by_type
where OBJECT_NAME in ("first", "second");
OBJECT_TYPE OBJECT_SCHEMA   OBJECT_NAME COUNT_STAR  SUM_TIMER_WAIT  MIN_TIMER_WAIT  AVG_TIMER_WAIT  MAX_TIMER_WAIT
TABLE   test    first   1003    5211074603  969338  5195454 61066176
TABLE   test    second  24  458656783   510085  19110361    66229860

可扩展的查询在表second 中几乎没有表IO。不伸缩的查询在表second做6K表IO，或者是表first大小的6倍。

这是因为查询计划不同，反过来又因为查询不同（IS NOT NULL 与 IS NULL）。

我认为这回答了与性能相关的问题。

请注意，在我的测试中，这两个查询都返回了 1000 行，这可能不是您想要的。在调整查询以使其更快之前，请确保它按预期工作。

【讨论】：

【解决方案2】：

我建议尝试一种方法，其中每个fork 使用自己的连接（在我看来，现在拥有数据库连接的$children_dbh 是一个共享变量）。或者，更好的是，实现所谓的connection pool，每个客户端进程将在需要时从中获取连接，并在不再需要时将其“归还”。

查看this answer 以获取更多详细信息：给出它的线程是关于 Java 的，但它实际上是关于 MySQL 组织的一些通用原则。 this answer 也可能有用。

附： here描述了有点类似的情况（我认为），并且有关于如何组织连接池的详细说明。

【讨论】：

这个字符串$children_dbh ||= dbi_connect(); 的_run_sql() 会做什么？
对我来说似乎没有线程：每个进程都有一个线程。你在哪里看到线程？ Parallel::Benchmark 使用分叉的Parallel::ForkManager。
我的意思是有一个共享连接，这解释了为什么每个新进程的性能实际上变得更差。而且，我再说一遍，很容易检查实际使用了多少与 DB 的连接。说“看起来像”并就理论进行辩论是没有意义的：要么使用单个连接，要么不使用。
它在创建连接之前分叉，因此每个进程都获得独立的连接。
@raina77ow Raina，我很抱歉成为这样的人，但这个答案实际上与问题无关。如果从源头不清楚没有共享连接，可以复制/粘贴并运行代码来获取它。