mysql select in 怎么优化

2017年7月10日 22:36:54 星期一

原理: 把select where in 变换成 "where = " 或者 "where between and " 这样的子句, 然后用union all 拼接, 一次查询

CRUD.php::selectIn()

环境(尽量模拟生产环境):

php7,

summerPHP框架,

代码部署在杭州(华东)阿里云的机器上,

mysql数据库部署在青岛(华北),

表的记录数量为8200+,

`id`是主键,`hanzi`不在索引中,要检索物理硬盘

每次id的生成都是随机数, 所以数据比较分散

结果:

用 union all 重新组装的sql查询总是慢......

思考,

可能是数据太过分散了, 导致生成了很多条子句, 没有太多的利用between and来减少查询, 但数据分散也许符合生产环境的特点吧

也许分布式计算下可以用到这个步骤, 将这些子的sql语句发送到计算集群中并行计算, 而不是在一台机器上计算好后union all

下边是in的数量为10, 50, 100, 200, 300的计算结果

有此优化想法的同学可以绕道了, 或者我这里有哪里不对的地方也请指出来, 让我哭一会儿先.....

测试代码:

 1         //随机数数组
 2         $rand = [];
 3         for ($i=0; $i<100; $i++) {
 4             $rand[] = mt_rand(1,8200);
 5         }
 6 
 7         $rs = Test::link('pinyin')->fields('id,hanzi')
 8             ->whereIn('id', $rand)
 9             ->select()//不改写select in
10             ->getAll();
11         echo Test::$sql,'<br>'; //打印出当前sql语句
12 
13         $rs = Test::link('pinyin')->fields('id,hanzi')
14             ->whereIn('id', $rand)
15             ->selectIn() //改写select in
16             ->getAll();
17 
18         echo Test::$sql,'<br>'; //打印当前的sql语句
19 
20         print_r(Timer::$list); //输出耗时统计

1. in的数量在10以下的时候, 直接in快一点

2. in的数量是50, union all 总是稍慢

 1     [5] => Array
 2         (
 3             [key] => SELECT id,hanzi FROM pinyin WHERE (id IN ( 417,670,874,969,1106,1130,1151,1223,1253,1351,1376,1384,1996,2435,2535,2547,2626,2695,2820,2883,3157,3576,3617,3780,3809,3814,4262,4413,4722,4888,5003,5041,5449,5715,5893,5929,6005,6029,6081,6126,6321,6477,6902,7035,7079,7143,7303,7417,7736,8073 ))    
 4             [start] => 1499698628.4481
 5             [over] => 1499698628.4756
 6             [cost] => 0.0275
 7         )
 8 
 9     [6] => Array
10         (
11             [key] => (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 417)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 670)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 874)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 969)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 1106)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 1130)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 1151)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 1223)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 1253)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 1351)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 1376)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 1384)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 1996)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 2435)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 2535)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 2547)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 2626)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 2695)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 2820)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 2883)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 3157)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 3576)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 3617)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 3780)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 3809)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 3814)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 4262)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 4413)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 4722)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 4888)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 5003)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 5041)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 5449)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 5715)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 5893)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 5929)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 6005)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 6029)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 6081)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 6126)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 6321)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 6477)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 6902)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 7035)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 7079)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 7143)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 7303)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 7417)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 7736)    ) UNION ALL (SELECT id,hanzi FROM pinyin WHERE 1 AND (id = 8073)    )
12             [start] => 1499698628.4759
13             [over] => 1499698628.5056
14             [cost] => 0.0297
15         )

in 50