业务背景
订单服务和消息盒子服务使用典型的生产者消费者模式，
订单状态变更, 产生消息, 发往SQS队列,
消息盒子服务通过订阅SQS队列消费消息，更新DB中订单消息的状态。
出现死锁问题
消息盒子服务多个线程消费订单状态消息, update DB中对应的订单记录, 出现死锁。
update语句如下:
update msgbox_message set record_status = -1 where record_status = 0 and gmt_create >= now() - INTERVAL 3 MONTH and msg_key = ‘SO146213662’ and target = ‘201307438’.
看起来是很普通的update语句, 不应该出现死锁。
msgbox_message 表结构如下 (精简版)
CREATE TABLE msgbox_message (
id BIGINT ( 20 ) UNSIGNED AUTO_INCREMENT,
gmt_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
record_status TINYINT ( 3 ),
msg_key VARCHAR ( 64 ),
target VARCHAR ( 32 ) COMMENT \'uid\',
target_type TINYINT ( 3 ),
url VARCHAR ( 255 ),
PRIMARY KEY (id),
KEY target(target, target_type),
KEY msg_key(msg_key),
KEY gmt_create(gmt_create)
) ENGINE = INNODB
死锁日志分析
show engine innodb status的显示结果如下(删除了不必要的部分):
LATEST DETECTED DEADLOCK
2019-09-26 15:20:13
(1) TRANSACTION: TRANSACTION 149683649, thread id 659442, query id 956107029 IP1 ops_write updating
update msgbox_message set record_status = -1 where record_status = 0 and gmt_create >= now() - INTERVAL 3 MONTH and msg_key = ‘SO146213662’ and target = ‘201307438’
WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 118 page no 1399014 n bits 640 index target_idx of table cf_msgbox.msgbox_message;
trx id 149683649 lock_mode X locks rec but not gap waiting Record lock,
heap no 539 PHYSICAL RECORD: n_fields 4; compact format; info bits 0 ——(索引树的叶子节点数据)
0: len 9; hex 323031333037343338; asc 201307438;;
1: len 1; hex 00; asc ;;
2: len 8; hex 0000000004a135cb; asc 5 ;; ——(4a135cb是 16 进制的主键 ID 值77673931)
3: len 4; hex 5d8af714; asc ] ;;
..
(2) TRANSACTION: TRANSACTION 149683648, thread id 660492, query id 956107024 IP2 ops_write updating
update msgbox_message set record_status = -1 where record_status = 0 and gmt_create >= now() - INTERVAL 3 MONTH and msg_key = ‘SOxxxxxxx’ and target = ‘201307438’
HOLDS THE LOCK(S):
RECORD LOCKS space id 118 page no 1399014 n bits 640 index target_idx of table cf_msgbox.msgbox_message;
trx id 149683648 lock_mode X locks rec but not gap Record lock,
heap no 539 PHYSICAL RECORD: n_fields 4; compact format; info bits 0 ——(索引树的叶子节点数据)
0: len 9; hex 323031333037343338; asc 201307438;;
1: len 1; hex 00; asc ;;
2: len 8; hex 0000000004a135cb; asc 5 ;; ——(4a135cb是 16 进制的主键 ID 值77673931)
3: len 4; hex 5d8af714; asc ] ;;
WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 118 page no 1936298 n bits 152 index PRIMARY of table cf_msgbox.msgbox_message;
trx id 149683648 lock_mode X locks rec but not gap waiting Record lock,
heap no 13 PHYSICAL RECORD: ——(索引树的叶子节点数据)
n_fields 18; compact format; info bits 0
0: len 8; hex 0000000004a135cb; asc 5 ;;—— (4a135cb是 16 进制的主键 ID 值77673931)
1: len 4; hex 5d8af714; asc ] ;;
…….. 省略一些字段
WE ROLL BACK TRANSACTION (2)
我把部分关键的信息重点标注了出来，其中thread id是mysql的两个内部线程,
两个IP地址就是消息盒子服务机器的IP地址, 两条update语句只有where条件中的msg_key值不同。
- 事务 1 在等待 target_idx 索引树 (index target_idx) 中叶子节点 (heap no 539 PHYSICAL RECORD) 的 X 锁(互斥锁)—标记为 1 号锁