Yolov3 中作者列出了使用coco数据集训练的数据结果,其中显示在输入图片大小为416x416的时候,模型的mAP大小为31%,当输入图片大小为608x608的时候,模型的mAP达到33%,论文中结果截图如下:
我们做了个实验,目的是验证当我们自己训练的时候能否达到论文中的给出的结果,判断我们平时训练的方法是否有问题。
这里使用了3块TITAN X显卡进行实验,batch设置为128,subdivision为32,其他max_batches没有改,按原来的设为500200,其他的不变。计算mAP使用的是cocoAPI(估计作者也是用的这个,使用detector valid这个命令可以直接将测试结果生成为cocoAPI所需要的json格式,thresh设置为0.001)
1、前期训练:
./darknet detector train cfg/coco.data cfg/yolov3.cfg darknet53.conv.74 -gpus 0,1,2 | tee -a yolov3-coco-log.txt
PS: 保存log会拖慢训练的速度
然后提取log中的loss,解析并画出loss的曲线,曲线如下:
可以看到训练到后面的时候loss 基本就不在降低了,但是模型的mAP 只有25左右,远远还没达到论文中写的31%
2、不保存log,边训练边测试mAP
测试的结果如下:
| Steps | AP | AP50 | AP75 | APS | APM | APL |
| 105700 | 26 | 50.1 | 24.8 | 9.3 | 27.3 | 40.5 |
| 120000 | 26.7 | 50 | 26.6 | 9.9 | 28 | 41.3 |
| 123000 | 26.4 | 50.3 | 25.4 | 9.3 | 28 | 40.5 |
| 126000 | 26.9 | 50.7 | 26.3 | 10 | 28.8 | 41.3 |
| 129000 | 26.1 | 49.3 | 25.2 | 9.6 | 28 | 39.7 |
| 132000 | 26.7 | 50.5 | 25.8 | 10.6 | 28.7 | 39.9 |
| 135000 | 26.6 | 50.8 | 25.4 | 9.8 | 28 | 41.3 |
| 136600 | 25.8 | 48.5 | 24.9 | 8.9 | 27.8 | 40.1 |
| 139300 | 26.8 | 50.7 | 25.9 | 9.9 | 28.7 | 41.8 |
| 139500 | 26.6 | 50.2 | 25.8 | 10 | 27.5 | 42.3 |
| 141000 | 27.2 | 50.9 | 26.8 | 10.7 | 28.6 | 42 |
| 144500 | 26.2 | 51.5 | 24.2 | 11 | 28.5 | 39.1 |
| 148500 | 27.1 | 51.1 | 26.3 | 10.2 | 29.1 | 41.4 |
| 152500 | 26.7 | 49.8 | 26 | 10.4 | 27.2 | 42.5 |
| 156500 | 26.9 | 50.8 | 25.8 | 10.4 | 27.8 | 42.2 |
| 160500 | 26.7 | 49.9 | 26.3 | 9.8 | 28.1 | 42 |
| 164500 | 27.3 | 51.3 | 26.9 | 10.3 | 29.7 | 41.2 |
| 168500 | 27.7 | 51.4 | 27.7 | 10.4 | 30.1 | 41.5 |
| 172500 | 27.7 | 51.3 | 27.5 | 10.7 | 29.7 | 42.2 |
| 176500 | 27.3 | 51.7 | 26.6 | 10.5 | 30 | 41 |
| 180500 | 27.5 | 51.6 | 26.7 | 10.3 | 29.5 | 42.4 |
| 184500 | 27.9 | 51.2 | 27.5 | 9.9 | 30.2 | 42.9 |
| 188500 | 26.9 | 50.5 | 26.2 | 10 | 28.8 | 40.9 |
| 192500 | 28 | 52.2 | 27.5 | 11 | 29.4 | 42.1 |
| 196500 | 26.9 | 50.2 | 26.2 | 10.5 | 29.4 | 40.5 |
| 200500 | 27.1 | 50.7 | 27 | 9.9 | 29.8 | 41.1 |
| 204500 | 27.4 | 51.8 | 26.7 | 10.4 | 29.4 | 41.7 |
| 208500 | 27.1 | 50.5 | 26.2 | 10.7 | 28.9 | 41.8 |
| 212500 | 26.9 | 51.5 | 26 | 11 | 28.8 | 40.8 |
| 216500 | 28 | 51.5 | 28.1 | 10.9 | 30.1 | 41.7 |
| 220500 | 27.3 | 50.2 | 27 | 10.2 | 29.9 | 41.7 |
| 224500 | 27.6 | 51.8 | 27.2 | 11 | 30.1 | 41.9 |
| 228500 | 28.4 | 52.2 | 28.1 | 10.7 | 31 | 43.1 |
| 232500 | 27.8 | 51.4 | 27.6 | 10.8 | 30.4 | 41.8 |
| 236500 | 28.3 | 52.1 | 28.2 | 11.2 | 30.8 | 42.6 |
| 240500 | 28.1 | 51.1 | 28.1 | 11 | 30.8 | 42.3 |
| 244500 | 27 | 50.2 | 26.7 | 10 | 29.2 | 41.4 |
| 248500 | 27.9 | 52.2 | 27.3 | 11.4 | 29.3 | 42.4 |
| 252500 | 27.4 | 52.1 | 26.3 | 11.3 | 30 | 41.3 |
| 256500 | 27.9 | 52.1 | 27.6 | 9.8 | 29.1 | 44.8 |
| 262500 | 28.2 | 51.5 | 28.4 | 11.1 | 30.9 | 43.1 |
| 266500 | 28.3 | 52.2 | 27.8 | 11.2 | 30.2 | 43.6 |
| 270500 | 28.5 | 52.1 | 28.5 | 11.6 | 30.5 | 43.2 |
| 274500 | 27.9 | 51.4 | 27.6 | 11.1 | 29.3 | 43.4 |
| 278500 | 28.7 | 51.9 | 29 | 10.9 | 31.4 | 43.4 |
| 282500 | 27.7 | 51.2 | 27.2 | 11 | 30.2 | 42.1 |
| 286500 | 29 | 52.4 | 29.3 | 11.4 | 31 | 44.3 |
| 290500 | 28.1 | 51.5 | 27.6 | 11.2 | 30.7 | 43.2 |
| 294500 | 28.6 | 52.6 | 28.7 | 12.2 | 30.9 | 42.4 |
| 298500 | 27.5 | 50.8 | 27.1 | 10.6 | 30.2 | 41.8 |
| 302500 | 28.6 | 52 | 28.7 | 12.2 | 30.8 | 42.8 |
| 306500 | 28.2 | 51 | 28.6 | 11.2 | 30.7 | 42.6 |
| 310500 | 28.1 | 52.5 | 27.8 | 11.3 | 30.5 | 42.7 |
| 314500 | 28.6 | 52.5 | 28.3 | 11.9 | 30.7 | 44.1 |
| 317500 | 28.1 | 51.4 | 28.2 | 11.4 | 30.5 | 42.5 |
| 323500 | 28.7 | 51.9 | 29.1 | 11.8 | 31.3 | 43.6 |
| 326500 | 27.3 | 50.5 | 27.5 | 10.6 | 30.2 | 41.3 |
| 329500 | 28.9 | 52 | 29.3 | 11.3 | 30.6 | 45.1 |
| 332500 | 29 | 52.8 | 29.2 | 11.1 | 31.2 | 44.6 |
| 335500 | 28.6 | 52 | 29 | 10.9 | 31.8 | 43.6 |
| 338500 | 28.9 | 53.1 | 28.9 | 12.2 | 30.8 | 44.1 |
| 341500 | 28.9 | 52.2 | 29 | 11 | 30.9 | 44.3 |
| 344500 | 29.6 | 53 | 30 | 11.9 | 32.1 | 44.1 |
| 347000 | 28.6 | 52.8 | 28.2 | 10.8 | 31.5 | 43.9 |
| 351000 | 28.6 | 52 | 28.9 | 11.2 | 31.3 | 43.4 |
| 355000 | 28.7 | 53.2 | 28.1 | 11.6 | 30.4 | 43.4 |
| 359000 | 29.3 | 53.3 | 29.3 | 13 | 30.8 | 44 |
| 363000 | 28.1 | 51.3 | 27.8 | 11.1 | 30.2 | 42.5 |
| 367000 | 29.5 | 52.6 | 29.9 | 11.7 | 31.7 | 44.1 |
| 369500 | 28.7 | 52.8 | 28.8 | 11.3 | 31.7 | 43.4 |
| 372500 | 28 | 50.6 | 28.5 | 9.8 | 31.1 | 43.5 |
| 375500 | 28.8 | 52.2 | 29.3 | 12.3 | 31.2 | 43.2 |
| 378500 | 29.2 | 53.1 | 29.4 | 11.9 | 31 | 44.4 |
| 381500 | 28.1 | 50.8 | 28.3 | 10.9 | 30.8 | 42.2 |
| 384500 | 28.8 | 52.8 | 28.9 | 11.4 | 31.7 | 43.1 |
| 387500 | 28.1 | 52.8 | 28.2 | 11.5 | 31.6 | 41.1 |
| 390500 | 29.4 | 52.6 | 30 | 11 | 32.1 | 45 |
| 393500 | 29.1 | 52.6 | 29.3 | 11.5 | 31.5 | 44.6 |
| 396500 | 27.8 | 50.8 | 27.9 | 11 | 29.8 | 42 |
| 399500 | 28.2 | 51.5 | 28.2 | 10.7 | 31.3 | 42.4 |
| 402500 | 32.4 | 55.7 | 33.9 | 13.6 | 35.2 | 48.4 |
由结果可以看到loss其实还是慢慢在降的,只是降的幅度较小,作者采用的是迂回往前的方式,慢慢爬出来的,并且可以看到基本在320000 iteration的时候,mAP就基本不在升高了,所以yolov3.cfg中作者设置在400000 的时候 learning rate 乘0.1,之后可以看到mAP一下就升高到32.4%。
以下是结果的对比:
|
|
Steps |
AP |
AP50 |
AP75 |
APS |
APM |
APL |
|
Paper |
/ |
31 |
/ |
/ |
/ |
/ |
/ |
|
Paper 608x608 |
/ |
33 |
57.9 |
34.4 |
18.3 |
35.4 |
41.9 |
|
Author |
/ |
31.6 |
56.3 |
31.8 |
14.3 |
34.3 |
46.6 |
|
416x416 |
402500 |
32.4 |
55.7 |
33.9 |
13.6 |
35.2 |
48.4 |
|
608x608 |
402500 |
34.3 |
58.1 |
36.3 |
18.7 |
37.7 |
46.0 |