在最后一个时期训练时出现断言错误答案

【问题标题】：Assertion Error while training on last epoch在最后一个时期训练时出现断言错误
【发布时间】：2020-08-04 11:25:40
【问题描述】：

我使用了来自this repo. 的模型，我想再次训练它以提高性能。这是我的训练命令。

!python train.py --snapshot efficientdet-d0.h5 --phi 0 --gpu 0 --weighted-bifpn --random-transform --compute-val-loss --freeze-bn --batch-size 4 --steps 100 coco datasets/coco

以及文件夹的目录：

前 99 个 epoch 运行良好。

99/100 [============================>.] - ETA: 0s - loss: 0.3978 - classification_loss: 0.2983 - regression_loss: 0.0995datasets/coco/images/train2017/i.rf.4eba631a9cf8fef57df45f64cf16f258.jpg
datasets/coco/images/train2017/i.rf.3bb1c054edd9e0b91f8944b797c28eac.jpg
datasets/coco/images/train2017/i.rf.8dba641d2187f073238f34fddd5499fc.jpg
datasets/coco/images/train2017/i.rf.bfd984b934688cdca720056e6733219d.jpg
datasets/coco/images/train2017/i.rf.2d0be24e1ac8afdd69ebb9cf03367be5.jpg
datasets/coco/images/train2017/i.rf.a6a1177aa50ae605fe184b766421d4c4.jpg
100/100 [==============================] - ETA: 0s - loss: 0.3959 - classification_loss: 0.2966 - regression_loss: 0.0993datasets/coco/images/val2017/000000061960.jpg

之后，我看到了这个错误：

Traceback (most recent call last):
  File "train.py", line 381, in <module>
    main()
  File "train.py", line 376, in main
    validation_data=validation_generator
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 1482, in fit_generator
    initial_epoch=initial_epoch)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 875, in fit
    return_dict=True)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py", line 1060, in evaluate
    model=self)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/data_adapter.py", line 1112, in __init__
    model=model)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/data_adapter.py", line 908, in __init__
    **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/data_adapter.py", line 772, in __init__
    peek, x = self._peek_and_restore(x)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/data_adapter.py", line 912, in _peek_and_restore
    return x[0], x
  File "/content/drive/My Drive/EfficientDet-master/generators/common.py", line 424, in __getitem__
    inputs, targets = self.compute_inputs_targets(group)
  File "/content/drive/My Drive/EfficientDet-master/generators/common.py", line 373, in compute_inputs_targets
    image_group = self.load_image_group(group)
  File "/content/drive/My Drive/EfficientDet-master/generators/common.py", line 225, in load_image_group
    return [self.load_image(image_index) for image_index in group]
  File "/content/drive/My Drive/EfficientDet-master/generators/common.py", line 225, in <listcomp>
    return [self.load_image(image_index) for image_index in group]
  File "/content/drive/My Drive/EfficientDet-master/generators/coco.py", line 132, in load_image
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(4.1.2) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

【问题讨论】：

尝试检查在0.0993datasets/coco/images/val2017/000000061960.jpg 上训练的最后一张图像是否存在于目录中（或具有不同的文件扩展名）。

标签： python tensorflow deep-learning google-colaboratory training-data

【解决方案1】：

该错误表示无法找到当前尝试处理的图像（来自文件路径）。如果您查看训练输出，您会看到所有图像路径都来自datasets/coco/images/train2017/，但最后一行表示正在寻找datasets/coco/images/val2017 中的文件。

脚本已完成训练，正在寻找验证/测试数据，我怀疑它找不到。检查val2017目录是否存在，并且该目录下是否有可以用cv.imread()加载的图片。

【讨论】：