Tensorflow：OutOfRangeError tf.train.string_input_producer 已关闭且元素不足

Question

---

我一直在尝试以下代码段来试验输入管道

import tensorflow as tf 
with tf.Session() as sess:

   filename = ['/data/read/A.JPG', '/data/read/B.JPG', '/data/read/C.JPG']    
   filename_queue = tf.train.string_input_producer(filename, shuffle=False, num_epochs=5)

   reader = tf.WholeFileReader()
   key, value = reader.read(filename_queue)    
   tf.local_variables_initializer().run()    
   threads = tf.train.start_queue_runners(sess=sess)
   i = 0
   while True:
      i += 1

      image_data = sess.run(value)
      with open('/data/read/test_%d.jpg' % i, 'wb') as f:
         f.write(image_data)

运行上面的代码得到如下错误信息，看起来是由string_input_producer生成的filename_queue引起的。但我不清楚是什么问题以及如何纠正它。谢谢。

caused by op 'ReaderReadV2', defined at:
File "test_input.py", line 12, in <module>
key, value = reader.read(filename_queue)
File "lib/python3.6/site-packages/tensorflow/python/ops/io_ops.py", line 193, in read
return gen_io_ops._reader_read_v2(self._reader_ref, queue_ref, name=name)
File "lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 411, in _reader_read_v2
queue_handle=queue_handle, name=name)
File "lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
op_def=op_def)
File "lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1228, in __init__
self._traceback = _extract_stack()

OutOfRangeError (see above for traceback): FIFOQueue '_1_input_producer' is closed and has insufficient elements (requested 1, current size 0)
     [[Node: ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/cpu:0"](WholeFileReaderV2, input_producer)]]

Answer 1

queue runners 应始终与Coordinator 一起使用。

协调器帮助多个线程一起停止并报告 等待它们停止的程序的异常。

它们还捕获和处理队列生成的异常，包括用于报告队列已关闭的tf.errors.OutOfRangeError 异常。

所以你的训练应该是：

with tf.Session() as sess:

   filename = ['/data/read/A.JPG', '/data/read/B.JPG', '/data/read/C.JPG']    
   filename_queue = tf.train.string_input_producer(filename, shuffle=False, num_epochs=5)

   reader = tf.WholeFileReader()
   key, value = reader.read(filename_queue)
   tf.local_variables_initializer().run() 

   # Create a coordinator, launch the queue runner threads.
   coord = tf.train.Coordinator()
   threads = tf.train.start_queue_runners(sess=sess, coord=coord)
   i = 0
   try:
       while not coord.should_stop():
          while True:
            i += 1
            image_data = sess.run(value)
            with open('test_%d.jpg' % i, 'wb') as f:
                f.write(image_data)

   except tf.errors.OutOfRangeError:
       # When done, ask the threads to stop.
       print('Done training -- epoch limit reached')
   finally:
       coord.request_stop()
       # Wait for threads to finish.
   coord.join(threads)

Answer 2

这段代码：

filename = ['/data/read/A.JPG', '/data/read/B.JPG', '/data/read/C.JPG'
filename_queue = tf.train.string_input_producer(filename, shuffle=False, num_epochs=5)

创建一个文件名队列，这将允许将 3 个文件中的每一个准确地出列 5 次。查看文档（重点是我的）：

num_epochs：一个整数（可选）。 如果指定，string_input_producer 从 string_tensor 生成每个字符串 num_epochs 次，然后生成 OutOfRange 错误。 如果未指定，string_input_producer 可以无限次循环遍历 string_tensor 中的字符串。

在无限循环中执行 15 次出队操作后，将引发异常 OutOfRangeError。如果你不指定num_epochs，循环将一直运行，直到你用另一种方式停止它。