将 tensorflow 数据格式从 nhwc（通道在后）切换到 nchw（通道在前）时出现问题

Question

我目前正在尝试为 OCR 任务训练 CRNN（卷积递归神经网络）。我通常将数据格式保持为默认值，即 NHWC（通道最后）。但由于一些部署限制，我想使用 NCHW（通道优先）数据格式。

为此，我使用tensorflow.keras.backend.set_image_data_format('channels_first')。

我正在使用以下方法相应地更改我的数据管道：

if my_configs.data_format == 'channels_first':
    img = tf.transpose(img, [2, 0, 1])

我的模型有卷积层，后跟双向 lstm 层。这是我的模型的一部分：

    conv_7 = Conv2D(64, (2,2), activation = "relu")(pool_6)
    
    if data_format == 'channels_first':
        squeezed = Lambda(lambda x: tf.squeeze(x, 2))(conv_7)
        # squeezed = Reshape((1,64,31))(squeezed)
    else:
        squeezed = Lambda(lambda x: tf.squeeze(x, 1))(conv_7)

    if data_format == "channels_first":
        use_bias = False
    else:
        use_bias = True
    # bidirectional LSTM layers with units=128
    blstm_1 = Bidirectional(LSTM(128, return_sequences=True, use_bias=use_bias))(squeezed)
    blstm_2 = Bidirectional(LSTM(128, return_sequences=True, use_bias=use_bias))(blstm_1)

现在，这是我的问题。如果我在 LSTM 层中保持偏差，所以use_bias=True，那么我会收到关于尺寸的错误。这是错误：

Traceback (most recent call last):
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\ops.py", line 1812, in _create_c_op
    c_op = pywrap_tf_session.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be at least rank 3 but is rank 2 for '{{node BiasAdd}} = BiasAdd[T=DT_FLOAT, data_format="NCHW"](add, bias)' with input shapes: [?,512], [512].      

During handling of the above exception, another exception occurred:       

Traceback (most recent call last):   
  File "run_training.py", line 292, in <module>
    model = model_architecture_1_bis(len(char_list), data_format)
  File "D:\GdPicture\dev\MachineLearning\OCR\ModelsArchitectures.py", line 178, in model_architecture_1_bis    
    blstm_1 = Bidirectional(LSTM(128, return_sequences=True, use_bias=use_bias))(squeezed)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\wrappers.py", line 530, in __call__
    return super(Bidirectional, self).__call__(inputs, **kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 925, in __call__       
    return self._functional_construction_call(inputs, args, kwargs,       
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1117, in _functional_construction_call
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\wrappers.py", line 643, in call
    y = self.forward_layer(forward_inputs,
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent.py", line 663, in __call__        
    return super(RNN, self).__call__(inputs, **kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 925, in __call__       
    return self._functional_construction_call(inputs, args, kwargs,       
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1117, in _functional_construction_call
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent_v2.py", line 1183, in call        
    runtime) = lstm_with_backend_selection(**normal_lstm_kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent_v2.py", line 1557, in lstm_with_backend_selection
    last_output, outputs, new_h, new_c, runtime = defun_standard_lstm(    
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\eager\function.py", 
line 2828, in __call__
    graph_function, args, kwargs = self._maybe_define_function(args, kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\eager\function.py", 
line 3213, in _maybe_define_function 
    graph_function = self._create_graph_function(args, kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\eager\function.py", 
line 3065, in _create_graph_function 
    func_graph_module.func_graph_from_py_func(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\func_graph.py", line 986, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent_v2.py", line 1305, in standard_lstm
    last_output, outputs, new_states 
= K.rnn(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)   
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\backend.py", line 4213, in rnn
    output_time_zero, _ = step_function(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\layers\recurrent_v2.py", line 1293, in step        
    z = K.bias_add(z, bias)
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)   
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\keras\backend.py", line 5772, in bias_add
    return nn.bias_add(x, bias, data_format='NCHW')
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)   
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 3366, in bias_add
    return gen_nn_ops.bias_add(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\ops\gen_nn_ops.py", 
line 693, in bias_add
    _, _, _op, _outputs = _op_def_library._apply_op_helper(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 742, in _apply_op_helper
    op = g._create_op_internal(op_type_name, inputs, dtypes=None,
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\func_graph.py", line 591, in _create_op_internal
    return super(FuncGraph, self)._create_op_internal(  # pylint: disable=protected-access
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\ops.py", line 3477, in _create_op_internal     
    ret = Operation(
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\ops.py", line 1974, in __init__
    self._c_op = _create_c_op(self._graph, node_def, inputs,
  File "C:\Users\*\miniconda3\envs\tf2-gpu-env\lib\site-packages\tensorflow\python\framework\ops.py", line 1815, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be at least rank 3 but is rank 2 for '{{node BiasAdd}} = BiasAdd[T=DT_FLOAT, data_format="NCHW"](add, bias)' with input shapes: [?,512], [512].

另一方面，如果我消除了偏差，所以use_bias=False，那么我不会收到关于尺寸的错误，但我的模型很糟糕。我得到了大约 50% 的准确率（在这种情况下，准确率意味着我正确识别了一个词）。 我相信这些糟糕的结果是由于没有在我的 LSTM 层中使用偏差，但我可能是错的。

所以我的问题是，消除偏差会对结果产生那么大的影响吗？ 另外，当我使用偏差时，您认为尺寸误差的来源是什么？

Answer 1

为了快速检查，您可以使用带有和不带有偏差的通道最后格式训练模型并监控准确性，因此如果偏差有重大影响，您可以调试您的通道优先设置。

Answer 2

所以我的问题是，消除偏差会对结果有那么大的影响吗？

我怀疑偏见是造成如此低绩效的原因。如果我们退后一步看看网络中发生了什么，每一层都在应用一系列W * X + B 形式的线性变换，然后是非线性变换tanh, sigmoid...。

线性部分W * X 是应用若干变换的数学等价物，例如：拉伸空间、收缩空间、旋转......

这是从优秀的3 Blue 1 Brown频道拍摄的图片

X + B 相当于在空间中移动原点，其他条件相同。

从深度学习的角度来看，这意味着梯度方差的很大一部分位于空间中的一个方向上，通过简单地沿该方向平移输入，您可以显着降低成本函数（即：与所有可能的线性和非线性组合）。

这并非不可能，但似乎不太可能发生。

另外，当我使用偏差时，您认为尺寸误差的来源是什么？

查看 cmets 以获得更精确的信息。