如何使用 Intel 推理引擎 (OpenVINO) 优化带有 batchnorm 层的 keras 模型？

Question

使用 Intel 推理引擎 (OpenVINO toolkit R.5) 优化 keras 模型失败

我按照tutorial 的建议冻结我的模型。 keras 模型经过训练和测试。我需要对其进行优化以进行推理。 但是，在自定义模型上运行模型优化器（mo.py 脚本）时出现错误。

[ ERROR ] shapes (128,9) and (0,) not aligned: 9 (dim 1) != 0 (dim 0)

我的模型的最后几层（9 是类的输出数）是：

conv2d_4 (Conv2D) (None, 4, 4, 128) 204928 batch_normalization_3[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, 4, 4, 128) 0 conv2d_4[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 4, 4, 128) 512 activation_4[0][0]
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, 1, 1, 128) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 1, 1, 128) 0 average_pooling2d_2[0][0]
__________________________________________________________________________________________________
flatten (Flatten) (None, 128) 0 dropout_2[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 128) 16512 flatten[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, 128) 0 dense[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 128) 512 activation_5[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 128) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 9) 1161 dropout_3[0][0]
__________________________________________________________________________________________________
color_prediction (Activation) (None, 9) 0 dense_1[0][0]
__________________________________________________________________________________________________

模型优化器由于存在 BatchNormalization 层而失败。当我删除它们时，它会成功运行。但是我用

冻结图表

tf.keras.backend.set_learning_phase(0) 

所以像 BatchNormalization 和 Dropout 这样的节点必须在冻结图中删除，我不知道为什么不删除它们。

非常感谢！

Answer 1

我设法在带有批标准化层的 Keras 模型上运行 OpenVINO 模型优化器。该模型的收敛速度似乎也快了一点。尽管测试分类率比没有 BN 的模型低约 5-7%（并且测试和训练数据集的分类率之间的差距更大）。我不确定 BatchNormalization 是否已从我的解决方案中的模型中正确删除（但 openVINO 模型文件不包含，因此已将其删除）。

移除 BN 和 Dropout 层：

#Clear any previous session.
tf.keras.backend.clear_session()
#This line must be executed before loading Keras model.
tf.keras.backend.set_learning_phase(0) 
model = tf.keras.models.load_model(weights_path)

for layer in model.layers:
    layer.training = False
    if isinstance(layer, tf.keras.layers.BatchNormalization):
        layer._per_input_updates = {}
    elif isinstance(layer, tf.keras.layers.Dropout):
        layer._per_input_updates = {}

比冻结会话：

def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
"""
Freezes the state of a session into a pruned computation graph.

Creates a new computation graph where variable nodes are replaced by
constants taking their current value in the session. The new graph will be
pruned so subgraphs that are not necessary to compute the requested
outputs are removed.
@param session The TensorFlow session to be frozen.
@param keep_var_names A list of variable names that should not be frozen,
                    or None to freeze all the variables in the graph.
@param output_names Names of the relevant graph outputs.
@param clear_devices Remove the device directives from the graph for better portability.
@return The frozen graph definition.
"""
from tensorflow.python.framework.graph_util import convert_variables_to_constants
graph = session.graph
with graph.as_default():
    freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
    output_names = output_names or []
    output_names += [v.op.name for v in tf.global_variables()]
    # Graph -> GraphDef ProtoBuf
    input_graph_def = graph.as_graph_def()
    if clear_devices:
        for node in input_graph_def.node:
            node.device = ""
    frozen_graph = convert_variables_to_constants(session, input_graph_def,
                                                output_names, freeze_var_names)
    return frozen_graph