InceptionResnetV2 STEM block keras implementation 与原始论文中的不匹配？答案

【问题标题】：InceptionResnetV2 STEM block keras implementation mismatch the one in the original paper?InceptionResnetV2 STEM block keras implementation 与原始论文中的不匹配？
【发布时间】：2020-10-22 18:02:36
【问题描述】：

我一直在尝试将来自 Keras implementation 的 InceptionResnetV2 模型摘要与他们论文中指定的模型摘要进行比较，但它似乎与 filter_concat 块没有太大相似之处。

模型summary()的第一行如下图所示。（对于我的情况，输入改为512x512，但据我所知，它不影响每层过滤器的数量，所以我们也可以使用它们来跟进纸质代码翻译）：

Model: "inception_resnet_v2"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_1 (InputLayer)            (None, 512, 512, 3)  0
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 255, 255, 32) 864         input_1[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 255, 255, 32) 96          conv2d_1[0][0]
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 255, 255, 32) 0           batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 253, 253, 32) 9216        activation_1[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 253, 253, 32) 96          conv2d_2[0][0]
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 253, 253, 32) 0           batch_normalization_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 253, 253, 64) 18432       activation_2[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 253, 253, 64) 192         conv2d_3[0][0]
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 253, 253, 64) 0           batch_normalization_3[0][0]
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 126, 126, 64) 0           activation_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 126, 126, 80) 5120        max_pooling2d_1[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 126, 126, 80) 240         conv2d_4[0][0]
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 126, 126, 80) 0           batch_normalization_4[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 124, 124, 192 138240      activation_4[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 124, 124, 192 576         conv2d_5[0][0]
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 124, 124, 192 0           batch_normalization_5[0][0]
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)  (None, 61, 61, 192)  0           activation_5[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D)               (None, 61, 61, 64)   12288       max_pooling2d_2[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 61, 61, 64)   192         conv2d_9[0][0]
__________________________________________________________________________________________________
activation_9 (Activation)       (None, 61, 61, 64)   0           batch_normalization_9[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 61, 61, 48)   9216        max_pooling2d_2[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D)              (None, 61, 61, 96)   55296       activation_9[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 61, 61, 48)   144         conv2d_7[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 61, 61, 96)   288         conv2d_10[0][0]
__________________________________________________________________________________________________
activation_7 (Activation)       (None, 61, 61, 48)   0           batch_normalization_7[0][0]
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 61, 61, 96)   0           batch_normalization_10[0][0]
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, 61, 61, 192)  0           max_pooling2d_2[0][0]
__________________________________________________________________________________________________
.
.
. 
many more lines

their paper 的图 3（下图）显示了 InceptionV4 和 InceptionResnetV2 的 STEM 块是如何形成的。在图 3 中，STEM 块中有三个过滤器串联，但在我上面向您展示的输出中，串联似乎是顺序最大池或类似的混合（第一个串联应该出现在 @987654335 之后@)。它增加了连接应该做的过滤器的数量，但没有进行连接。过滤器似乎是按顺序放置的！任何人都知道此输出中发生了什么？它的作用与论文中描述的一样吗？

为了比较，我找到了InceptionV4 keras implementation，他们似乎确实在concatenate_1 中为 STEM 块中的第一个串联做了一个 filter_concat。这是summary() 第一行的输出。

Model: "inception_v4"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_1 (InputLayer)            (None, 512, 512, 3)  0
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 255, 255, 32) 864         input_1[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 255, 255, 32) 96          conv2d_1[0][0]
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 255, 255, 32) 0           batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 253, 253, 32) 9216        activation_1[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 253, 253, 32) 96          conv2d_2[0][0]
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 253, 253, 32) 0           batch_normalization_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 253, 253, 64) 18432       activation_2[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 253, 253, 64) 192         conv2d_3[0][0]
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 253, 253, 64) 0           batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 126, 126, 96) 55296       activation_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 126, 126, 96) 288         conv2d_4[0][0]
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 126, 126, 64) 0           activation_3[0][0]
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 126, 126, 96) 0           batch_normalization_4[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 126, 126, 160 0           max_pooling2d_1[0][0]
                                                                 activation_4[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 126, 126, 64) 10240       concatenate_1[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 126, 126, 64) 192         conv2d_7[0][0]
__________________________________________________________________________________________________
activation_7 (Activation)       (None, 126, 126, 64) 0           batch_normalization_7[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 126, 126, 64) 28672       activation_7[0][0]
__________________________________________________________________________________________________
.
.
.
and many more lines

因此，如本文所示，两种架构的第一层应该相同。还是我遗漏了什么？

编辑：我发现，Keras 的 InceptionResnetV2 实现不遵循 InceptionResnetV2 的 STEM 块，而是 InceptionResnetV1（图 14 来自他们的论文，附在下面）。在 STEM 块之后，它似乎很好地遵循了 InceptionResnetV2 的其他块。

InceptionResnetV1 的性能不如 InceptionResnetV2（图 25），因此我对使用来自 V1 的块而不是来自 keras 的完整 V2 持怀疑态度。我会尝试从我找到的 InceptionV4 中删除 STEM，并继续使用 InceptionResnetV2。

同样的问题在 tf-models github 中没有解释就关闭了。如果有人感兴趣，我把它留在这里：https://github.com/tensorflow/models/issues/1235

编辑 2： 出于某种原因，GoogleAI（Inception 架构的创建者）在发布“inception-resnet-v2”代码时在their blog 中显示了一张图片。但是 STEM 块是来自 InceptionV3 的块，而不是 InceptionV4 中的块，正如论文中指定的那样。所以，要么论文错了，要么代码由于某种原因没有跟上论文。

【问题讨论】：

标签： keras deep-learning neural-network conv-neural-network

【解决方案1】：

实现了类似的结果。

我刚刚收到一封来自 Google 高级研究科学家兼 blog post regarding the release of the code for InceptionResnetV2 的原始发布者 Alex Alemi 的确认错误的电子邮件。似乎在内部实验期间，STEM 块被切换了，并且版本一直保持这样。

引用：

丹妮·阿泽马尔，

看来你是对的。不完全确定发生了什么，但代码显然是真相的来源，因为被释放的检查点适用于也发布的代码。当我们开发架构我们做了很多内部工作实验，我想在某些时候茎被交换了。不是当然我现在有时间更深入地挖掘，但就像我说的，已发布的检查点是已发布代码的检查点，您可以通过运行评估管道来验证自己。我同意你的看法似乎这是使用原始的 Inception V1 词干。最好的问候，

亚历克斯·阿莱米

我会在这篇文章中更新有关此主题的更改。

更新：Christian Szegedy，也是原论文的出版商，tweeted me：

最初的实验和模型是在 DistBelief 中创建的，这是一个与 Tensorflow 完全不同的框架。

TF 版本是在一年后添加的，可能与原始模型存在差异，但确保可以达到类似的结果。

因此，由于它取得了相似的结果，因此您的实验将大致相同。

【讨论】：