Keras 激活层无法正常工作答案

【问题标题】：Keras activation layer is not working wellKeras 激活层无法正常工作
【发布时间】：2020-08-24 17:50:00
【问题描述】：

一开始我做了一个模型如下：

from tensorflow.keras.layers import Dense, Flatten, Conv2D, Dropout, BatchNormalization, 
     AveragePooling2D, ReLU, Activation
from tensorflow.keras import Model

class MyModel(Model):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv = Conv2D(4, (3,3), padding = 'same', activation = 'linear'
                           ,input_shape = x_train.shape[1:])
        self.bn = BatchNormalization()
        self.RL = ReLU()
        self.FL = Flatten()
        self.d1 = Dense(4, activation = 'relu')
        self.d2 = Dense(100, activation = 'softmax')
    def call(self,x):
        x = self.conv(x)
        x = self.bn(x)
        x = self.RL(x)
        x = self.FL(x)
        x = self.d1(x)
        return self.d2(x)

但是，此模型效果不佳。准确率只有 1%，这意味着它什么也没学到。（我用 CIFAR100 训练了这个模型——简单只是为了检查代码）但是当我将代码更改如下时，它起作用了。

class MyModel(Model):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv = Conv2D(4, (3,3), padding = 'same', activation = 'linear'
                           ,input_shape = x_train.shape[1:])
        self.bn = BatchNormalization()

        # The below code is changed from ReLU() -> Activation('relu')
        self.RL = Activation('relu')

        self.FL = Flatten()
        self.d1 = Dense(4, activation = 'relu')
        self.d2 = Dense(100, activation = 'softmax')
    def call(self,x):
        x = self.conv(x)
        x = self.bn(x)
        x = self.RL(x)
        x = self.FL(x)
        x = self.d1(x)
        return self.d2(x)

为什么会这样？我不知道问题所在。感谢您的阅读。

【问题讨论】：

如你所想？提供一些相关的上下文。
表示模型什么也没学到。我编辑了这个问题。谢谢！

标签： tensorflow keras activation relu

【解决方案1】：

它们完全等效，没有任何区别会使您的网络不稳定。这是一个例子：

from tensorflow.keras.layers import Dense, Flatten, Conv2D, Dropout, BatchNormalization, AveragePooling2D, ReLU, Activation, Input
from tensorflow.keras import Model
import numpy as np

ip = Input(shape = 5)
rl = ReLU()(ip)

model1 = Model(ip, rl)

ip = Input(shape = (5,))
rl = Activation('relu')(ip)

model2 = Model(ip, rl)

i = np.array([[1., 2., 3., 4., 5.], [-1., -100., -123213., 0., 100000.], [-1., 100234323423., -123213., 0., 100000.]])

print(model1(i))
print(model2(i))

输出：

tf.Tensor(
[[1.0000000e+00 2.0000000e+00 3.0000000e+00 4.0000000e+00 5.0000000e+00]
 [0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.0000000e+05]
 [0.0000000e+00 1.0023432e+11 0.0000000e+00 0.0000000e+00 1.0000000e+05]], shape=(3, 5), dtype=float32)
tf.Tensor(
[[1.0000000e+00 2.0000000e+00 3.0000000e+00 4.0000000e+00 5.0000000e+00]
 [0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.0000000e+05]
 [0.0000000e+00 1.0023432e+11 0.0000000e+00 0.0000000e+00 1.0000000e+05]], shape=(3, 5), dtype=float32)

这两个模型具有相同的输出。

【讨论】：