keras 在对两个输出使用两个损失函数时会出错

Question

我有一个网络。最后一层之前的一层是致密层。我希望最后一层返回前一层的最大值和该最大值的索引。

所以如果密集层的输出是 [0,4,5,120,1]，那么最后一层应该返回 [120, 3]。 我需要网络处理的损失仅根据最大值计算，而不是索引。因此，我为第二个输出（即索引）编写了一个损失函数，该函数始终返回零 - 但如果有更好的解决方案，除了如何修复此错误之外，我还想听听。

代码是：

def ignor_loss(preds, trues):
    return 0

# build deep q network
inputs = Input((240,256,3))
pool0 = MaxPooling2D()(inputs)
conv1 = Conv2D(30,3,activation='relu')(pool0)
pool1 = MaxPooling2D()(conv1)
conv2 = Conv2D(40,3,activation='relu')(pool1)
pool2 = MaxPooling2D()(conv2)
conv3 = Conv2D(50,3,activation='relu')(pool2)
pool3 = MaxPooling2D()(conv3)
conv4 = Conv2D(60,3,activation='relu')(pool3)
pool4 = MaxPooling2D()(conv4)
conv5 = Conv2D(80,3,activation='relu')(pool4)
flat = Flatten()(conv5)
dense1 = Dense(70)(flat)
dense2 = Dense(40)(dense1)
values = Dense(env.action_space.n)(dense2)
max_, ind = Lambda(lambda x : [K.max(x),K.argmax(x)])(values)
m = Model(inputs, [max_, ind])
m.compile('adam', ['mse',ignor_loss])

错误是：

TypeError: unsupported operand type(s) for -: 'int' and 'NoneType'

有什么想法吗？

编辑：

这是我更新的代码：

# build deep q network
inputs = Input((240,256,3))
pool0 = MaxPooling2D()(inputs)
conv1 = Conv2D(30,3,activation='relu')(pool0)
pool1 = MaxPooling2D()(conv1)
conv2 = Conv2D(40,3,activation='relu')(pool1)
pool2 = MaxPooling2D()(conv2)
conv3 = Conv2D(50,3,activation='relu')(pool2)
pool3 = MaxPooling2D()(conv3)
conv4 = Conv2D(60,3,activation='relu')(pool3)
pool4 = MaxPooling2D()(conv4)
conv5 = Conv2D(80,3,activation='relu')(pool4)
flat = Flatten()(conv5)
dense1 = Dense(70)(flat)
dense2 = Dense(40)(dense1)
q_values = Dense(env.action_space.n)(dense2)
max_q =  Lambda(lambda x : K.max(x), name='max')(q_values)
ind = Lambda(lambda x : K.argmax(x), name='ind')(q_values)
m = Model(inputs, [max_q,ind])
m.compile('adam', {'max':'mse','ind':'mse'}, loss_weights=[1., 0.0])

我仍然遇到同样的错误：

unsupported operand type(s) for -: 'int' and 'NoneType'

我需要知道为什么会出现这个错误？有什么想法吗？

编辑 2： 现在我将 keepdims=True 添加到 max 函数，并将 K.expand_dims 添加到 argmax 函数，如下所示：

q_values = Dense(env.action_space.n)(dense2)
max_q =  Lambda(lambda x : K.max(x, keepdims=True), name='max')(q_values)
ind = Lambda(lambda x : K.expand_dims(K.argmax(x)), name='ind')(q_values)
m = Model(inputs, [max_q,ind])
m.compile('adam', {'max':'mse','ind':'mse'}, loss_weights=[1., 0.0])

但我得到一个不同的错误：

TypeError: Expected int64, got 0.0 of type 'float' instead.

Answer 1

我认为这是一个更清洁的解决方案

1 步：将模型拟合到最大值

X = np.random.uniform(0,1, (2,240,256,3))
y = np.random.uniform(0,1, 2)

inputs = Input((240,256,3))
pool0 = MaxPooling2D()(inputs)
conv1 = Conv2D(30,3,activation='relu')(pool0)
pool1 = MaxPooling2D()(conv1)
conv2 = Conv2D(40,3,activation='relu')(pool1)
pool2 = MaxPooling2D()(conv2)
conv3 = Conv2D(50,3,activation='relu')(pool2)
pool3 = MaxPooling2D()(conv3)
conv4 = Conv2D(60,3,activation='relu')(pool3)
pool4 = MaxPooling2D()(conv4)
conv5 = Conv2D(80,3,activation='relu')(pool4)
flat = Flatten()(conv5)
dense1 = Dense(70)(flat)
dense2 = Dense(40)(dense1)
values = Dense(10)(dense2) # in my case env.action_space.n is 10
max_ = Lambda(lambda x: tf.reduce_max(x, axis=1, keepdims=True))(values)

m = Model(inputs, max_)
m.compile('adam', 'mse')
m.fit(X,y, epochs=3)

2 步：根据拟合模型返回 max 和 argmax 进行推断（这只需要构建一个新模型）

ind = Lambda(lambda x: tf.expand_dims(tf.argmax(x, axis=1),-1))(values)
final_model = Model(inputs, [max_, ind])
final_model.predict(X) this return max and argmax

EDIT：这里是操作所有操作的紧凑模型。如果你有两个输出，你需要传递给 keras 两个目标。因此，我将第二个目标生成为 0 的数组（它没有影响）

def ignor_loss(trues, preds):
    return 0.

X = np.random.uniform(0,1, (2,240,256,3))
y = np.random.uniform(0,1, 2)

inputs = Input((240,256,3))
pool0 = MaxPooling2D()(inputs)
conv1 = Conv2D(30,3,activation='relu')(pool0)
pool1 = MaxPooling2D()(conv1)
conv2 = Conv2D(40,3,activation='relu')(pool1)
pool2 = MaxPooling2D()(conv2)
conv3 = Conv2D(50,3,activation='relu')(pool2)
pool3 = MaxPooling2D()(conv3)
conv4 = Conv2D(60,3,activation='relu')(pool3)
pool4 = MaxPooling2D()(conv4)
conv5 = Conv2D(80,3,activation='relu')(pool4)
flat = Flatten()(conv5)
dense1 = Dense(70)(flat)
dense2 = Dense(40)(dense1)
values = Dense(10)(dense2) # in my case env.action_space.n is 10
max_ = Lambda(lambda x: tf.reduce_max(x, axis=1, keepdims=True), name='max')(values)
ind = Lambda(lambda x: tf.expand_dims(tf.argmax(x, axis=1),-1), name='ind')(values)

m = Model(inputs, [max_,ind])
m.compile('adam', loss={'max':'mse', 'ind':ignor_loss}, 
      loss_weights={'max':1., 'ind':0.})
m.fit(X, {'max':y, 'ind':np.zeros_like(y)}, epochs=3)
m.predict(X)