相对于 Tensorflow 2.0 中的输入图像，如何计算过滤器激活中间层的梯度？

Question

我正在尝试将激活中间层的特定过滤器的图像类型可视化。为此，我需要计算该滤波器中激活的平均值相对于输入图像的梯度，然后使用梯度上升更新图像。

我一直纠结于如何在 Tensorflow 2.0 中计算这个梯度。我试过这个；在这里，我试图在block3_conv1 层中获取索引为0 的过滤器的输出：

input = tf.convert_to_tensor(np.random.random((1, 150, 150, 3))

activation_model = Model(inputs=model.input,
                         outputs=model.get_layer("block3_conv1").output)

with tf.GradientTape() as tape:
    tape.watch(inputs)
    preds = activation_model.predict(inputs)
    loss = np.mean(preds[:,:,:,0]) # defining the mean of all activations as the loss, in the filter with index 0

grads = tape.gradient(tf.convert_to_tensor(loss), inputs)

但这给了我grads 作为None。这是模型摘要：

Model: "vgg16"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, None, None, 3)]   0         
_________________________________________________________________
block1_conv1 (Conv2D)        (None, None, None, 64)    1792      
_________________________________________________________________
block1_conv2 (Conv2D)        (None, None, None, 64)    36928     
_________________________________________________________________
block1_pool (MaxPooling2D)   (None, None, None, 64)    0         
_________________________________________________________________
block2_conv1 (Conv2D)        (None, None, None, 128)   73856     
_________________________________________________________________
block2_conv2 (Conv2D)        (None, None, None, 128)   147584    
_________________________________________________________________
block2_pool (MaxPooling2D)   (None, None, None, 128)   0         
_________________________________________________________________
block3_conv1 (Conv2D)        (None, None, None, 256)   295168    
_________________________________________________________________
block3_conv2 (Conv2D)        (None, None, None, 256)   590080    
_________________________________________________________________
block3_conv3 (Conv2D)        (None, None, None, 256)   590080    
_________________________________________________________________
block3_pool (MaxPooling2D)   (None, None, None, 256)   0         
_________________________________________________________________
block4_conv1 (Conv2D)        (None, None, None, 512)   1180160   
_________________________________________________________________
block4_conv2 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block4_conv3 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block4_pool (MaxPooling2D)   (None, None, None, 512)   0         
_________________________________________________________________
block5_conv1 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_conv2 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_conv3 (Conv2D)        (None, None, None, 512)   2359808   
_________________________________________________________________
block5_pool (MaxPooling2D)   (None, None, None, 512)   0         
=================================================================
Total params: 14,714,688
Trainable params: 14,714,688
Non-trainable params: 0
_________________________________________________________________

Answer 1

只是不要使用model.predict。这将返回 numpy 数组，并且您不能通过 numpy 操作进行反向传播。下面的代码通过使用模型的call 函数保持在张量域中。

with tf.GradientTape() as tape:
    tape.watch(inputs)
    preds = activation_model(inputs)
    loss = tf.reduce_mean(preds[:,:,:,0]) # defining the mean of all activations as the loss, in the filter with index 0

grads = tape.gradient(loss, inputs)