做 GRAD-CAM 时的“连接层”问题。如何在我的自定义功能模型中克服这个问题？

Question

我在使用 grad-cam 时遇到问题。如果有人能提供帮助，我将不胜感激。我的代码在这里

https://www.kaggle.com/mervearmagan/gradcamproblem

抱歉，我无法修复我遇到的错误

ValueError：输入 0 与层 model_1 不兼容：预期 shape=(None, 512, 512, 3), 找到 shape=(512, 512, 3)

img = tf.keras.layers.Input(shape = IMG_SHAPE)
gender = tf.keras.layers.Input(shape=(1,))
base_model = tf.keras.applications.InceptionV3(input_shape = IMG_SHAPE, include_top = False, weights = 'imagenet')

cnn_vec=base_model(img)
cnn_vec = tf.keras.layers.GlobalAveragePooling2D()(cnn_vec)
cnn_vec = tf.keras.layers.Dropout(0.20)(cnn_vec)
gender_vec = tf.keras.layers.Dense(32,activation = 'relu')(gender)

features = tf.keras.layers.Concatenate(axis=-1)([cnn_vec,gender_vec])

dense_layer = tf.keras.layers.Dense(256,activation = 'relu')(features)
dense_layer = tf.keras.layers.Dropout(0.1)(dense_layer)
dense_layer = tf.keras.layers.Dense(128,activation = 'relu')(dense_layer)
dense_layer = tf.keras.layers.Dropout(0.1)(dense_layer)
dense_layer = tf.keras.layers.Dense(64,activation = 'relu')(dense_layer)
output_layer = tf.keras.layers.Dense(1, activation = 'linear')(dense_layer)
model = tf.keras.Model(inputs=[img,gender],outputs=output_layer`

def make_gradcam_heatmap(img_array, model, last_conv_layer_name, classifier_layer_names):
        last_conv_layer = model.get_layer(last_conv_layer_name)
        last_conv_layer_model = tf.keras.Model(model.inputs, last_conv_layer.output)
        
        classifier_input = tf.keras.layers.Input(shape=last_conv_layer.output.shape)
        #classifier_input = tf.keras.layers.Input(shape=last_conv_layer.output.shape[1:])
        x = classifier_input
        for layer_name in classifier_layer_names:
            x = model.get_layer(layer_name)(x)
        classifier_model = tf.keras.Model(classifier_input, x)
        
        with tf.GradientTape() as tape:
            last_conv_layer_output =last_conv_layer_model(img_array)
             #last_conv_layer_model(img_array)
            tape.watch(last_conv_layer_output)
            
            preds = classifier_model(last_conv_layer_output)
            top_pred_index = tf.argmax(preds[0])
            top_class_channel = preds[:, top_pred_index]
            
        grads = tape.gradient(top_class_channel, last_conv_layer_output)
        pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))
        
        last_conv_layer_output = last_conv_layer_output.numpy()[0]
        pooled_grads = pooled_grads.numpy()
        for i in range(pooled_grads.shape[-1]):
            last_conv_layer_output[:, :, i] *= pooled_grads[i]
        
        heatmap = np.mean(last_conv_layer_output, axis=-1)
        heatmap = np.maximum(heatmap, 0) / np.max(heatmap)
        return heatmap
    
last_conv_layer_name = 'global_average_pooling2d'
classifier_layer_names = ['dense_4']
img = get_input('4360.png' ) 
inputgender=tf.ones((1,1))
image=tf.reshape(img,(1,512,512,3))
heatmap = make_gradcam_heatmap([image,inputgender], model, last_conv_layer_name, classifier_layer_names)

Answer 1

在运行模型时，请记住使用表单中的输入来测试模型：

model([tf.ones((1,512,512,3)),tf.ones((1,1))])

...如果您向网络输入一张图像和一种性别。张量中的第一个“1”表示第一批“样本”，依此类推。这种输入应该给出如下结果：

...在这个阶段看起来还可以。浏览你的代码并首先检查这个“阶段”，然后在你的程序中前进。

Answer 2

这是一种将 numpy 数组格式的图像转换为具有额外维度的张量的便捷方法，以与神经网络输入兼容：

#Advice how to convert image to format of tensor...
import tensorflow as tf
import numpy as np

#Download image...and suppose it has size 512x512,3...e.g. using PIL or whatever suitable library...
#image = Image.open('smile_or_not.png')

#Convert the image to numpy...here we simulate it because no real image was loaded...
image_np=np.random.rand(512,512,3)

#Let's see its shape...
print("Size of input image:",image_np.shape)

#And convert it to a tensor of shape (1,height,widht,3)
in_tensor_format=tf.reshape(image_np,(1,512,512,3))

print("...has a shape of: ", in_tensor_format.shape, "...when converted to tensor")