在运行时使用 keras 从自动编码器模型中保存隐藏层的特征

Question

我正在训练一个自动编码器模型，并希望在运行时从编码器部分保存每个图像的特征，并稍后将其用于特征匹配。

我的模型结构是-

train_data_dir = '/train'
test_data_dir = '/test'
nb_train_samples = 100
nb_validation_samples = 25
nb_epoch = 2
batch_size = 5
input_img = Input(shape=( img_width, img_height,3))

x = Conv2D(128, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)

x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(32, (3, 3), activation='relu',padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(128, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(3, (3, 3), activation='sigmoid', padding='same')(x)
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

def fixed_generator(generator):
    for batch in generator:
        yield (batch, batch)

train_datagen = ImageDataGenerator(
        rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True)
# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
        train_data_dir,
        target_size=(img_width, img_height),
        batch_size=batch_size,
        class_mode=None)
#print(type(train_generator))
test_generator = test_datagen.flow_from_directory(
        test_data_dir,
        target_size=(img_width, img_height),
        batch_size=batch_size,
        class_mode=None)

autoencoder.fit_generator(
        fixed_generator(train_generator),
        epochs=nb_epoch,
        samples_per_epoch=nb_train_samples,
        validation_data=fixed_generator(test_generator),
        validation_steps=nb_validation_samples
        )

如何在模型拟合期间保存编码器部分的特征。有什么办法吗，请指教

Answer 1

Keras Functional API 是关键词，甚至在自动编码器上有一个非常有见地的blog post。

以下是取自博文的示例的要点：

input_img = Input(shape=(784,))
encoded = Dense(encoding_dim, activation='relu')(input_img)
decoded = Dense(784, activation='sigmoid')(encoded)
# this model maps an input to its reconstruction
autoencoder = Model(input_img, decoded)
# this model maps an input to its encoded representation
encoder = Model(input_img, encoded)

现在您可以将autoencoder.fit(...,epochs=1) 方法放入for 循环中，并使用encoder.predict(your_data) 查看编码表示以保存特征。

Answer 2

要从自动编码器模型中保存特征，首先需要加载模型并从模型中提取最后一个编码器层。

以下是提取编码器层并保存图像特征的代码：

autoencoder= load_model('model_weights.h5')
encoder = Model(autoencoder.input, autoencoder.layers[-1].output)

# Read your input image for which you need to extract features

img = cv2.imread(img)
newimg = cv2.resize(img,(512,512))
encoded_imgs = encoder.predict(newimg[None])[0]

# Save your features as an numpy array

np.save('features.npy', encoded_imgs)

为输入图像保存特征后，您需要找到欧几里德距离以进行特征匹配。

file = 'sample.npy'
file = np.load(file)
file1 = "features.npy"
file1 = np.load(file1)
distance = np.linalg.norm(file-file1)
print(distance)

此代码将从图像中提取特征并计算两个 numpy 数组之间的距离。

Answer 3

在处理文本数据的类似任务中，我从经过训练的模型中提取编码器并将其保存。我加载编码器并执行encoder.predict(input_sample) 以生成编码向量（用于检索目的）。然而，每次我加载和预测输出都会变化（保持相同的输入）。这种行为正常吗？

Answer 4

我认为你应该使用

encoder = Model(input_img , encoded)

而不是

encoder = Model(autoencoder.input, autoencoder.layers[-1].output)

因为 autoencoder.layers[-1].output 表示解码层，即重建的原始图像