Tensorflow keras：加载优化器权重的问题

Question

我已经以良好的准确度运行了基本模型，现在我想加载这些权重并将它们用于具有一些附加层的模型，然后用于超参数调整。

首先我构建了这个新模型

  input_tensor = Input(shape=train_generator.image_shape)

  base_model = applications.ResNet152(weights='imagenet', include_top=False, input_tensor=input_tensor)

  for layer in base_model.layers[:]:
    layer.trainable = False 

  x = Flatten()(base_model.output)
  x = Dense(1024, kernel_regularizer=tf.keras.regularizers.L2(l2=0.01), 
          kernel_initializer=tf.keras.initializers.HeNormal(), kernel_constraint=tf.keras.constraints.UnitNorm(axis=0))(x)
  x = LeakyReLU()(x)
  x = BatchNormalization()(x)
  x = Dropout(rate=0.1)(x)
  x = Dense(512, kernel_regularizer=tf.keras.regularizers.L2(l2=0.01), 
          kernel_initializer=tf.keras.initializers.HeNormal(), kernel_constraint=tf.keras.constraints.UnitNorm(axis=0))(x)
  x = LeakyReLU()(x)
  x = BatchNormalization()(x)
  
  predictions = Dense(num_classes, activation= 'softmax')(x)
  model = Model(inputs = base_model.input, outputs = predictions)

然后我编译它，因为在这个阶段这是必要的，因为我必须在加载权重之前运行与虚拟输入拟合的模型。 （我想，我曾尝试将这些代码块按许多不同的顺序排列以使其工作，但每次都失败了）

opt = tfa.optimizers.LazyAdam(lr=0.000074)

model.compile(
    loss='sparse_categorical_crossentropy',
    optimizer=opt,
    metrics=['accuracy']
    )

dummy_input = tf.random.uniform([32, 224, 224, 3]) 
dummy_label = tf.random.uniform([32,]) 
hist = model.fit(dummy_input, dummy_label)

然后我加载基本模型的权重：

base_model.load_weights('/content/drive/MyDrive/MODELS_SAVED/model_RESNET152/model_weights2.h5', by_name=True)

然后我为优化器加载权重：

import pickle
with open("/content/drive/MyDrive/weight_values2optimizer.pkl", "rb") as f: 
  weights = pickle.load(f) 
opt = model.optimizer.set_weights(weights) 

这会导致以下错误：

ValueError: You called `set_weights(weights)` on optimizer LazyAdam 
with a  weight list of length 1245, 
but the optimizer was expecting 13 weights. 
Provided weights: [63504, array([[[[ 0.00000000e+00, -5.74126025e-04...

有人知道如何解决这个问题吗？

如果您有 Adam 而不是 LazyAdam 的解决方案，那也很好。（我不知道这是否会有所作为）

编辑： 过去几天我尝试了许多新事物，但没有任何效果。这是我现在所在的整个代码。它包括我正在保存的部分和我正在加载的部分。

import tarfile
my_tar2 = tarfile.open('test.tgz')
my_tar2.extractall('test') # specify which folder to extract to
my_tar2.close()

import zipfile
with zipfile.ZipFile("/content/tot_train_bremoved2.zip", 'r') as zip_ref:
    zip_ref.extractall("/content/train/")

import pandas as pd   

train_info = pd.read_csv("/content/drive/MyDrive/train_info.csv")
test_info = pd.read_csv("/content/drive/MyDrive/test_info.csv")
train_folder = "/content/train"
test_folder = "/content/test/test"

import tensorflow as tf
import tensorflow.keras as keras

from keras.layers import Input, Lambda, Dense, Flatten, BatchNormalization, Dropout, PReLU, GlobalAveragePooling2D, LeakyReLU, MaxPooling2D
from keras.models import Model
from tensorflow.keras.applications.resnet_v2 import ResNet152V2, preprocess_input
from keras import applications

from keras.preprocessing import image
from keras.preprocessing.image import ImageDataGenerator
from keras.losses import sparse_categorical_crossentropy

from keras.callbacks import ReduceLROnPlateau, ModelCheckpoint, EarlyStopping, TensorBoard

import tensorflow_addons as tfa

from sklearn.metrics import confusion_matrix
import numpy as np
import matplotlib.pyplot as plt

num_classes = 423
epochs = 20
batch_size = 32
img_height = 224
img_width = 224
IMAGE_SIZE = [img_height, img_width]

_train_generator = ImageDataGenerator(
        rotation_range=180,
        zoom_range=0.2,
        width_shift_range=0.2,
        height_shift_range=0.2,
        shear_range=0.3,
        horizontal_flip=True,
        vertical_flip=True,
        preprocessing_function=preprocess_input)


_val_generator = ImageDataGenerator(
        preprocessing_function=preprocess_input)

train_generator = _train_generator.flow_from_dataframe(dataframe = train_info, 
directory = train_folder, x_col = "filename", 
y_col = "artist", seed = 42,
batch_size = batch_size, shuffle = True, 
class_mode="sparse", target_size = IMAGE_SIZE)

valid_generator = _val_generator.flow_from_dataframe(dataframe = test_info, 
directory = test_folder, x_col = "filename", 
y_col = "artist", seed = 42,
batch_size = batch_size, shuffle = True, 
class_mode="sparse", target_size = IMAGE_SIZE)

def get_uncompiled_model():
   
  input_tensor = Input(shape=train_generator.image_shape)

  base_model = applications.ResNet152(weights='imagenet', include_top=False, input_tensor=input_tensor)

  for layer in base_model.layers[:]:
    layer.trainable = True

  x = Flatten()(base_model.output)
  
  predictions = Dense(num_classes, activation= 'softmax')(x)
  model = Model(inputs = base_model.input, outputs = predictions)

  return model

opt = keras.optimizers.Adam(lr=0.000074)

def get_compiled_model():
    model = get_uncompiled_model()
    model.compile(
    loss='sparse_categorical_crossentropy',
    optimizer=opt,
    metrics=['accuracy']
    )
    return model

earlyStopping = EarlyStopping(monitor='val_loss', patience=5, verbose=0, mode='min')

reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=2, verbose=1, min_delta=1e-4, mode='min')

model = get_compiled_model()

from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True

model.fit(
  train_generator,
  validation_data=valid_generator,
  epochs=epochs,
  verbose = 1,
  steps_per_epoch=len_train // batch_size,
  validation_steps=len_test // batch_size,
  callbacks=[earlyStopping, reduce_lr]
)

import keras.backend as K
import pickle

model.save_weights('/content/drive/MyDrive/MODELS_SAVED/model_RESNET152/model_weights5.h5')
symbolic_weights = getattr(model.optimizer, 'weights')
weight_values = K.batch_get_value(symbolic_weights)
with open('/content/drive/MyDrive/MODELS_SAVED/optimizer3.pkl', 'wb') as f:
    pickle.dump(weight_values, f)

#Here i am building the new model and its from here i am having problems

  input_tensor = Input(shape=train_generator.image_shape)

  base_model = applications.ResNet152(weights='imagenet', include_top=False, input_tensor=input_tensor)

  for layer in base_model.layers[:]:
    layer.trainable = False 

  x = Flatten()(base_model.output)
 
  x = Dense(512, kernel_regularizer=tf.keras.regularizers.L2(l2=0.01), 
          kernel_initializer=tf.keras.initializers.HeNormal(), 
          kernel_constraint=tf.keras.constraints.UnitNorm(axis=0))(x)
  x = LeakyReLU()(x)
  x = BatchNormalization()(x)
  
  predictions = Dense(num_classes, activation= 'softmax')(x)
  model = Model(inputs = base_model.input, outputs = predictions)

model.compile(
    loss='sparse_categorical_crossentropy',
    optimizer='adam',
    metrics=['accuracy']
    )

base_model.load_weights('/content/drive/MyDrive/MODELS_SAVED/model_RESNET152/model_weights5.h5', by_name=True)

with open('/content/drive/MyDrive/MODELS_SAVED/optimizer3.pkl', 'rb') as f:
    weight_values = pickle.load(f)
model.optimizer.set_weights(weight_values)

from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True

epochs = 2

model.fit(
  train_generator,
  validation_data=valid_generator,
  epochs=epochs,
  steps_per_epoch=len_train // batch_size,
  validation_steps=len_test // batch_size,
  verbose = 1,
  callbacks=[earlyStopping, reduce_lr]
)

现在我在运行此代码块时遇到以下错误（上面完整代码中就在 model.fit 之前）：

with open('/content/drive/MyDrive/MODELS_SAVED/optimizer3.pkl', 'rb') as f:
    weight_values = pickle.load(f)
model.optimizer.set_weights(weight_values)

ValueError: You called `set_weights(weights)` on optimizer Adam with a  weight list of length 1245, but the optimizer was expecting 13 weights. Provided weights: [11907, array([[[[ 0.00000000e+00, -8.27514916e-04...

我要做的就是保存模型和优化器的权重，然后构建一个新模型，在其中添加几层并从模型的基础加载权重和优化器的权重。

Answer 1

请参考以下笔记本： https://colab.research.google.com/drive/1j_zLqG1zUMi6UYPdc6gtmkJvHuawL4Sk?usp=sharing

1. 模型上的{save,load}_weights 包括优化器的权重。这将是初始化优化器权重的首选方式。

2. 您可以将优化器从一个模型复制到另一个模型。

3. 您收到上述错误的原因是优化器在训练开始之前不会分配其权重；如果您真的想手动执行，只需触发 model.fit() 1 个 epoch 1 个数据点，然后手动加载数据。

你可以替换

base_model.load_weights('/content/drive/MyDrive/MODELS_SAVED/model_RESNET152/model_weights5.h5', by_name=True)

with open('/content/drive/MyDrive/MODELS_SAVED/optimizer3.pkl', 'rb') as f:
    weight_values = pickle.load(f)
model.optimizer.set_weights(weight_values)

与：

base_model.load_weights('/content/drive/MyDrive/MODELS_SAVED/model_RESNET152/model_weights5.h5', by_name=True)

model.optimizer = base_model.optimizer

Answer 2

两种模型具有不同的架构，因此不能将一个模型的权重加载到另一个模型中，无论它们继承了相同的基本模型。我认为这是微调模型的简单案例（在您的案例中是保存的模型）。

你应该做的是改变创建新模型的方式，即不要使用include_top = False将原始resnet模型加载为基本模型，你应该尝试加载保存的模型并实现你自己的include_top。这可以这样做：

for layer in saved_model.layers[:]:
    layer.trainable = False 

  x = Flatten()(saved_model.layers[-2].output)

这里的关键是saved_model.layers[-2].output，意思是倒数第二层的输出。

希望对您有所帮助，如果没有，请澄清您的疑问或让我知道我错过了什么。

Answer 3

用 model.save_weights('name.h5') 保存第一个模型的权重后，您应该构建第二个模型，就像第一个模型一样，我们称之为model2。然后将您之前保存的权重加载到其中。代码应为model2.load_weights('name.h5')。请参阅model.summary() 以查看第一个模型层的名称和数量。对于每一层，您需要定义一个变量，并使用称为get_weights() 的方法将这些权重（以及偏差）添加到该变量。这是一个例子：

x1 = model2.layers[1].get_weights()

在这里，我将第一层（在我这里是卷积层）的权重和偏差放入变量 x1。

x1[0] 是层 #1 的权重列表。

x1[1] 是第 1 层的偏差列表。