合并 Conv2D 和 Dense 模型会导致“RuntimeError: You must compile your model before using it.”，尽管已经编译了合并的模型

Question

我正在尝试训练 AI 根据图像和患者信息识别病变。我正在使用 Keras 的 Sequential 模型来做到这一点。我制作了两个顺序模型，然后将它们合并并编译合并的模型。 当我尝试拟合模型时，即使我的模型定义了输入形状，我也会收到错误 RuntimeError: You must compile your model before using it.。

我尝试将 input_dim=dim 切换为 input_shape=(dim,)。 我在这个问题上唯一能找到的东西，比如this post 或this one 只说要确保你合并在一起的模型中的第一层有一个定义的 input_shape，我有。我无法想象你也必须为 Concatenate 层这样做。

我首先为患者信息创建密集层：

metadata_model = Sequential()
metadata_model.add(Dense(32, input_dim=X_train.iloc[:, L*W:].shape[1], activation="relu"))
metadata_model.add(Dense(64))

然后是图像的模型：

model = Sequential()
model.add(Conv2D(32, (3, 3), padding="same", input_shape=(W, L, 3)))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=-1))
model.add(MaxPooling2D(pool_size=(3,3)))
model.add(Dropout(rate = 0.25))
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=-1))
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=-1))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(128, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=-1))
model.add(Conv2D(128, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=-1))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(1024))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Dropout(0.5))

然后我合并它们：

merged_model = Sequential()
merged_model.add(Concatenate([model, metadata_model]))
merged_model.add(Dense(7)) #7 lesion classes
merged_model.add(Activation("softmax"))

编译并创建一个 ImageDataGenerator：

opt = Adam(lr=INIT_LR, decay=INIT_LR/EPOCHS)
merged_model.compile(loss="categorical_crossentropy", optimizer = opt, metrics=["accuracy"])
aug = ImageDataGenerator(rotation_range=25, width_shift_range=0.1, height_shift_range=0.1, shear_range=0.2, zoom_range=0.2, horizontal_flip=True, fill_mode="nearest")

并尝试训练它：

train = merged_model.fit_generator(
aug.flow([trainInput, X_train.iloc[:, L*W:]], labels, batch_size=BS),
validation_data=([testInput, X_test.iloc[:, L*W:]], labels_test),
steps_per_epoch=500,
epochs=EPOCHS,
verbose=1)

此行导致以下错误：

RuntimeError                              Traceback (most recent call last)
<ipython-input-114-fc6c254db390> in <module>
      4 steps_per_epoch=500,
      5 epochs=EPOCHS,
----> 6 verbose=1)

c:\users\megag\appdata\local\programs\python\python37\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

c:\users\megag\appdata\local\programs\python\python37\lib\site-packages\keras\engine\training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   1416             use_multiprocessing=use_multiprocessing,
   1417             shuffle=shuffle,
-> 1418             initial_epoch=initial_epoch)
   1419 
   1420     @interfaces.legacy_generator_methods_support

c:\users\megag\appdata\local\programs\python\python37\lib\site-packages\keras\engine\training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
     38 
     39     do_validation = bool(validation_data)
---> 40     model._make_train_function()
     41     if do_validation:
     42         model._make_test_function()

c:\users\megag\appdata\local\programs\python\python37\lib\site-packages\keras\engine\training.py in _make_train_function(self)
    494     def _make_train_function(self):
    495         if not hasattr(self, 'train_function'):
--> 496             raise RuntimeError('You must compile your model before using it.')
    497         self._check_trainable_weights_consistency()
    498         if self.train_function is None:

RuntimeError: You must compile your model before using it.

Answer 1

您的合并模型不再是顺序的（因为它有两个输入层/分支），因此您不能使用顺序 API。相反，您需要使用Functional API of Keras 来合并您的模型：

from keras.models import Model

x = Concatenate()([model.output, metadata_model.output])
x = Dense(7)(x)
out = Activation("softmax")(x)

merged_model = Model([model.input, metadata_model.input], out)

# the rest is the same...