Tensorflow 无法处理多个输入

Question

有一个模型来处理图像和一个模型来处理数值。 合并两个模型得到这个： Merged model

对于图像输入，我从数据框 + 图片创建了一个图像数据生成器：

print('Importing keras...')
from keras_preprocessing.image import ImageDataGenerator

print('Defining datagens and dataflows...')

train_datagen = ImageDataGenerator(rescale = 1./255.,
                                   rotation_range = 25,
                                   width_shift_range = [0.1, 0.2, 0.3],
                                   height_shift_range = [0.1, 0.2, 0.3],
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   channel_shift_range = 0.3,
                                   horizontal_flip = True,
                                   vertical_flip = True)

test_datagen = ImageDataGenerator( rescale = 1.0/255. )

# Flow training images batch size should consider RAM
train_generator = train_datagen.flow_from_dataframe(dataframe=df_train_img,
                                                    directory='/kaggle/input/siim-isic-melanoma-classification/jpeg/train',
                                                    x_col = 'image_name',
                                                    y_col = 'benign_malignant',
                                                    batch_size = 20,
                                                    class_mode = 'binary', 
                                                    target_size = (256, 256))     

# Flow validation images batch size should consider RAM
test_generator = test_datagen.flow_from_dataframe(dataframe=df_test_img,
                                                    directory='/kaggle/input/siim-isic-melanoma-classification/jpeg/train',
                                                    x_col = 'image_name',
                                                    y_col = 'benign_malignant',
                                                    batch_size = 20,
                                                    class_mode = 'binary', 
                                                    target_size = (256, 256))

print('Done')

对于数字部分，我创建了一个具有特征的数据框：

Dataframe

编译模型：

img_model = keras.Sequential([
  layers.Flatten(input_shape=(256, 256, 3)),
  layers.Dense(512, activation='relu'),
  layers.Dense(12, activation='softmax')
])

# Flatten the output layer to 1 dimension
reg_model = keras.Sequential([
  layers.Flatten(input_shape=(12, )),
  layers.Dense(512, activation='relu'),
  layers.Dense(12, activation='softmax')
])

z = layers.concatenate([img_model.output,reg_model.output])
# Flatten the output layer to 1 dimension
z = layers.Flatten()(z)
# Add a fully connected layer with 1,024 hidden units and ReLU activation
z = layers.Dense(1024, activation='relu')(z)
# Add a dropout rate of 0.2
z = layers.Dropout(0.2)(z)                  
# Add a final sigmoid layer for classification
z = layers.Dense(2, activation='softmax')(z)


model = Model( inputs=[reg_model.input, img_model.input], outputs=z) 

model.compile(optimizer = RMSprop(lr=0.0001), 
              loss = 'binary_crossentropy', 
              metrics = ['accuracy'])

当我试图适应时：

history = model.fit(
            {'flatten_15_input':train_generator,'flatten_16_input':X_train},
            {'dense_31' : y_train},
            epochs = 2,
            verbose = 2)

得到以下错误：

ValueError: Failed to find data adapter that can handle input: (<class 'dict'> containing {"<class 'str'>"} keys and {"<class 'keras_preprocessing.image.dataframe_iterator.DataFrameIterator'>", "<class 'numpy.ndarray'>"} values), (<class 'dict'> containing {"<class 'str'>"} keys and {"<class 'numpy.ndarray'>"} values)

尝试将数据框、数组和数据集作为输入传递。 每个模型在合并之前都可以正常工作。我做错了什么？

Answer 1

在 fit 方法中，您应该传递输入列表而不是 dict。输出也是一样。你的代码应该是这样的

history = model.fit(
            [train_generator, X_train],
            y_train,
            epochs = 2,
            verbose = 2)