Tensorflow 数据增强 - 如何将新图像添加到数据集中

Question

我正在尝试一个非常简单的数据扩充，为图像非常有限的类增加我的数据集。我对数据增强的想法是简单地获取缺少数据的类并运行一个函数，该函数生成 3 个通过旋转而不同的附加图像。

但是，我不确定如何添加 concat 或将数据添加回数据集，然后将该数据集连接回包含我所有其他类的主数据集。

我的尝试

# Loading the dataset of the class that are lacking in images
lacking_ds = tf.data.Dataset.list_files(DATA_DIR + '/3/*')

# Some preprocessing function that do rescaling and normalizing the pixel value
processed_lacking_ds = lacking_ds.map(preprocess_function)

# For every image of the dataset, I would like to produce additional 3 images through image rotation
for image, label in processed_lacking_ds:
    imageShape = image.shape
    imageType = image.dtype
    for i in range(3):
        image = tf.keras.preprocessing.image.random_rotation(image, 180)
        imageTensor = tf.constant(image, dtype=imageType, shape=imageShape)
        imageDS = tf.data.Dataset.from_tensors(imageTensor)
        # This is where I'm stuck at, I'm would like to concat or append the newly generated imageTensor and its respective label into processed_lacking_ds.
        # Thereafter concat processed_lacking_ds into another dataset (aka processed_ds below) that contains the other classes for training

其他一些可能有用的信息

# This is how my main dataset will look like
list_ds = tf.data.Dataset.list_files(DATA_DIR + '/*/*')
preprocess_function = partial(preprocessing, target_size=image_size)
processed_data = list_ds.map(preprocess_function).shuffle(100)
train_data = processed_data.take(train_data_size).batch(batch_size)
test_data = processed_data.skip(train_data_size).batch(batch_size)

# This is how I will run my model
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

epochs=10
history = model.fit(
  train_data,
  validation_data=test_data,
  batch_size=batch_size,
  epochs=epochs,
  callbacks=[tensorboard_callback]
)

如果有更有效的方法，请告诉我。

Answer 1

您可以使用preprocessing.RandomRotation 层来扩充您的训练数据：

model = tf.keras.Sequential([
  # ...
  layers.experimental.preprocessing.RandomRotation(0.2),
  # ...
])

见官方例子：https://www.tensorflow.org/tutorials/images/data_augmentation#data_augmentation_2

训练结束后，您将删除该层。