如何使用 Tensorflow 和 Keras 在 UTKFace 数据集上实现多标签分类？

Question

基本上，我试图通过使用 Tensorflow 和 Keras 构建多标签分类模型来从 UTKFace dataset 预测年龄、性别和种族。这就是我的预处理数据集的样子。我这里有几个问题

1. ImageDataGenerator class_mode="multi_output" 或 class_mode="raw" 中的 class_mode 应该是什么，为什么？我试过class_mode="multi_output"(抛出AttributeError: 'tuple' object has no attribute 'shape')和class_mode="raw"(抛出InvalidArgumentError: Input to reshape is a tensor with 15745024 values, but the requested shape requires a multiple of 294912. [[node sequential_1/flatten_1/Reshape (defined at <ipython-input-22-9c3a9b687782>:4) ]] [Op:__inference_train_function_2086])
2. 每个特征（年龄、性别、种族）的损失函数应该是什么？为什么？

这是我到目前为止所做的事情

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Conv2D, Activation, MaxPool2D, Dropout, Flatten
import numpy as np
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import matplotlib.image as mimg
import pandas as pd
from tensorflow.keras.preprocessing.image import ImageDataGenerator

!tar -xvf /content/UTKFace.tar.gz
!tar -xvf /content/crop_part1.tar.gz

!mv /content/UTKFace/* /content/data
!mv /content/crop_part1/* /content/data

contentdata = []

import os
for i in os.listdir("data"):
  content = i.split("_")
  if ((content[0].isnumeric()) and (content[1].isnumeric()) and (content[2].isnumeric())):
      contentdata.append([content[0],content[1],content[2],os.path.join("data",i)])
  #print(content[2])
  #imgarray = plt.imread(os.path.join("data",i))

data = pd.DataFrame(contentdata,columns=["Age","Gender","Racevalues","Filepath"])

data.head(10)

data.Age = data.Age.astype('float')
data.Gender = data.Gender.astype('float')
data['Racevalues'] = data['Racevalues'].astype('float')
data.Filepath = data.Filepath.astype('string')

data.dtypes

train, test = train_test_split(data, test_size=0.1)

testdatagenerator = ImageDataGenerator(rescale=1. /255)
testdata = testdatagenerator.flow_from_dataframe(dataframe=test,directory=None,x_col="Filepath",y_col=["Age","Gender","Racevalues"],class_mode="raw")

traindatagenerator = ImageDataGenerator(rescale=1. /255,shear_range =0.2,zoom_range=0.2,horizontal_flip =True)
traindata = traindatagenerator.flow_from_dataframe(dataframe=train,directory=None,x_col="Filepath",y_col=["Age","Gender","Racevalues"],class_mode="raw")

#model = []

model = Sequential()

model.add(Conv2D(32, (3, 3), padding='same',
                 input_shape=(100,100,3)))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(5, activation='sigmoid'))

model.compile(optimizer="Adam",loss="binary_crossentropy",metrics=["accuracy"])

model.fit(traindata,
                   steps_per_epoch=100,epochs=100,
                   validation_data=testdata,
                       validation_steps=100,batch_size=20)

有人可以指导我完成这个吗？

Answer 1

首先，您应该在标签中使用to_categorical（一个热编码）：

df['Age'] = tf.keras.utils.to_categorical(df['Age'])
df['Gender'] = tf.keras.utils.to_categorical(df['Gender'])
df['Racevalues'] = tf.keras.utils.to_categorical(df['Racevalues'])

所以：

traindata = traindatagenerator.flow_from_dataframe(
    train, 
    IMG_PATH, 
    x_col='Filepath',
    y_col=["Age", "Gender", "Racevalues"],
    target_size=IMAGE_SIZE,
    class_mode="multi_output",
    batch_size=BATCH_SIZE
)

最后一步是使您的模型调整为3输出：

input_node = base_model.get_layer('last_layer_model').output

x = Flatten()(input_node)
age_output = Dense(n_age_labels, activation='softmax', name='age')(x)

x = Flatten()(input_node)
gender_output = Dense(2, activation='softmax', name='gender')(x)

x = Flatten()(input_node)
race_output = Dense(n_race_labels, activation='softmax', name='race')(x)

model = Model(base_model.input, [age_output, gender_output, race_output])

model.compile(optimizer = Adam(learning_rate=1e-4), 
              loss = {
                  'age': 'categorical_crossentropy',
                  'gender': 'binary_crossentropy',
                  'race': 'categorical_crossentropy',
              }, 
              metrics = {
                  'age': 'accuracy',
                  'gender': 'accuracy',
                  'race': 'accuracy'
              }
             )

p.s.:在这个例子中，我正在使用一个分类器方法来预测年龄，可以更改此作为回归。使用分类器，您应该补充预测：

output_indexes = np.array([i for i in range(0, 101)])
apparent_predictions = np.sum(age_predictions * output_indexes, axis = 1)