【问题标题】:Transfer Learning Audio classification迁移学习音频分类
【发布时间】:2021-10-27 14:12:54
【问题描述】:

我需要对昆虫数据进行 CNN 音频分类。我有一个非常小的 158 个录音和 9 个类的数据集。我需要使用 AudioSet 预训练模型进行迁移学习。我按照 tensorflow 网站上的说明进行操作:https://www.tensorflow.org/tutorials/audio/transfer_learning_audio 但我无法让它工作,因为我在所有时期都得到了持续的低训练 (40%) 和验证准确度 (10%) 和 80% 的测试。我不明白为什么会这样,我做错了什么。我认为问题出在训练验证测试拆分中。下面是代码:

yamnet_model_handle = 'https://tfhub.dev/google/yamnet/1'
yamnet_model = hub.load(yamnet_model_handle)

def load_wav_16k_mono(filename):
    """ Load a WAV file, convert it to a float tensor, resample to 16 kHz single-channel audio. """
    file_contents = tf.io.read_file(filename)
    wav, sample_rate = tf.audio.decode_wav(
          file_contents,
          desired_channels=1)
    wav = tf.squeeze(wav, axis=-1)
    sample_rate = tf.cast(sample_rate, dtype=tf.int64)
    wav = tfio.audio.resample(wav, rate_in=sample_rate, rate_out=16000)
    return wav

class_map_path = yamnet_model.class_map_path().numpy().decode('utf-8')
class_names =list(pd.read_csv(class_map_path)['display_name'])

for name in class_names[:20]:
  print(name)
print('...')
enter code here

metadataf = "C:\\Users\\Thesis_PC\\orthoptera\\metadata.csv"
base_data_path = "C:\\Users\\Thesis_PC\\orthoptera\\resized16\\"

metadata = pd.read_csv(metadataf)
metadata.head()

from sklearn.preprocessing import LabelEncoder
encoder = LabelEncoder()
metadata['species'] = encoder.fit_transform(metadata['species'])
metadata.head()

my_classes = ["Chorthippus biguttulus",
              "Chorthippus brunneus",
              "Gryllus campestris",
              "Nemobius sylvestris",
              "Oecanthus pellucens",
              "Pholidoptera griseoaptera",
              "Pseudochorthippus parallelus" ,
              "Roeseliana roeselii",
              "Tettigonia viridissima"]

full_path = metadata['id'].apply(lambda row: os.path.join(base_data_path, row))
metadata = metadata.assign(id=full_path)

filenames = metadata['id']
targets = metadata['species']

main_ds = tf.data.Dataset.from_tensor_slices((filenames, targets))
main_ds.element_spec
metadata.head(100)

def load_wav_for_map(filename, label):
    return load_wav_16k_mono(filename), label

main_ds = main_ds.map(load_wav_for_map)
main_ds.element_spec

# applies the embedding extraction model to a wav data
def extract_embedding(wav_data, label):
  ''' run YAMNet to extract embedding from the wav data '''
  scores, embeddings, spectrogram = yamnet_model(wav_data)
  num_embeddings = tf.shape(embeddings)[0]
  return (embeddings,
            tf.repeat(label, num_embeddings))

# extract embedding
main_ds = main_ds.map(extract_embedding).unbatch()
main_ds.element_spec

main_ds = main_ds.cache().shuffle(1000)

train_size = 130
val_size = 18
test_size = 10

train_ds = main_ds.take(train_size)
test_ds = main_ds.skip(train_size)
val_ds = test_ds.skip(test_size)
test_ds = test_ds.take(test_size)


train_ds = train_ds.cache().shuffle(15).batch(5).prefetch(tf.data.AUTOTUNE)
val_ds = val_ds.cache().batch(5).prefetch(tf.data.AUTOTUNE)
test_ds = test_ds.cache().batch(5).prefetch(tf.data.AUTOTUNE)

my_model = tf.keras.Sequential([
    tf.keras.layers.Input(shape=(1024), dtype=tf.float32,
                          name='input_embedding'),
    tf.keras.layers.Dense(256, activation='softmax'),
    tf.keras.layers.Dense(9)
], name='my_model')

my_model.summary()

my_model.compile(optimizer="adam",loss="sparse_categorical_crossentropy",metrics=["accuracy"])

history = my_model.fit(train_ds,
                       epochs=5,
                       validation_data=val_ds)

【问题讨论】:

    标签: python tensorflow audio deep-learning transfer-learning


    【解决方案1】:

    您使用分类交叉熵作为损失函数,因此您需要在模型的最后一个密集层中放置一个 softmax 激活:

    my_model = tf.keras.Sequential([
        tf.keras.layers.Input(shape=(1024), dtype=tf.float32,
                              name='input_embedding'),
        tf.keras.layers.Dense(256, activation='relu'),
        tf.keras.layers.Dense(9, activation='softmax')
    ], name='my_model')
    

    另一种选择是在最后一层不激活,并在损失函数中取logits,如您参考的tensorflow代码:

    my_model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
                         optimizer="adam",
                         metrics=['accuracy'])
    

    【讨论】:

    • 感谢您的回答。现在看起来好多了,但还有另一个问题。训练准确率约为 90%,验证准确率恒定在 23% 左右,而测试集准确率为 80%。怎么会?模型是冻结最后一层还是只是在顶部添加层?
    猜你喜欢
    • 2020-08-09
    • 1970-01-01
    • 2017-09-14
    • 2020-05-13
    • 2018-10-04
    • 2020-11-07
    • 2020-10-31
    • 2019-12-17
    • 1970-01-01
    相关资源
    最近更新 更多