如何为拥抱脸变压器模型的序列分类设置自定义输入管道？答案

【问题标题】：How do I set up a custom input-pipeline for sequence classification for the huggingface transformer models?如何为拥抱脸变压器模型的序列分类设置自定义输入管道？
【发布时间】：2020-03-02 02:52:08
【问题描述】：

我想使用 huggingface 提供的序列分类模型之一。似乎他们提供了一个名为的函数 glue_convert_examples_to_features() 用于准备数据，以便可以将其输入到模型中。

但是，这个转换函数似乎只适用于胶水数据集。我找不到将转换应用于我的服装数据的简单解决方案。我是否监督了上述预建功能？将具有一个序列和两个标签的自定义数据转换为模型期望的格式的简单方法是什么？

【问题讨论】：

似乎唯一的方法就是将上述函数修改为您的数据集。

标签： python-3.x tensorflow machine-learning nlp

【解决方案1】：

Huggingface 添加了一个fine-tuning with custom datasets 指南，其中包含很多有用的信息。我能够使用 IMDB sequence classification 部分中的信息成功地使用带有我自己的 pandas 数据框的胶水数据集调整笔记本。

from transformers import (
   AutoConfig,
   AutoTokenizer,
   TFAutoModelForSequenceClassification,
   AdamW
)
import tensorflow as tf
import pandas as pd
from sklearn.model_selection import train_test_split

model_name = 'bert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)

df = pd.read_pickle('data.pkl')

train_texts = df.text.values  # an array of strings
train_labels = df.label.values  # an array of integers

train_texts, val_texts, train_labels, val_labels = train_test_split(train_texts, train_labels, test_size=.2)

train_encodings = tokenizer(train_texts.tolist(), truncation=True, max_length=96, padding=True)
val_encodings = tokenizer(val_texts.tolist(), truncation=True, max_length=96, padding=True)

train_dataset = tf.data.Dataset.from_tensor_slices((
    dict(train_encodings),
    train_labels
))
val_dataset = tf.data.Dataset.from_tensor_slices((
    dict(val_encodings),
    val_labels
))

num_labels = 3
num_train_examples = len(train_dataset)
num_dev_examples = len(val_dataset)

train_dataset = train_dataset.shuffle(100).batch(train_batch_size)
val_dataset = val_dataset.shuffle(100).batch(eval_batch_size)

learning_rate = 2e-5
train_batch_size =  8
eval_batch_size = 8
num_epochs = 1

train_steps_per_epoch = int(num_train_examples / train_batch_size)
dev_steps_per_epoch = int(num_dev_examples / eval_batch_size)

config = AutoConfig.from_pretrained(model_name, num_labels=num_labels)
model = TFAutoModelForSequenceClassification.from_pretrained(model_name, config=config)

optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metrics = [tf.keras.metrics.SparseCategoricalAccuracy('accuracy', dtype=tf.float32)]

model.compile(optimizer=optimizer, loss=loss, metrics=metrics)

history = model.fit(train_dataset, 
  epochs=num_epochs,
  steps_per_epoch=train_steps_per_epoch,
  validation_data=val_dataset,
  validation_steps=dev_steps_per_epoch)

笔记本学分：digitalepidemiologylab covid-twitter-bert colab

【讨论】：