dropout(): 参数 'input' (位置 1) 必须是 Tensor，而不是使用 XLNet 和 HuggingfCE 时的元组答案

【问题标题】：dropout(): argument 'input' (position 1) must be Tensor, not tuple when using XLNet with HuggingfCEdropout(): 参数 'input' (位置 1) 必须是 Tensor，而不是使用 XLNet 和 HuggingfCE 时的元组
【发布时间】：2021-06-28 05:39:28
【问题描述】：

我收到一条错误消息，提示输入应该是张量类型，而不是元组。我不知道如何解决这个问题，因为我已经按照迁移计划中的说明实现了 return_dict=False 方法。

我的模型如下：

class XLNetClassifier(torch.nn.Module):
    def __init__(self, dropout_rate=0.1):
        super(XLNetClassifier, self).__init__()
        self.XLNet = XLNetModel.from_pretrained('xlnet-base-cased', return_dict=False)
        self.d1 = torch.nn.Dropout(dropout_rate)
        self.l1 = torch.nn.Linear(768, 64)
        self.bn1 = torch.nn.LayerNorm(64)
        self.d2 = torch.nn.Dropout(dropout_rate)
        self.l2 = torch.nn.Linear(64, 3)
        
    def forward(self, input_ids, attention_mask):
        x = self.XLNet(input_ids=input_ids, attention_masks = attention_mask)
        x = self.d1(x)
        x = self.l1(x)
        x = self.bn1(x)
        x = torch.nn.Tanh()(x)
        x = self.d2(x)
        x = self.l2(x)
        
        return x

调用dropout时出错。

【问题讨论】：

标签： tensor huggingface-transformers

【解决方案1】：

XLNetModel 返回两个output values：

last_hidden_state
内存

这意味着你得到了一个元组，而不是错误消息所说的单个张量。因此，您的类定义应该是：

from transformers import XLNetModel, XLNetTokenizerFast
import torch

class XLNetClassifier(torch.nn.Module):
    def __init__(self, dropout_rate=0.1):
        super(XLNetClassifier, self).__init__()
        self.XLNet = XLNetModel.from_pretrained('xlnet-base-cased', return_dict=False)
        self.d1 = torch.nn.Dropout(dropout_rate)
        self.l1 = torch.nn.Linear(768, 64)
        self.bn1 = torch.nn.LayerNorm(64)
        self.d2 = torch.nn.Dropout(dropout_rate)
        self.l2 = torch.nn.Linear(64, 3)
        
    def forward(self, input_ids, attention_mask):
        x = self.XLNet(input_ids=input_ids, attention_masks = attention_mask)
        x = self.d1(x[0])
        x = self.l1(x)
        x = self.bn1(x)
        x = torch.nn.Tanh()(x)
        x = self.d2(x)
        x = self.l2(x)
        
        return x


tokenizer = XLNetTokenizerFast.from_pretrained('xlnet-base-cased')
model = XLNetClassifier()
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt", return_token_type_ids=False)
outputs = model(**inputs)

没有return_dict=False甚至更好

class XLNetClassifier(torch.nn.Module):
    def __init__(self, dropout_rate=0.1):
        super(XLNetClassifier, self).__init__()
        self.XLNet = XLNetModel.from_pretrained('xlnet-base-cased')
        self.d1 = torch.nn.Dropout(dropout_rate)
        self.l1 = torch.nn.Linear(768, 64)
        self.bn1 = torch.nn.LayerNorm(64)
        self.d2 = torch.nn.Dropout(dropout_rate)
        self.l2 = torch.nn.Linear(64, 3)
        
    def forward(self, input_ids, attention_mask):
        x = self.XLNet(input_ids=input_ids, attention_masks = attention_mask)
        x = self.d1(x.last_hidden_state)
        x = self.l1(x)
        x = self.bn1(x)
        x = torch.nn.Tanh()(x)
        x = self.d2(x)
        x = self.l2(x)
        
        return x

【讨论】：