分类准确率太低（Word2Vec）

Question

我正在研究一个由 word2vec 解决的多标签情感分类问题。这是我从几个教程中学到的代码。现在准确率很低。大约 0.02 告诉我我的代码有问题。但我找不到它。我为 TF-IDF 和 BOW 尝试了这段代码（显然除了 word2vec 部分），我得到了更好的准确度分数，例如 0.28，但似乎这个有点错误：

np.set_printoptions(threshold=sys.maxsize)
wv = gensim.models.KeyedVectors.load_word2vec_format("E:\\GoogleNews-vectors-negative300.bin", binary=True)
wv.init_sims(replace=True)

#Pre-Processor Function
pre_processor = TextPreProcessor(
    omit=['url', 'email', 'percent', 'money', 'phone', 'user',
        'time', 'url', 'date', 'number'],
    
    normalize=['url', 'email', 'percent', 'money', 'phone', 'user',
        'time', 'url', 'date', 'number'],
     
    segmenter="twitter", 
    
    corrector="twitter", 
    
    unpack_hashtags=True,
    unpack_contractions=True,
    
    tokenizer=SocialTokenizer(lowercase=True).tokenize,
    
    dicts=[emoticons]
)

#Averaging Words Vectors to Create Sentence Embedding
def word_averaging(wv, words):
    all_words, mean = set(), []
    
    for word in words:
        if isinstance(word, np.ndarray):
            mean.append(word)
        elif word in wv.vocab:
            mean.append(wv.syn0norm[wv.vocab[word].index])
            all_words.add(wv.vocab[word].index)

    if not mean:
        logging.warning("cannot compute similarity with no input %s", words)
        # FIXME: remove these examples in pre-processing
        return np.zeros(wv.vector_size,)

    mean = gensim.matutils.unitvec(np.array(mean).mean(axis=0)).astype(np.float32)
    return mean

def  word_averaging_list(wv, text_list):
    return np.vstack([word_averaging(wv, post) for post in text_list ])

#Secondary Word-Averaging Method
def get_mean_vector(word2vec_model, words):
# remove out-of-vocabulary words
words = [word for word in words if word in word2vec_model.vocab]
if len(words) >= 1:
    return np.mean(word2vec_model[words], axis=0)
else:
    return []

#Loading data
raw_train_tweets = pandas.read_excel('E:\\train.xlsx').iloc[:,1] #Loading all train tweets
train_labels = np.array(pandas.read_excel('E:\\train.xlsx').iloc[:,2:13]) #Loading corresponding train labels (11 emotions)

raw_test_tweets = pandas.read_excel('E:\\test.xlsx').iloc[:,1] #Loading 300 test tweets
test_gold_labels = np.array(pandas.read_excel('E:\\test.xlsx').iloc[:,2:13]) #Loading corresponding test labels (11 emotions)
print("please wait")

#Pre-Processing
train_tweets=[]
test_tweets=[]
for tweets in raw_train_tweets:
    train_tweets.append(pre_processor.pre_process_doc(tweets))

for tweets in raw_test_tweets:
    test_tweets.append(pre_processor.pre_process_doc(tweets))

#Vectorizing 
train_array = word_averaging_list(wv,train_tweets)
test_array = word_averaging_list(wv,test_tweets)

#Predicting and Evaluating    
clf = LabelPowerset(LogisticRegression(solver='lbfgs', C=1, class_weight=None))
clf.fit(train_array,train_labels)
predicted= clf.predict(test_array)
intersect=0
union=0
accuracy=[]
for i in range(0,3250): #i have 3250 test tweets.
    for j in range(0,11): #11 emotions
        if predicted[i,j]&test_gold_labels[i,j]==1:
            intersect+=1
        if predicted[i,j]|test_gold_labels[i,j]==1:
            union+=1
    
    accuracy.append(intersect/union) if union !=0 else accuracy.append(0.0)
    intersect=0
    union=0
print(np.mean(accuracy))

结果：

0.4674498168498169

我打印了预测变量（用于推文 0 到 10）以查看它的样子：

  (0, 0)    1
  (0, 2)    1
  (2, 0)    1
  (2, 2)    1
  (3, 4)    1
  (3, 6)    1
  (4, 0)    1
  (4, 2)    1
  (5, 0)    1
  (5, 2)    1
  (6, 0)    1
  (6, 2)    1
  (7, 0)    1
  (7, 2)    1
  (8, 4)    1
  (8, 6)    1
  (9, 3)    1
  (9, 8)    1

如您所见，它只显示 1。例如 (6,2) 表示在推文编号 6 中，情绪编号 2 为 1。 (9,8) 表示在推文编号 9 中，情绪编号 8 为 1。其他情绪视为 0。但您可以这样想象为了更好地了解我在 Accuracy 方法中所做的事情：

gold emotion for tweet 0:      [1 1 0 0 0 0 1 0 0 0 1]
predicted emotion for tweet 0: [1 0 1 0 0 0 0 0 0 0 0]

我已经对索引一一使用了 union 和 intersect。 1 到 1. 1 到 1. 0 到 1，直到黄金情感 11 到预测情感 11。我在两个 for 循环中对所有推文都这样做了。

在我的推文上创建 Word2Vec 向量：

现在我想使用 gensim 在我的推文数据集上创建 Word2Vec 向量。我将上面的部分代码更改如下：

#Averaging Words Vectors to Create Sentence Embedding
def word_averaging(wv, words):
    all_words, mean = set(), []

    for word in words:
        if isinstance(word, np.ndarray):
            mean.append(word)
        elif word in wv.vocab:
            mean.append(wv.syn0norm[wv.vocab[word].index])
            all_words.add(wv.vocab[word].index)

    if not mean:
        logging.warning("cannot compute similarity with no input %s", words)
        # FIXME: remove these examples in pre-processing
        return np.zeros(wv.vector_size,)

    mean = gensim.matutils.unitvec(np.array(mean).mean(axis=0)).astype(np.float32)
    return mean

def  word_averaging_list(wv, text_list):
    return np.vstack([word_averaging(wv, post) for post in text_list ])

#Loading data
raw_aggregate_tweets = pandas.read_excel('E:\\aggregate.xlsx').iloc[:,0] #Loading all train tweets

raw_train_tweets = pandas.read_excel('E:\\train.xlsx').iloc[:,1] #Loading all train tweets
train_labels = np.array(pandas.read_excel('E:\\train.xlsx').iloc[:,2:13]) #Loading corresponding train labels (11 emotions)

raw_test_tweets = pandas.read_excel('E:\\test.xlsx').iloc[:,1] #Loading 300 test tweets
test_gold_labels = np.array(pandas.read_excel('E:\\test.xlsx').iloc[:,2:13]) #Loading corresponding test labels (11 emotions)
print("please wait")

#Pre-Processing
aggregate_tweets=[]
train_tweets=[]
test_tweets=[]
for tweets in raw_aggregate_tweets:
    aggregate_tweets.append(pre_processor.pre_process_doc(tweets))

for tweets in raw_train_tweets:
    train_tweets.append(pre_processor.pre_process_doc(tweets))

for tweets in raw_test_tweets:
    test_tweets.append(pre_processor.pre_process_doc(tweets))
    
print(len(aggregate_tweets))
#Vectorizing 
w2v_model = gensim.models.Word2Vec(aggregate_tweets, min_count = 10, size = 300, window = 8)

print(w2v_model.wv.vectors.shape)

train_array = word_averaging_list(w2v_model.wv,train_tweets)
test_array = word_averaging_list(w2v_model.wv,test_tweets)

但我收到此错误：

TypeError                                 Traceback (most recent call last)
<ipython-input-1-8a5fe4dbf144> in <module>
    110 print(w2v_model.wv.vectors.shape)
    111 
--> 112 train_array = word_averaging_list(w2v_model.wv,train_tweets)
    113 test_array = word_averaging_list(w2v_model.wv,test_tweets)
    114 

<ipython-input-1-8a5fe4dbf144> in word_averaging_list(wv, text_list)
     70 
     71 def  word_averaging_list(wv, text_list):
---> 72     return np.vstack([word_averaging(wv, post) for post in text_list ])
     73 
     74 #Averaging Words Vectors to Create Sentence Embedding

<ipython-input-1-8a5fe4dbf144> in <listcomp>(.0)
     70 
     71 def  word_averaging_list(wv, text_list):
---> 72     return np.vstack([word_averaging(wv, post) for post in text_list ])
     73 
     74 #Averaging Words Vectors to Create Sentence Embedding

<ipython-input-1-8a5fe4dbf144> in word_averaging(wv, words)
     58             mean.append(word)
     59         elif word in wv.vocab:
---> 60             mean.append(wv.syn0norm[wv.vocab[word].index])
     61             all_words.add(wv.vocab[word].index)
     62 

TypeError: 'NoneType' object is not subscriptable

Answer 1

尚不清楚您的 TextPreProcessor 或 SocialTokenizer 类可能会做什么。您应该编辑您的问题以显示他们的代码，或显示结果文本的一些示例，以确保它符合您的预期。 （例如：显示all_tweets的前几个和最后几个条目。）

您的线路all_tweets = train_tweets.append(test_tweets) 不太可能达到您的预期。 （它将整个列表 test_tweets 作为 all_tweets 的最后一个元素 - 但随后返回您分配给 all_tweets 的 None。您的 Word2Vec 模型可能为空 - 您应该启用 INFO 日志记录到观察其进度并查看异常输出，并在训练后添加代码以打印有关模型的一些详细信息，以确认发生了有用的训练。）

您确定train_tweets 是您的管道与.fit() 对抗的正确格式吗？ （发送到Word2Vectraining 的文本似乎已经通过.split() 进行了标记化，但pandas.Series train_tweets 中的文本可能从未被标记化过。）

通常，一个好主意是启用日志记录，并在每个步骤之后添加更多代码，通过检查属性值或打印较长集合的摘录来确认每个步骤都具有预期的效果。