ML 中的文本和虚拟变量 - 特征选择

Question

我有一个这样的数据框：

       Text                                             A   B   C    Label
337 nobodi can explain gave what we did ...             0   1   0      1
338 provide an example                                  1   1   0      0
339 another one????                                     1   0   0      1

我想了解如何构建 ML 分类器。 目前，我做了如下：

X = train[['Text','A','B','C']]
y = train['Label']

# Split into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=40)
X_train, X_valid, y_train, y_valid  = train_test_split(X, y, test_size=0.25, random_state=40) 
# Returning to one dataframe
train_df = pd.concat([X_train, y_train], axis=1)
test_df = pd.concat([X_test, y_test], axis=1)
valid_df = pd.concat([X_valid, y_valid], axis=1)

然后我使用 BOW 和 TFIDF 创建特征：

countV = CountVectorizer()
train_count = countV.fit_transform(train_df['Text'].values)

# To create tfidf frequency features

tfidfV = TfidfTransformer()
train_tfidf = tfidfV.fit_transform(train_count)

tfidf_ngram = TfidfVectorizer(stop_words='english',ngram_range=. (1,2),use_idf=True,smooth_idf=True)

但是，当我构建模型时，例如 NB 模型：

nb_pipeline = Pipeline([
        ('NBCV', countV),
        ('nb_clf',MultinomialNB())])

nb_pipeline.fit(train_df['Text'],train_df['Label'])
predicted_nb = nb_pipeline.predict(test_df['Text'])
np.mean(predicted_nb == test_df['Label'])

有些东西不起作用，因为我丢失了关于我的虚拟变量 A、B、C 的信息。我只有 Text 的功能。当我尝试查看功能重要性时，我可以检查一下：

feature_names = nb_pipeline.named_steps["NBCV"].get_feature_names()
coefs = nb_pipeline.named_steps["nb_clf"].coef_.flatten()

import pandas as pd
zipped = zip(feature_names, coefs)
df = pd.DataFrame(zipped, columns=["feature", "value"])
df["ABS"] = df["value"].apply(lambda x: abs(x))
df["colors"] = df["value"].apply(lambda x: "green" if x > 0 else "red")
df = df.sort_values("ABS", ascending=True)

您能解释一下为什么我会丢失这些信息以及如何将我的虚拟变量保留在模型中吗？这些变量对模型应该非常有意义，因此我不能将它们从模型构建中排除。我需要检查模型的准确性并查看这些变量对此的影响。

Answer 1

这里有一些东西要解压，所以让我们来看看它们：

验证、测试、训练数据

首先，切勿混合您的验证和测试数据。 This article 提供了来自各种学术教科书的超过 5 种不同的引用，包括 Max Kuhn 等领先的行业领导者，这些引用特别强调了为什么您需要为模型提供完全未触及和看不见的数据以进行最终评估。我建议阅读。

让管道正常工作

为了让您的pipeline 正常工作，您需要使用 [make_column_transformer][3]，这是 sklearn v 20.0.0 的新功能。我试图从您的示例中回收尽可能多的代码，因此请注意任何细微的差异。

#!/usr/bin/env python
# coding: utf-8

import pandas as pd
import numpy as np
from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.naive_bayes import MultinomialNB

# Added
from sklearn.compose import make_column_transformer

# Create some test data to mimic OP's example

bad_sentences = ["I hated this restauraunt!", "I do not like Stack Overflow", "I am sad about the economy", 
                 "I do not feel good about the new Python update", "I think I am allergic to hazelnut!", "I will never eat here",
                 "I am never coming back!", "I feel really sad after hearing the news", "George is getting upset!", "Python!"]
good_sentences = ["I really liked this place", "Stack Overflow is a great resource", "I am glad we are getting vaccinated",
                  "I love anime", "I miss you too!", "I am upset we don't hang out more, I love it!", 
                  "Why are we feeling sad? Be happy!", "Hi, my name is wundermahn, what is your name?", "Lol!", 
                  "I am the master of my domain"]

# Create things a bit more verbosely so OP understands what we are doing

sentences = bad_sentences+good_sentences

a_bad = [1 for i in range(len(bad_sentences))]
a_good = [0 for i in range(len(good_sentences))]
a = a_bad+a_good

b_bad = [1 for i in range(len(bad_sentences))]
b_good = [0 for i in range(len(good_sentences))]
b = b_bad+b_good

c_bad = [1 for i in range(len(bad_sentences))]
c_good = [0 for i in range(len(good_sentences))]
c = c_bad+c_good

label_bad = [1 for i in range(len(bad_sentences))]
label_good = [0 for i in range(len(good_sentences))]
label = label_bad+label_good

# Create dataframe
df = pd.DataFrame({'Text': sentences, 'A': a, 'B': b, 'C': c, 'label': label})

# NEVER mix your validation and testing data!
# https://stackoverflow.com/questions/28556942/pandas-remove-rows-at-random-without-shuffling-dataset

np.random.seed(38)
remove_n = 2
drop_indices = np.random.choice(df.index, remove_n, replace=False)
valid_df = df.iloc[drop_indices]
remaining_df = df.drop(drop_indices)

X_train, X_test, y_train, y_test = train_test_split(remaining_df[[col for col in remaining_df.columns if col != 'label']],
                                                    remaining_df['label'], test_size=0.2, random_state=38)

# Get CountVectorizer working as an example, you can add tfidf later on

# Now, create your pipeline which should include your vectorizer, as well as your model you plan on training
nb_pipeline = Pipeline([
                        ('vectorizer', make_column_transformer((CountVectorizer(), 'Text'), remainder='passthrough')),
                        ('classifier', MultinomialNB())
                      ])

# Now, we can effectively train our model using the proper feature set
nb_pipeline.fit(X_train, y_train)


# Now, get prediction
predicted_nb = nb_pipeline.predict(X_test)

# Print accuracy
print(np.mean(predicted_nb==y_test))

# Get feature names

# Note, we need to slightly edit how we get the names now that we are using a different transformation pipeline
# https://stackoverflow.com/questions/54646709/sklearn-pipeline-get-feature-names-after-onehotencode-in-columntransformer
feature_names = nb_pipeline['vectorizer'].transformers_[0][1].get_feature_names()
coefs = nb_pipeline.named_steps["classifier"].coef_.flatten()

# Your code
zipped = zip(feature_names, coefs)
features_df = pd.DataFrame(zipped, columns=["feature", "value"])
features_df["ABS"] = features_df["value"].apply(lambda x: abs(x))
features_df["colors"] = features_df["value"].apply(lambda x: "green" if x > 0 else "red")
features_df = df.sort_values("ABS", ascending=True)

# See results on validation set
valid_preds = nb_pipeline.predict(valid_df[['Text', 'A', 'B', 'C']])
print(np.mean(valid_preds==valid_df['label']))

请注意，我使用的是：Python 3.8.8, sklearn 0.24.1, pandas 1.2.4, numpy 1.20.2