Scikit Learn ValueError：找到暗淡 3 的数组。估计器预期 <= 2

Question

我有一个包含 144 个学生反馈的训练数据集，分别有 72 个正面反馈和 72 个负面反馈。该数据集有两个属性，即数据和目标，分别包含句子和情绪（正面或负面）。测试数据集包含 106 个未标记的反馈。 考虑以下代码：

import pandas as pd
feedback_data = pd.read_csv('output_svm.csv')
print(feedback_data)


data    target
0      facilitates good student teacher communication.  positive
1                           lectures are very lengthy.  negative
2             the teacher is very good at interaction.  positive
3                       good at clearing the concepts.  positive
4                       good at clearing the concepts.  positive
5                                    good at teaching.  positive
6                          does not shows test copies.  negative
7                           good subjective knowledge.  positive
8                           good communication skills.  positive
9                               good teaching methods.  positive
10   posseses very good and thorough knowledge of t...  positive

feedback_data_test = pd.read_csv('classified_feedbacks_test.csv')
print(feedback_data_test)

          data  target
0                                       good teaching.     NaN
1                                         punctuality.     NaN
2                    provides good practical examples.     NaN
3                              weak subject knowledge.     NaN
4                                   excellent teacher.     NaN
5                                         no strength.     NaN
6                      very poor communication skills.     NaN
7                      not able to clear the concepts.     NaN
8                                            punctual.     NaN
9                             lack of proper guidance.     NaN
10                                  fantastic speaker.     NaN
from sklearn.feature_extraction.text import CountVectorizer
cv = CountVectorizer(binary = True)
ct = CountVectorizer(binary= True)
cv.fit(feedback_data['data'].values)
ct.fit(feedback_data_test['data'].values)
X = feedback_data['data'].apply(lambda X : cv.transform([X])).values
X = list([list(x.toarray()[0]) for x in X])
X_test = feedback_data_test['data'].apply(lambda X_test : ct.transform([X_test])).values
X_test = list([list(x.toarray()[0]) for x in X_test])




from sklearn import svm
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
target = [1 if i<72 else 0 for i in range(144)]
X_train, X_val, y_train, y_val = train_test_split(X, target, train_size = 0.50)
clf = svm.SVC(kernel = 'linear', gamma = 0.001, C = 0.05)
clf.fit(X, target)
#The below line gives error
print("Accuracy = %s" %accuracy_score(target,clf.predict([X_test])) )

我不知道出了什么问题。请帮忙。

Answer 1

你得到的错误不是样本数量而是特征数量，这来自那些代码行：

cv = CountVectorizer(binary = True)
ct = CountVectorizer(binary= True)
cv.fit(feedback_data['data'].values)
ct.fit(feedback_data_test['data'].values)

您需要以相同的方式对测试和训练进行编码

您将 Count Vectorizer 应用于所有数据，然后将其应用于测试和训练，如果不是，则您没有相同的词汇表，因此编码也不同。

cv = CountVectorizer(binary = True)
cv.fit(np.concatenate((feedback_data['data'].values,feedback_data_test['data'].values))

编辑

你只是不使用 ct，只使用 cv

X = feedback_data['data'].apply(lambda X : cv.transform([X])).values
X = list([list(x.toarray()[0]) for x in X])
X_test = feedback_data_test['data'].apply(lambda X_test :cv.transform([X_test])).values
X_test = list([list(x.toarray()[0]) for x in X_test])