CountVectorizer 错误：ValueError：使用序列设置数组元素

Question

我有一个包含 144 个学生反馈的数据集，分别有 72 个正面反馈和 72 个负面反馈。数据集有两个属性，即数据和目标，分别包含句子和情绪（正面或负面）。 考虑以下代码：

import pandas as pd
feedback_data = pd.read_csv('output.csv')
print(feedback_data)  


    data    target
0      facilitates good student teacher communication.  positive
1                           lectures are very lengthy.  negative
2             the teacher is very good at interaction.  positive
3                       good at clearing the concepts.  positive
4                       good at clearing the concepts.  positive
5                                    good at teaching.  positive
6                          does not shows test copies.  negative
7                           good subjective knowledge.  positive
8                           good communication skills.  positive
9                               good teaching methods.  positive
10   posseses very good and thorough knowledge of t...  positive
11   posseses superb ability to provide a lots of i...  positive
12   good conceptual skills and knowledge for subject.  positive
13                      no commuication outside class.  negative
14                                     rude behaviour.  negative
15            very negetive attitude towards students.  negative
16   good communication skills, lacks time punctual...  positive
17   explains in a better way by giving practical e...  positive
18                               hardly comes on time.  negative
19                          good communication skills.  positive
20   to make students comfortable with the subject,...  negative
21                       associated to original world.  positive
22                             lacks time punctuality.  negative

from sklearn.feature_extraction.text import CountVectorizer
cv = CountVectorizer(binary = True)
cv.fit(feedback_data['data'].values)
X = feedback_data['data'].apply(lambda X : cv.transform([X])).values
X_test = cv.transform(feedback_data_test)

from sklearn import svm
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split

target = [1 if i<72 else 0 for i in range(144)]
print(target)

X_train, X_val, y_train, y_val = train_test_split(X, target, train_size = 0.50)

clf = svm.SVC(kernel = 'linear', gamma = 0.001, C = 0.05)
#The below line gives the error
clf.fit(X , target)

我不知道出了什么问题。请帮忙

Answer 1

错误来自 X 的方式。您不能在 Fit 方法中直接使用 X。您需要先对其进行更多改造（对于其他问题，我无法告诉您，因为我没有相关信息）

现在您拥有以下内容：

array([<1x23 sparse matrix of type '<class 'numpy.int64'>'
with 5 stored elements in Compressed Sparse Row format>,
   ...
   <1x23 sparse matrix of type '<class 'numpy.int64'>'
with 3 stored elements in Compressed Sparse Row format>], dtype=object)

这足以进行拆分。 我们只是将其转换为您可以理解的，fit 方法也将如此：

X = list([list(x.toarray()[0]) for x in X])

我们所做的是将稀疏矩阵转换为 numpy 数组，获取第一个元素（它只有一个元素），然后将其转换为列表以确保它具有正确的维度。

现在我们为什么要这样做：

X 是这样的

>>>X[0]
   <1x23 sparse matrix of type '<class 'numpy.int64'>'
   with 5 stored elements in Compressed Sparse Row format>

所以我们对其进行改造，看看它到底是什么：

>>>X[0].toarray()
   array([[0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0,
         0]], dtype=int64)

然后如您所见，尺寸存在小问题，因此我们采用第一个元素。

返回列表没有任何作用，只是为了让您更好地理解您所看到的。 （您可以将其转储以提高速度）

您的代码现在是这样的：

cv = CountVectorizer(binary = True)
cv.fit(df['data'].values)
X = df['data'].apply(lambda X : cv.transform([X])).values
X = list([list(x.toarray()[0]) for x in X])
clf = svm.SVC(kernel = 'linear', gamma = 0.001, C = 0.05)
clf.fit(X, target)