【发布时间】:2016-01-23 18:33:24
【问题描述】:
我正在尝试使用 tfidf 和朴素贝叶斯分类器对我的文本数据进行分类
cls = MultinomialNB()
vec = TfidfVectorizer(input='file', analyzer=word_tokenize, stop_words=stop_w, use_idf=False)
for i, filename in enumerate(files):
with codecs.open(filename, encoding='utf8') as f:
bow = vec.fit_transform(f)
# and i have one target for this bow. (each file has unique subject)
y = np.array([repeat(i, times=41253)])
cls.fit(bow, y)
bow.shape 输出是这样的
(41253, 15987)
但是遇到了这个异常
Traceback (most recent call last):
File "/home/x/PycharmProjects/PWC/naiive.py", line 35, in <module>
cls.fit(bow, y)
File "/usr/local/lib/python2.7/dist-packages/sklearn/naive_bayes.py", line 522, in fit
X, y = check_X_y(X, y, 'csr')
File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 516, in check_X_y
check_consistent_length(X, y)
File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 176, in check_consistent_length
"%s" % str(uniques))
ValueError: Found arrays with inconsistent numbers of samples: [ 1 41253]
我知道我的 y 尺寸/形状有问题,但我不知道该如何解决 首先我的 y 实现是否正确?
【问题讨论】:
-
请始终提供完整的错误回溯,而不仅仅是最后一行。
标签: python nlp scikit-learn