TypeError：稀疏矩阵长度不明确；使用 getnnz() 或 shape[0]答案

【问题标题】：TypeError: sparse matrix length is ambiguous; use getnnz() or shape[0]TypeError：稀疏矩阵长度不明确；使用 getnnz() 或 shape[0]
【发布时间】：2020-07-30 04:53:23
【问题描述】：

我正在尝试在不使用 SKlearn MNB 的情况下制作多项朴素贝叶斯分类器。

这是分类器的代码：

class MultinomialNaiveBayes:

def __init__(self):
    # count is a dictionary which stores several dictionaries corresponding to each news category
    # each value in the subdictionary represents the freq of the key corresponding to that news category 
    self.count = {}
    # classes represents the different news categories
    self.classes = None

def fit(self,X_train,Y_train):
    # This can take some time to complete       
    self.classes = set(Y_train)
    for class_ in self.classes:
        self.count[class_] = {}
        for i in range(len(X_train[0])):
            self.count[class_][i] = 0
        self.count[class_]['total'] = 0
        self.count[class_]['total_points'] = 0
    self.count['total_points'] = len(X_train)

    for i in range(len(X_train)):
        for j in range(len(X_train[0])):
            self.count[Y_train[i]][j]+=X_train[i][j]
            self.count[Y_train[i]]['total']+=X_train[i][j]
        self.count[Y_train[i]]['total_points']+=1

def __probability(self,test_point,class_):

    log_prob = np.log(self.count[class_]['total_points']) - np.log(self.count['total_points'])
    total_words = len(test_point)
    for i in range(len(test_point)):
        current_word_prob = test_point[i]*(np.log(self.count[class_][i]+1)-np.log(self.count[class_]['total']+total_words))
        log_prob += current_word_prob

    return log_prob


def __predictSinglePoint(self,test_point):

    best_class = None
    best_prob = None
    first_run = True

    for class_ in self.classes:
        log_probability_current_class = self.__probability(test_point,class_)
        if (first_run) or (log_probability_current_class > best_prob) :
            best_class = class_
            best_prob = log_probability_current_class
            first_run = False

    return best_class


def predict(self,X_test):
    # This can take some time to complete
    Y_pred = [] 
    for i in range(X_test.size):
    # print(i) # Uncomment to see progress
        Y_pred.append( self.__predictSinglePoint(X_test[i]) )

    return Y_pred

def score(self,Y_pred,Y_true):
    # returns the mean accuracy
    count = 0
    for i in range(len(Y_pred)):
        if Y_pred[i] == Y_true[i]:
            count+=1
    return count/len(Y_pred)

但是，当我尝试将训练数据拟合到这个模型时，会出现这样的错误。

TypeError：稀疏矩阵长度不明确；使用 getnnz() 或 shape[0]

然后我尝试将我的火车数据更改为密集数组。但这导致了这样的错误。

IndexError: 数组索引过多

我该怎么办？我使用 tfidf 矢量化器对我的火车数据进行了预处理，所以我的火车数据看起来像这样。

(0, 3838)   0.3188116041495794
(0, 1314)   0.47611391519517965
(0, 3852)   0.6748521310739798
(0, 4460)   0.46502613045074925
(1, 2997)   0.5124730713021854
(1, 7283)   0.37699134972197723
(1, 7363)   0.29325712956005184
(1, 3226)   0.3049856710575078
(1, 7461)   0.44783186231025657
(1, 2275)   0.3687679018146693
(1, 3308)   0.28229369020160777

如果我的问题重复，我很抱歉，因为我仍然没有得到解决方案。谢谢。

【问题讨论】：

标签： python machine-learning nlp naivebayes multinomial

【解决方案1】：

您应该用shape[0] 替换代码中对X_train（又称稀疏矩阵）的任何len() 调用，例如：

def fit(self,X_train,Y_train):
    # This can take some time to complete       
    self.classes = set(Y_train)
    for class_ in self.classes:
        self.count[class_] = {}
        for i in range(X_train[0].shape[0]):
            self.count[class_][i] = 0
        self.count[class_]['total'] = 0
        self.count[class_]['total_points'] = 0
    self.count['total_points'] = X_train.shape[0]

    for i in range(X_train.shape[0]):
        for j in range(X_train[0].shape[0]):
            self.count[Y_train[i]][j]+=X_train[i][j]
            self.count[Y_train[i]]['total']+=X_train[i][j]
        self.count[Y_train[i]]['total_points']+=1

【讨论】：