为什么多标签分类不能对训练数据（sklearn）进行 100% 分类？

Question

我有以下数据：

 [
{"Q" : "What nationality is Laplace?", "Q_TYPE_COURSE" : ["LOCATION", "DESCRIPTION"], "Q_TYPE_FINE" : ["LOCATION-COUNTRY", "DESCRIPTION-DESCRIPTION"] },
{"Q" : "Who wrote 'Celestial Mechanics'?", "Q_TYPE_COURSE" : ["HUMAN"], "Q_TYPE_FINE" : ["HUMAN-IND"]},
{"Q" : "Who created Laplace's equation?", "Q_TYPE_COURSE" : ["HUMAN"], "Q_TYPE_FINE" : ["HUMAN-IND"]},
{"Q" : "What operator is named after Laplace?", "Q_TYPE_COURSE" : ["ENTITY"], "Q_TYPE_FINE" : ["ENTITY-SYMBOL","ENTITY-WORD","ENTITY-CREATIVE"]},
{"Q" : "Who was one of the first scientists to postulate the existence of black holes?", "Q_TYPE_COURSE" : ["HUMAN"], "Q_TYPE_FINE" : ["HUMAN-IND"]},
{"Q" : "Who was one of Napoleon's examiners while he was in school?", "Q_TYPE_COURSE" : ["HUMAN"], "Q_TYPE_FINE" : ["HUMAN-IND"]},
{"Q" : "Where was Laplace born?", "Q_TYPE_COURSE" : ["LOCATION"], "Q_TYPE_FINE" : ["LOCATION-CITY","LOCATION-STATE","LOCATION-COUNTRY"]},
{"Q" : "Where did Laplace go to school?", "Q_TYPE_COURSE" : ["LOCATION", "ENTITY"], "Q_TYPE_FINE" : ["LOCATION-CITY","LOCATION-STATE","LOCATION-OTHER","ENTITY-OTHER"]},
{"Q" : "What did Laplace think of d'Alembert?", "Q_TYPE_COURSE" : ["DESCRIPTION","HUMAN"], "Q_TYPE_FINE" : ["DESCRIPTION-REASON", "HUMAN-DESCRIPTION"]},
{"Q" : "What did d'Alembert think of Laplace?", "Q_TYPE_COURSE" : ["DESCRIPTION","HUMAN"], "Q_TYPE_FINE" : ["DESCRIPTION-REASON", "HUMAN-DESCRIPTION"]},
{"Q" : "When did Laplace become a member of the Academie Des Sciences?", "Q_TYPE_COURSE" : ["NUMERIC"], "Q_TYPE_FINE" : ["NUMERIC-DATE"]},
{"Q" : "Are Laplace's theories on celestial motion sufficient to describe the stability of the Solar System?", "Q_TYPE_COURSE" : ["DESCRIPTION"], "Q_TYPE_FINE" : ["DESCRIPTION-REASON"]},
{"Q" : "How did Laplace's theory of ocean tides differ from that of Newton or Bernoulli?", "Q_TYPE_COURSE" : ["DESCRIPTION"], "Q_TYPE_FINE" : ["DESCRIPTION-REASON","DESCRIPTION-DESCRIPTION"]},
{"Q" : "What sequence of functions, made by Legendre, did Laplace expand on?", "Q_TYPE_COURSE" : ["ENTITY"], "Q_TYPE_FINE" : ["ENTITY-SYMBOL", "ENTITY-CREATIVE"]},
{"Q" : "What is a potential function?", "Q_TYPE_COURSE" : ["ENTITY","DESCRIPTION"], "Q_TYPE_FINE" : ["ENTITY-SYMBOL","DESCRIPTION-DESCRIPTION"]},
{"Q" : "In what year did Laplace publish his book?", "Q_TYPE_COURSE" : ["NUMERIC"], "Q_TYPE_FINE" : ["NUMERIC-DATE"]},
{"Q" : "What hypothesis was Laplace known for?", "Q_TYPE_COURSE" : ["DESCRIPTION","ENTITY"], "Q_TYPE_FINE" : ["DESCRIPTION-DESCRIPTION","ENTITY-CREATIVE"]},
{"Q" : "What did Laplace do in statistics?", "Q_TYPE_COURSE" : ["DESCRIPTION", "ENTITY"], "Q_TYPE_FINE" : ["DESCRIPTION-DESCRIPTION", "ENTITY-CREATIVE"]},
{"Q" : "Was Laplace involved in politics?", "Q_TYPE_COURSE" : ["DESCRIPTION", "HUMAN"], "Q_TYPE_FINE" : ["DESCRIPTION-DESCRIPTION", "HUMAN-DESCRIPTION"]},
{"Q" : "What are Laplace's thoughts on governance?", "Q_TYPE_COURSE" : ["DESCRIPTION"], "Q_TYPE_FINE" : ["DESCRIPTION-REASON", "DESCRIPTION-DESCRIPTION"]},
{"Q" : "Where did Laplace die?", "Q_TYPE_COURSE" : ["LOCATION"], "Q_TYPE_FINE" : ["LOCATION-CITY", "LOCATION-COUNTRY", "LOCATION-STATE"]},
{"Q" : "What was Laplace's full name?", "Q_TYPE_COURSE" : ["HUMAN"], "Q_TYPE_FINE" : ["HUMAN-TITLE"]}
 ]

我只使用 LI 和 ROTH “Q_TYPE_COURSE”标签。我有一个特征提取类，其中提取特征然后使用以下两种方法将其转换为向量形式（feature_extractor.create_features 方法将 spacy 文档作为输入并返回字符串特征列表）（向量化特征方法的输出为create_features 并将其转换为 1D coo_matrix，然后转换为 numpy 数组进行预测，其中 self 是 qa_classifier 类）：

 feature_extractor.create_features(nlp(doc["Q"]), ngram_range=(1,3), lemmatize=True)

 self.vectorize_features(self, features)

现在这里是 qa_classifier 类定义（假设 ngram 范围和 lemmatize 值始终一致）

class qa_classifier(feature_extractor):

clfs = []
mlb = MultiLabelBinarizer()
def _dummy_fun(s): return s
vectorizer = TfidfVectorizer(analyzer="word", tokenizer=_dummy_fun, preprocessor=_dummy_fun, token_pattern=None, norm="l2") 


def __init__(self, questions, tags, ngram_range=(2,2), lemmatize=False): 

    #train_questions is a list of lists of strings
    #assume that questions have already had the feature_extractor.create_features(question) method called on them
    #tags are strings
    self.ngram_range = ngram_range
    self.lemmatize = lemmatize

    self.q_matrix = qa_classifier.vectorizer.fit_transform(questions)

    self.tags_matrix = qa_classifier.mlb.fit_transform(tags)

    #here try to make a classifier for each tag
    for tag_idx in range(len(qa_classifier.mlb.classes_)):
        clf = svm.LinearSVC()
        clf.fit(self.q_matrix, self.tags_matrix.take(indices=tag_idx,axis=1))
        qa_classifier.clfs.append(clf)

def vectorize_features(self, features):
     #here we just use the q_matrix to turn new features into tf_idf docs
     #returns coo_matrix representing the feature vector

def predict(self,query):
    # query is a spacy doc
    query_features = feature_extractor.create_features(query) #turns a spacy doc into a list of strings
    feature_vector = self.vectorize_features(query_features).T.toarray()

    tags = []
    for tag_idx, clf in enumerate(qa_classifier.clfs):
        if clf.predict(feature_vector)[0] == 1:
            tags.append(qa_classifier.mlb.classes_[tag_idx])
    return tags

所以基本上我有各种标签，并使用多标签二值化器为每个标签创建二元分类器，并在预测方法中在文档上运行每个分类器。但是，当我在上面显示的训练数据上运行 predict 方法时，它并没有 100% 重新创建标签，这是怎么回事？

Answer 1

因为您使用的是线性分类器。

clf = svm.LinearSVC()

您的数据集不是线性可分的。这意味着，如果您要在图表上绘制数据点，您将无法在它们之间画一条线，从而可以将所有带有标签 X 的类标签完全放在一侧，而在另一侧说标签 Y。这取决于数据点本身，这个问题可以通过一些非线性分类器理想地解决，该分类器可以通过在它们周围绘制曲线来覆盖所有点。

顺便说一句，100% 的训练准确率通常是一个非常糟糕的主意，这意味着您的模型过度拟合并且无法很好地泛化到它以前从未见过的数据。