将多标签的概率二进制值转换为目标标签答案

【问题标题】：Convert probability binary values of multi labels to target labels将多标签的概率二进制值转换为目标标签
【发布时间】：2021-03-18 01:34:15
【问题描述】：

我正在尝试将文本分类为多个标签，并且效果很好，但是由于我想考虑低于 0.5 阈值的预测标签，因此将 predict() 更改为 predict_proba() 以获取标签的所有概率并选择基于不同阈值的值，但我无法将每个标签的二进制概率值转换为实际文本标签。这是可重现的代码：

import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.multiclass import OneVsRestClassifier
from sklearn.preprocessing import MultiLabelBinarizer

X_train = np.array(["new york is a hell of a town",
                "new york was originally dutch",
                "the big apple is great",
                "new york is also called the big apple",
                "nyc is nice",
                "people abbreviate new york city as nyc",
                "the capital of great britain is london",
                "london is in the uk",
                "london is in england",
                "london is in great britain",
                "it rains a lot in london",
                "london hosts the british museum",
                "new york is great and so is london",
                "i like london better than new york"])
y_train_text = [["new york"],["new york"],["new york"],["new york"],["new york"],
            ["new york"],["london"],["london"],["london"],["london"],
            ["london"],["london"],["new york","london"],["new york","london"]]

X_test = np.array(['nice day in nyc',
               'welcome to london',
               'london is rainy',
               'it is raining in britian',
               'it is raining in britian and the big apple',
               'it is raining in britian and nyc',
               'hello welcome to new york. enjoy it here and london too'])
target_names = ['New York', 'London']

lb = MultiLabelBinarizer()
Y = lb.fit_transform(y_train_text)

classifier = Pipeline([
 ('tfidf', TfidfVectorizer()),
 ('clf', OneVsRestClassifier(LinearSVC()))])

classifier.fit(X_train, Y)
predicted = classifier.predict_proba(X_test)

这给了我每个 X_test 值的标签概率值现在，当我尝试 lb.inverse_transform(predicted[0]) 获取第一个 X_test 的实际标签时，它不起作用。

任何帮助，我做错了什么以及如何获得预期的结果。

注意：以上是虚拟数据，但我有500 labels，其中每个特定文本都可以有not more than 5 labels。

【问题讨论】：

标签： python machine-learning scikit-learn multilabel-classification scikit-multilearn

【解决方案1】：

我试图通过index和predicted probalities和predicted probalities并匹配它们以获得实际标签，因为sklearn中没有直接方法。

这里是我的方式。

multilabel = MultiLabelBinarizer()
y = multilabel.fit_transform('target_labels')

predicted_list = classifier.predict_proba(X_test)

def get_labels(predicted_list):
    mlb =[(i1,c1)for i1, c1 in enumerate(multilabel.classes_)]    
    temp_list = sorted([(i,c) for i,c in enumerate(list(predicted_list))],key = lambda x: x[1], reverse=True)
    tag_list = [item1 for item1 in temp_list if item1[1]>=0.35] # here 0.35 is the threshold i choose
    tags = [item[1] for item2 in tag_list[:5] for item in mlb if item2[0] == item[0] ] # here I choose to get top 5 labels only if there are more than that
    return tags

get_labels(predicted_list[0]) 
>> ['New York']

【讨论】：