使用 SVM 的 coef_ 函数的特征重要性

Question

我正在从事一个文本分类项目并尝试使用 SVC(kernel= 'linear') 来获取特征重要性。这是我的代码：
（我把代码从this post改了）

X = df1[features]
y = df1['label']


# Create selector class for text and numbers
class TextSelector(BaseEstimator, TransformerMixin):
    """Transformer to select a single column from the data frame to perform additional transformations on"""
    def __init__(self, key):
        self.key = key

    def fit(self, X, y=None):
        return self

    def transform(self, X):
        return X[self.key]

class NumberSelector(BaseEstimator, TransformerMixin):
    """For data grouped by feature, select subset of data at a provided key."""
    def __init__(self, key):
        self.key = key

    def fit(self, X, y=None):
        return self

    def transform(self, X):
        return X[[self.key]]

scaler = StandardScaler()    
text = Pipeline([
                ('selector', TextSelector(key='title_mainText')),
                ('vect', TfidfVectorizer(ngram_range=(1, 2))),                
            ])

upper_title =  Pipeline([
                ('selector', NumberSelector(key='upper_title')),
                ('standard', scaler),
            ])

upper_mainText =  Pipeline([
                ('selector', NumberSelector(key='upper_mainText')),
                ('standard', scaler),
            ])

punct_title =  Pipeline([
                ('selector', NumberSelector(key='punct_title')),
                ('standard', scaler),
            ])

punct_mainText =  Pipeline([
                ('selector', NumberSelector(key='punct_mainText')),
                ('standard', scaler),
            ])


exclamations_title =  Pipeline([
                ('selector', NumberSelector(key='exclamations_title')),
                ('standard', scaler),
            ])


exclamations_text =  Pipeline([
                ('selector', NumberSelector(key='exclamations_text')),
                ('standard', scaler),
            ])


feats = FeatureUnion([('title_mainText', text), 
                      ('upper_title', upper_title),
                      ('upper_mainText', upper_mainText),
                      ('punct_title', punct_title),
                      ('punct_mainText', punct_mainText),                    
                      ('exclamations_text', exclamations_text),
                      ('exclamations_title', exclamations_title),                        

feature_processing = Pipeline([('feats', feats)])

pipeline = Pipeline([
        ('features', feats),
        ('classifier', SVC(C=1, kernel= 'linear', max_iter= 1000, tol=0.0001, probability=True))
    ])


    def f_importances(coef, names):
        imp = coef
        imp,names = zip(*sorted(zip(imp,names)))
        plt.barh(range(len(names)), imp, align='center')
        plt.yticks(range(len(names)), names)
        plt.show()

    features_names = ['title_mainText', 'upper_title', 'upper_mainText', 'punct_title', 'punct_mainText',
                      'exclamations_title', 'exclamations_text']
    pipeline.fit(X, y)
    clf = pipeline.named_steps['classifier']
    f_importances(clf.coef_, features_names)

但是，它显示一条错误消息，我不知道我在哪里做错了。 以前有人有过这方面的经验吗？

ValueError Traceback（最近调用 最后）在（） 13 管道.fit(X, y) 14 clf = pipeline.named_steps['分类器'] ---> 15 f_importances((clf.coef_[0]), features_names) 16

in f_importances(coef, names) 5 imp = coef 6 imp,names = zip(*sorted(zip(imp,names))) ----> 7 plt.barh(range(len(names)), imp, align='center') 8 plt.yticks（范围（len（名称）），名称） 9 plt.show()

/anaconda3/lib/python3.6/site-packages/matplotlib/pyplot.py 在 barh(*args, **kwargs) 2667 mplDeprecation)
2668 尝试： -> 2669 ret = ax.barh(*args, **kwargs) 2670 最后：2671 ax._hold = washold

/anaconda3/lib/python3.6/site-packages/matplotlib/axes/_axes.py 在 barh(self, *args, **kwargs) 2281
kwargs.setdefault('orientation', 'horizo​​ntal') 2282 个补丁 = self.bar(x=left, height=height, width=width, -> 2283 bottom=y, **kwargs) 2284 返回补丁 2285

/anaconda3/lib/python3.6/site-packages/matplotlib/init.py 内部（斧头，*args，**kwargs）1715
warnings.warn(msg % (label_namer, func.name), 1716
运行时警告，堆栈级别 = 2) -> 1717 return func(ax, *args, **kwargs) 1718 pre_doc = inner.doc 1719 如果 pre_doc 为 None：

/anaconda3/lib/python3.6/site-packages/matplotlib/axes/_axes.py 在 bar(self, *args, **kwargs) 2091 elif 方向 == “水平”：2092 r.sticky_edges.x.append(l) -> 2093 self.add_patch(r) 2094 patch.append(r) 2095

/anaconda3/lib/python3.6/site-packages/matplotlib/axes/_base.py 在 add_patch(self, p) 1852 如果 p.get_clip_path() 为无：
第1853章 -> 1854 self._update_patch_limits(p) 1855 self.patches.append(p) 1856 p._remove_method = lambda h: self.patches.remove(h)

/anaconda3/lib/python3.6/site-packages/matplotlib/axes/_base.py 在 _update_patch_limits(self, patch) 1868 # 或高度。第1869章 -> 1870 ((not patch.get_width()) 和 (not patch.get_height()))): 1871 返回 1872
vertices = patch.get_path().vertices

/anaconda3/lib/python3.6/site-packages/scipy/sparse/base.py 在 bool（自我） 286返回self.nnz！= 0 287 其他： --> 288 raise ValueError("一个以上数组的真值" 289 “元素不明确。使用 a.any() 或 a.all()。”) 290 非零 = bool

ValueError：具有多个元素的数组的真值是 模糊的。使用 a.any() 或 a.all()。

谢谢！

Answer 1

Scikit-Learn 的文档 states 认为 coef_ 属性是一个 shape = [n_class * (n_class-1) / 2, n_features] 的数组。假设有 4 个类和 9 个特征，_coef 的形状为 6 x 9（六行九列）。另一方面，barh 期望每个功能都有一个值而不是六个，因此您会收到错误消息。如果您将每列的系数相加，则可以消除它，如下例所示。

import numpy as np
import matplotlib.pyplot as plt

def f_importances(coef, names):
    imp = coef
    imp,names = zip(*sorted(zip(imp,names)))
    plt.barh(range(len(names)), imp, align='center')
    plt.yticks(range(len(names)), names)
    plt.show()

features_names = ['title_mainText', 'upper_title', 'upper_mainText', 'punct_title', 'punct_mainText',
                  'exclamations_title', 'exclamations_text', 'title_words_not_stopword', 'text_words_not_stopword']

n_classes = 4
n_features = len(features_names)

clf_coef_ = np.random.randint(1, 30, size=(int(0.5*n_classes*(n_classes-1)), n_features))

f_importances(clf_coef_.sum(axis=0), features_names)