LinearSVC 参数贝叶斯优化答案

【问题标题】：LinearSVC parameters Bayesian optimizationLinearSVC 参数贝叶斯优化
【发布时间】：2019-09-22 16:59:43
【问题描述】：

我在多标签分类问题中对 LinearSVC 进行贝叶斯优化时遇到 ValueError。

logger = JSONLogger(path=LOGS_PATH)


lSVC_param = {'C':(0.001, 0.01, 0.1, 1, 10),
                   'penalty':('l1','l2'),
                  'loss':('hinge','squared_hinge')}


def optimise_bayes_opt(X, y):
    def target(C_param,penalty_param,loss_param):       
        clf = LinearSVC(C=C_param,penalty=penalty_param,loss=loss_param)
        text_clf = Pipeline([('tfidf', TfidfVectorizer(ngram_range=(1,1),
                                                      norm='l2',
                                                      min_df=1,
                                                      use_idf=True)), 
                             ('clf', OneVsRestClassifier(clf))])
        cv_results = cross_val_score(text_clf, X_test, y_test, scoring='accuracy',cv=5)        
        print("CV",cv_results,cv_results.mean())
        return cv_results.mean()

    optimizer = BayesianOptimization(
                f=target,
                pbounds={'C_param':lSVC_param['C'],
                         'penalty_param':lSVC_param['penalty'],
                         'loss_param':lSVC_param['loss']},
                verbose=2,
                random_state=1)
    optimizer.subscribe(Events.OPTMIZATION_STEP, logger)
    optimizer.maximize(init_points=2, n_iter=2)
    return optimizer


with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    optimizer = optimise_bayes_opt(X_train,y_train)
    best_params = optimizer.max 

print(best_params)

错误

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-269-dc7962d023ef> in <module>
     34 with warnings.catch_warnings():
     35     warnings.simplefilter("ignore")
---> 36     optimizer = optimise_bayes_opt(X_train,y_train)
     37     best_params = optimizer.max
     38 

<ipython-input-269-dc7962d023ef> in optimise_bayes_opt(X, y)
     26                          'loss_param':lSVC_param['loss']},
     27                 verbose=2,
---> 28                 random_state=1)
     29     optimizer.subscribe(Events.OPTMIZATION_STEP, logger)
     30     optimizer.maximize(init_points=2, n_iter=2)

~/.local/lib/python3.6/site-packages/bayes_opt/bayesian_optimization.py in __init__(self, f, pbounds, random_state, verbose)
     71         # Data structure containing the function to be optimized, the bounds of
     72         # its domain, and a record of the evaluations we have done so far
---> 73         self._space = TargetSpace(f, pbounds, random_state)
     74 
     75         # queue

~/.local/lib/python3.6/site-packages/bayes_opt/target_space.py in __init__(self, target_func, pbounds, random_state)
     47         self._bounds = np.array(
     48             [item[1] for item in sorted(pbounds.items(), key=lambda x: x[0])],
---> 49             dtype=np.float
     50         )
     51 

ValueError: setting an array element with a sequence.

在引用了这个question 之后，我理解了这个valueError。此外，该行提到值应该是浮点类型。所以 C_param 的值是正确的，而其他的值是不正确的。现在我不知道如何优化具有非浮点值的参数，如惩罚、损失等。

【问题讨论】：

标签： python-3.x machine-learning optimization grid-search hyperparameters

【解决方案1】：

因为输入列表的形状不是可以变成多维数组的（广义）“盒子”。所以可能

  [item[1] for item in sorted(pbounds.items(), key=lambda x: x[0])]

包含不同长度的序列。

尝试改变

lSVC_param = {'C':(0.001, 0.01, 0.1, 1, 10),
                   'penalty':('l1','l2'),
                  'loss':('hinge','squared_hinge')

发送到pbounds的长度相同

【讨论】：