如何使用不同的输入维度执行 KerasClassifier 模型选择

Question

我正在尝试在 KerasClassifier 上为多个样本滑动窗口执行模型选择。但是，每个滑动窗口都有不同的输入维度（作为特征选择的结果）。

我编写的函数适用于常规 scikit-learn 分类器。即它返回一个包含最优 RF 模型的字典（使用随机网格搜索）：

# return a dictionary with optimal models for each sliding window
rf_optimal_models = model_selection(RandomForestClassifier(), 
param_distributions = random_grid_rf, n_iter = 10)

但是，我不确定如何设置 KerasClassifier，使其能够根据传递给它的滑动窗口的尺寸更改 input_dim 参数。

以下代码设置了 keras scikit-learn 包装器。

def create_model(optimizer='adam', kernel_initializer='normal', dropout_rate=0.0):
    with tf.device("/device:GPU:0"):
        # create model
        model = Sequential()
        model.add(Dense(20, input_dim=X_train.shape[1], activation='relu', kernel_initializer=kernel_initializer))
        model.add(Dropout(dropout_rate))
        model.add(Dense(20, activation='relu'))
        model.add(Dense(1, activation='sigmoid'))
        # Compile model
        model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])
        return model

... 以及对我的 model_selection() 函数的调用。

mlp_optimal_models =  model_selection(model = KerasClassifier(build_fn=create_model, verbose=0,), param_distributions = random_grid_mlp, n_iter = 10)

input_dim 参数是静态的，当它接收到 49 的尺寸（下一个滑动窗口的输入尺寸）时会抛出错误，但预期为 42。

ValueError: Error when checking input: expected dense_1_input to have shape 
(42,) but got array with shape (49,)

下面的代码是我的 model_selection() 函数的简化版本：

def model_selection(model, param_distributions, n_iter = 100):
    """
    This function performs model selection using random grid search *without cross validation*.
    Inputs:
        model: enter model such as RandomForestClassifier() (which is default)
        param_distributions: pre-defined grid to search over, specific to the input 'model'
        n_iter: Number of parameter settings that are sampled. n_iter trades off runtime vs quality of the solution.
    """

    # dictionary to hold optimal models for each sliding window
    optimal_models = {}

    # 'sets' is a dictionary containing sliding window dataframes e.g. 'X_train_0', 'y_train_0', 'X_test_0', 'y_test_0', 'X_train_1', 'y_train_1', 'X_test_1', 'y_test_1'

    for i in np.arange(0, len(sets), 4): # for each sliding window

            # assign the train and validation sets for the given sliding window
            X_train = list(sets_for_model_selection.values())[i] # THESE HAVE DIFFERENT DIMS FROM WINDOW TO WINDOW
            X_val = list(sets_for_model_selection.values())[i+1] # THESE HAVE DIFFERENT DIMS FROM WINDOW TO WINDOW
            y_train = list(sets_for_model_selection.values())[i+2]
            y_val = list(sets_for_model_selection.values())[i+3]

            # set up the grid search
            mdl_opt = RandomizedSearchCV(estimator = model, param_distributions = param_distributions, 
                                 n_iter = n_iter, cv = ps, verbose=2)

            # Fit the random search model: parameter combinations will be trained, then tested on the validation set
            mdl_opt.fit(np.concatenate((X_train, X_val), axis = 0), 
                np.concatenate((y_train.values.ravel(), y_val.values.ravel()), axis = 0))

            mdl = {'optimal_model_sw'+str(i) : mdl_opt.best_estimator_}

            # update the 'optimal models' dictionary
            optimal_models.update(mdl)

return optimal_models

Answer 1

解决方案涉及对 KerasClassifier 包装器进行轻微编辑并编辑我的函数 model_selection()。

首先我将input_dim 更改为“无”：

def create_model(optimizer='adam', kernel_initializer='normal', dropout_rate=0.0, input_dim=None):
    with tf.device("/device:GPU:0"):
        # create model
        model = Sequential()
        model.add(Dense(20, input_dim=None, activation='relu', kernel_initializer=kernel_initializer))
        model.add(Dropout(dropout_rate))
        model.add(Dense(20, activation='relu'))
        model.add(Dense(1, activation='sigmoid'))
        # Compile model
        model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])
        return model

然后在模型选择函数中，我添加了一个额外的参数“mlp”来断言所讨论的模型是否是神经网络。

如果为 True，KerasClassifier 模型将在 model_selection() 函数中创建，它可以访问相关滑动窗口的维数。这些在KerasClassifier 构造函数中用作input_dim keyarg 的输入（如Vivek Kumar 指出的链接）：

def model_selection(model, param_distributions, n_iter = 100, mlp=None):
    """
    This function performs model selection using random grid search *without cross validation*.
    Inputs:
        model: enter model such as RandomForestClassifier() (which is default)
        param_distributions: pre-defined grid to search over, specific to the input 'model'
        n_iter: Number of parameter settings that are sampled. n_iter trades off runtime vs quality of the solution.
    """
    # dictionary to hold optimal models for each sliding window
    optimal_models = {}

# 'sets' is a dictionary containing sliding window dataframes e.g. 'X_train_0', 'y_train_0', 'X_test_0', 'y_test_0', 'X_train_1', 'y_train_1', 'X_test_1', 'y_test_1'

    for i in np.arange(0, len(sets), 4): # for each sliding window

        # assign the train and validation sets for the given sliding window
        X_train = list(sets_for_model_selection.values())[i] # THESE HAVE DIFFERENT DIMS FROM WINDOW TO WINDOW
        X_val = list(sets_for_model_selection.values())[i+1] # THESE HAVE DIFFERENT DIMS FROM WINDOW TO WINDOW
        y_train = list(sets_for_model_selection.values())[i+2]
        y_val = list(sets_for_model_selection.values())[i+3]

        if mlp:
            input_dims = list(sets_for_model_selection.values())[i].shape[1]
            model =  KerasClassifier(build_fn=create_model, input_dim=input_dims, verbose=0)

        # set up the grid search
        mdl_opt = RandomizedSearchCV(estimator = model, param_distributions = param_distributions, 
                             n_iter = n_iter, cv = ps, verbose=2)

        # Fit the random search model: parameter combinations will be trained, then tested on the validation set
        mdl_opt.fit(np.concatenate((X_train, X_val), axis = 0), 
            np.concatenate((y_train.values.ravel(), y_val.values.ravel()), axis = 0))

        mdl = {'optimal_model_sw'+str(i) : mdl_opt.best_estimator_}

        # update the 'optimal models' dictionary
        optimal_models.update(mdl)
    return optimal_models