具有多输入 KerasClassifier 的 Sklearn cross_val_score

Question

目标是在具有多个输入的 Keras 模型上执行交叉验证。这适用于只有一个输入的正常顺序模型。但是，当使用功能 api 并扩展到两个输入时，sklearns cross_val_score 似乎无法按预期工作。

def create_model():
    input_text = Input(shape=(1,), dtype=tf.string)
    embedding = Lambda(UniversalEmbedding, output_shape=(512, ))(input_text)
    dense = Dense(256, activation='relu')(embedding)

    input_title = Input(shape=(1,), dtype=tf.string)
    embedding_title = Lambda(UniversalEmbedding, output_shape=(512, ))(input_title)
    dense_title = Dense(256, activation='relu')(embedding_title)

    out = Concatenate()([dense, dense_title])

    pred = Dense(2, activation='softmax')(out)
    model = Model(inputs=[input_text, input_title], outputs=pred)
    model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

    return model

失败的部分

keras_classifier = KerasClassifier(build_fn=create_model, epochs=10, batch_size=10, verbose=1)
cv = StratifiedKFold(n_splits=10, random_state=0)
results = cross_val_score(keras_classifier, [X1, X2], y, cv=cv, scoring='f1_weighted')

错误

Traceback (most recent call last):
  File "func.py", line 73, in <module>
    results = cross_val_score(keras_classifier, [X1, X2], y, cv=cv, scoring='f1_weighted')
  File "/home/timisb/.local/lib/python3.6/site-packages/sklearn/model_selection/_validation.py", line 402, in cross_val_score
    error_score=error_score)
  File "/home/timisb/.local/lib/python3.6/site-packages/sklearn/model_selection/_validation.py", line 225, in cross_validate
    X, y, groups = indexable(X, y, groups)
  File "/home/timisb/.local/lib/python3.6/site-packages/sklearn/utils/validation.py", line 260, in indexable
    check_consistent_length(*result)
  File "/home/timisb/.local/lib/python3.6/site-packages/sklearn/utils/validation.py", line 235, in check_consistent_length
    " samples: %r" % [int(l) for l in lengths])
ValueError: Found input variables with inconsistent numbers of samples: [2, 643]

有没有人对此有替代方法或解决方案的建议？谢谢！

Answer 1

您可以运行自己的交叉验证实现。示例 CV 实现可能如下所示：

import numpy as np
from sklearn.model_selection import StratifiedKFold

input_1 = [[1], [2], [3], [4], [5], [6], [7], [8], [9], [10]]
input_2 = [[11], [12], [13], [14], [15], [16], [17], [18], [19], [20]]
Y = [[0], [0], [0], [2], [2], [0], [1], [1], [2], [0]]

# Split a dataset into k folds
def cross_validation_split(X1, X2, Y, folds=4):
    skf = StratifiedKFold(n_splits=4, shuffle = True)
    skf.get_n_splits(X1, Y)
    dataset_split = []
    i = 0
    for train_index, test_index in skf.split(X1, Y):
        print("TRAIN:", train_index, "TEST:", test_index)
        train_index = train_index.astype(int)
        test_index = test_index.astype(int)
        X1 = np.array(X1)
        X2 = np.array(X2)
        Y = np.array(Y)
        X_1_train, X_1_test = X1[train_index], X1[test_index]
        X_2_train, X_2_test = X2[train_index], X2[test_index]
        y_train, y_test = Y[train_index], Y[test_index]
        k_fold_set = {
                    'k_fold': i,
                    'train': {'X_1': X_1_train, 'X_2': X_2_train, 'Y': y_train},
                    'test': {'X_1': X_1_test, 'X_2': X_2_test, 'Y': y_test}
                    }
        dataset_split.append(k_fold_set)
        i = i + 1

    return dataset_split

result = cross_validation_split(input_1, input_2, Y, folds=4)

然后简单地遍历创建的result 列表并执行您的训练/验证逻辑并将结果保存到一个列表中，该列表将为您提供 k 折交叉验证的结果。

Answer 2

我找到了下面的原因。

您可以使用顺序 Keras 模型（仅限单输入）作为 您的 Scikit-Learn 工作流程通过在 keras.wrappers.scikit_learn.py.

https://keras.io/scikit-learn-api/

Answer 3

您正在使用来自scikit-learn 的cross_val_score 函数，表示ValueError: Found input variables with inconsistent numbers of samples: [2, 643]

看起来 sklearn 需要不同的数据 shape。

你可以使用data.reshape()。

一般提示： 首先，我认为交叉验证通常是“没有足够的训练数据”的指标。 Keras 和 TensorFlow 团队一般都不太重视提供 CV 功能。