根据文档，PyGAD 未接收整数参数

Question

我正在尝试使用 PyGAD 来优化 ML 模型中的超参数。根据documentation

gene_space 参数自定义每个基因的值的空间...列表、元组、numpy.ndarray 或任何范围，如 range、numpy.arange() 或 numpy.linspace：它保存每个单独基因的空间。但是这个空间通常是离散的。也就是说，有一组有限值可供选择。

如您所见，gene_space 的第一个元素，对应于遗传算法定义中的solution[0]，是一个整数数组。根据文档，这应该是一个离散空间，它就是。然而，当这个整数数组（来自np.linspace，可以使用）时，它被随机森林分类器解释为numpy.float64'>（参见第三个代码块中的错误。）

我不明白这种数据类型的变化发生在哪里。这是一个 PyGAD 问题吗？我该如何解决？或者它是一个 numpy -> sklearn 问题？

gene_space = [ 
    # n_estimators
    np.linspace(50,200,25, dtype='int'),
    # min_samples_split, 
    np.linspace(2,10,5, dtype='int'),
    # min_samples_leaf,
    np.linspace(1,10,5, dtype='int'),
    # min_impurity_decrease
    np.linspace(0,1,10, dtype='float')
]

遗传算法的定义

def fitness_function_factory(data=data, y_name='y', sample_size=100):

    def fitness_function(solution, solution_idx):
        model = RandomForestClassifier(
            n_estimators=solution[0],
            min_samples_split=solution[1],
            min_samples_leaf=solution[2],
            min_impurity_decrease=solution[3]
        )
        
        X = data.drop(columns=[y_name])
        y = data[y_name]
        X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                            test_size=0.5)        

        train_idx = sample_without_replacement(n_population=len(X_train), 
                                              n_samples=sample_size)         
        
        test_idx = sample_without_replacement(n_population=len(X_test), 
                                              n_samples=sample_size) 
         
        model.fit(X_train.iloc[train_idx], y_train.iloc[train_idx])
        fitness = model.score(X_test.iloc[test_idx], y_test.iloc[test_idx])
        
        return fitness 

    return fitness_function

以及遗传算法的实例化

cross_validate = pygad.GA(gene_space=gene_space,
                      fitness_func=fitness_function_factory(),
                      num_generations=100,
                      num_parents_mating=2,
                      sol_per_pop=8,
                      num_genes=len(gene_space),
                      parent_selection_type='sss',
                      keep_parents=2,
                      crossover_type="single_point",
                      mutation_type="random",
                      mutation_percent_genes=25)

cross_validate.best_solution()
>>>
ValueError: n_estimators must be an integer, got <class 'numpy.float64'>.

有关解决此错误的任何建议？

编辑：我已经尝试了以下成功的结果：

model = RandomForestClassifier(n_estimators=gene_space[0][0])
model.fit(X,y)

所以问题不在于 numpy->sklearn，而在于 PyGAD。

Answer 1

我在这里发现了两个问题：

1. pygad.GA 不会从“gene_space”的相关基因值中导出数值类型，而是简单地将所有数值转换为“float”。
   为了解决这个问题，必须使用“gene_type”参数来指定受尊重的基因值类型。 https://pygad.readthedocs.io/en/latest/README_pygad_ReadTheDocs.html#more-about-the-gene-type-parameter

2. numpy.linspace() 不能像记录的那样用于自定义每个基因的值空间。此功能导致在填充时为所有基因产生零。
   所以，最好改用这个符号 {"low": 50, "high": 200, "step": 25} 或明确地将 numpy.ndarray 转换为列表，如 numpy.linspace().tolist()。

   基因空间

   gene_space = [
       # n_estimators
       {"low": 50, "high": 200, "step": 25},
       # min_samples_split,
       {"low": 2, "high": 10, "step": 5},
       # min_samples_leaf,
       {"low": 1, "high": 10, "step": 5},
       # min_impurity_decrease
       np.linspace(0, 1, 10).tolist()
   ]
   

   基因类型

   cross_validate = pygad.GA(
       gene_space=gene_space,
       fitness_func=fitness_function_factory(),
       num_generations=100,
       num_parents_mating=2,
       sol_per_pop=8,
       num_genes=len(gene_space),
       parent_selection_type='sss',
       keep_parents=2,
       crossover_type="single_point",
       mutation_type="random",
       mutation_percent_genes=25,
       gene_type=[int, int, int, float]
   )
   

   我用这种方式测试

   import numpy as np
   import pandas as pd
   import pygad
   from numpy.random import default_rng
   from sklearn.ensemble import RandomForestClassifier
   from sklearn.model_selection import train_test_split
   from sklearn.utils.random import sample_without_replacement
   
   gene_space = [
       # n_estimators
       {"low": 50, "high": 200, "step": 25},
       # min_samples_split,
       {"low": 2, "high": 10, "step": 5},
       # min_samples_leaf,
       {"low": 1, "high": 10, "step": 5},
       # min_impurity_decrease
       np.linspace(0, 1, 10).tolist()
   ]
   
   rng = default_rng()
   n = 1000
   data = pd.DataFrame({"x_1": rng.standard_normal(n), "x_2": rng.standard_normal(n), "y": rng.integers(0, 2, n)})
   
   
   def fitness_function_factory(data=data, y_name='y', sample_size=100):
   
       def fitness_function(solution, solution_idx):
   
           model = RandomForestClassifier(
               n_estimators=solution[0],
               min_samples_split=solution[1],
               min_samples_leaf=solution[2],
               min_impurity_decrease=solution[3]
           )
   
           X = data.drop(columns=[y_name])
           y = data[y_name]
           X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                               test_size=0.5)
   
           train_idx = sample_without_replacement(n_population=len(X_train),
                                                  n_samples=sample_size)
   
           test_idx = sample_without_replacement(n_population=len(X_test),
                                                 n_samples=sample_size)
   
           model.fit(X_train.iloc[train_idx], y_train.iloc[train_idx])
           fitness = model.score(X_test.iloc[test_idx], y_test.iloc[test_idx])
   
           return fitness
   
       return fitness_function
   
   
   cross_validate = pygad.GA(
       gene_space=gene_space,
       fitness_func=fitness_function_factory(),
       num_generations=100,
       num_parents_mating=2,
       sol_per_pop=8,
       num_genes=len(gene_space),
       parent_selection_type='sss',
       keep_parents=2,
       crossover_type="single_point",
       mutation_type="random",
       mutation_percent_genes=25,
       gene_type=[int, int, int, float]
   )
   
   print(cross_validate.best_solution())
   
   (array([75, 2, 1, 0.5555555555555556], dtype=object), 0.5, 3)