使用 pandas groupby 创建子模型并使用测试数据定位每个模型

Question

我有一个 pandas 数据框，其中列中的值用作创建子模型的分组依据。

import pandas as pd
from sklearn.linear_model import Ridge

data = pd.DataFrame({"Name": ["A", "A", "A", "B", "B", "B"], "Score": [90, 80, 90, 92, 87, 80], "Age": [10, 12, 14, 9, 11, 12], "Training": [0, 1, 2, 0, 1, 2]})

"Name" 被用作为每个人创建子模型的基础。我想使用变量"Age" 和"Training" 来预测一个人"Name" 的"Score"（即在这种情况下"A" 和"B"）。也就是说，如果我有"A" 并且知道"A" 的"Age" 和"Training"，我会喜欢使用"A"、"Age"、"Training" 来预测"Score"。但是，"A" 应该用于访问"A" 所属的模型而不是其他模型。

grouped_df = data.groupby(['Name'])
for key, item in grouped_df:
    Score = grouped_df['Score']
    Y = grouped_df['Age', 'Training']
    Score_item = Score.get_group(key)
    Y_item = Y.get_group(key)
    model = Ridge(alpha = 1.2)
    modelfit = model.fit(Y_item, Score_item)
    modelpred = model.predict(Y_item)
    modelscore = model.score(Y_item, Score_item)
    print modelscore

到目前为止，我已经为子组A 和B 构建了简单的 Ridge 模型。

我的问题是，测试数据如下：

test_data = [u"A, 13, 0", u"B, 12, 1", u"A 10, 0"] ##each element, respectively, represents `Name`, `Age` and `Training`

如何将数据提供给预测模型？ 我有

line = test_data
Name = [line[i].split()[0] for i in range(len(line))]
Age = [line[i].split()[1] for i in range(len(line))]
Training = [line[i].split()[2] for i in range(len(line))]
Y = pd.DataFrame({"Name": Name, "Age": Age, "Training": Training})

这给了我测试数据的熊猫数据框。但是，我不确定如何进一步将测试数据提供给模型。我非常感谢您的帮助。谢谢！！

更新

我采用了 Parfait 的代码后，现在的代码看起来更好了。但是，在这里我没有创建 testdata 的另一个 pandas 数据框（因为我不确定如何处理其中的行）。相反，我通过拆分字符串来输入测试值。我收到如下所示的错误。我在这里搜索并找到了一个相关的帖子Preprocessing in scikit learn - single sample - Depreciation warning。但是，我试图重塑测试数据，但它在列表形式上，因此它没有重塑的属性。我想我误解了。如果您能告诉我如何解决此错误，我将不胜感激。谢谢。

import pandas as pd
from sklearn.linear_model import Ridge
import numpy as np

data = pd.DataFrame({"Name": ["A", "A", "A", "B", "B", "B"], "Score": [90, 80, 90, 92, 87, 80], "Age": [10, 12, 14, 9, 11, 12], "Training": [0, 1, 2, 0,$


modeldict = {}                                           # INITIALIZE DICT
grouped_df = data.groupby(['Name'])

for key, item in grouped_df:
    Score = grouped_df['Score']
    Y = grouped_df['Age', 'Training']
    Score_item = Score.get_group(key)
    Y_item = Y.get_group(key)
    model = Ridge(alpha = 1.2)
    modelfit = model.fit(Y_item, Score_item)
    modelpred = model.predict(Y_item)
    modelscore = model.score(Y_item, Score_item)
    modeldict[key] = modelfit                            # SAVE EACH FITTED MODEL TO DICT


line = [u"A, 13, 0", u"B, 12, 1", u"A, 10, 0"]
Name = [line[i].split(",")[0] for i in range(len(line))]
Age = [line[i].split(",")[1] for i in range(len(line))]
Training = [line[i].split(",")[2] for i in range(len(line))]


for i in range(len(line)):
Name = line[i].split(",")[0]
Age = line[i].split(",")[1]
Training = line[i].split(",")[2]
model = modeldict[Name]
ip = [float(Age), float(Training)]
score = model.predict(ip)

print score

错误

/opt/conda/lib/python2.7/site-packages/sklearn/utils/validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample. DeprecationWarning)
86.6666666667
/opt/conda/lib/python2.7/site-packages/sklearn/utils/validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.DeprecationWarning)
83.5320600273
/opt/conda/lib/python2.7/site-packages/sklearn/utils/validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.DeprecationWarning)
86.6666666667
/opt/conda/lib/python2.7/site-packages/sklearn/utils/validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.DeprecationWarning)
[ 86.66666667]
/opt/conda/lib/python2.7/site-packages/sklearn/utils/validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.DeprecationWarning)
[ 83.53206003]
/opt/conda/lib/python2.7/site-packages/sklearn/utils/validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample. DeprecationWarning)
[ 86.66666667]

Answer 1

考虑将子模型保存在以 Name 为键的字典中，然后运行 ​​pandas.DataFrame.apply() 以在每一行上运行操作，将行的 Name 与相应的模型对齐。

注意：以下是未经测试的代码，但希望能提供一个总体思路，您可以据此进行调整。主要问题可能是在apply() 中使用的定义函数runModel 中的model.predict() 输入和输出。 model.predict() 中使用了 Age 和 Training 值的 numpy 矩阵，它希望返回一个等于样本大小（即每一行）的 numpy。见Ridge model:

modeldict = {}                                           # INITIALIZE DICT
grouped_df = data.groupby(['Name'])

for key, item in grouped_df:
    Score = grouped_df['Score']
    Y = grouped_df['Age', 'Training']
    Score_item = Score.get_group(key)
    Y_item = Y.get_group(key)
    model = Ridge(alpha = 1.2)
    modelfit = model.fit(Y_item, Score_item)
    modelpred = model.predict(Y_item)
    modelscore = model.score(Y_item, Score_item)
    print modelscore

    modeldict[key] = modelfit                            # SAVE EACH FITTED MODEL TO DICT

line = [u"A, 13, 0", u"B, 12, 1", u"A 10, 0"] 
Name = [line[i].split()[0] for i in range(len(line))]
Age = [line[i].split()[1] for i in range(len(line))]
Training = [line[i].split()[2] for i in range(len(line))]

testdata = pd.DataFrame({"Name": Name, "Age": Age, "Training": Training})

def runModel(row):
    # LOCATE MODEL BY NAME KEY 
    model = modeldict[row['Name']]
    # PREDICT VALUES
    score = model.predict(np.matrix([row['Age'], row['Training']])
    # RETURN SCALAR FROM score ARRAY 
    return(score[0])    

testdata['predictedScore'] = testdata.apply(runModel, axis=1)