根据条件为自定义列分配值

Question

所以，这是我正在使用的数据集的一个示例 (link here)

brand  model  column1  column2  column3  category  sub category  value
Dell   a      aa                bb       cat1      sc1           aaa
Dell   a      aa                bb       cat1      sc2           bbb
Dell   a      aa                cc       cat2      sc1           ccc
Dell   b      aa       bb                cat1      sc1           ddd
Dell   b      aa       bb                cat2      sc1           eee
Dell   b      aa       bb       cc       cat2                    fff
Asus   c      aa       cc       bb       cat1      sc1           ggg
Asus   c      aa       cc       bb       cat1                    hhh
Asus   c      aa                         cat1      sc2           iii
Asus   d      aa       cc       bb       cat1      sc1           jjj
Asus   d      aa       bb       bb       cat1      sc2           kkk
Asus   d      aa       bb       bb       cat1      sc3           lll

我首先需要做的是根据品牌、型号、column1-3 分开独特的模型，我这样做：

import pandas as pd

df = pd.read_csv("abhorrent.csv")
noDupes = df[["brand", "model name", "column1", "column2", "column3"]].drop_duplicates().copy()

返回一个像这样的表：

brand    model  column1  column2  column3
Dell     a      aa                bb
Dell     a      aa                cc
Dell     b      aa       bb
Dell     b      aa       bb       cc
Asus     c      aa       cc       bb
Asus     c      aa 
Asus     d      aa       cc       bb
Asus     d      aa       bb       bb

但是，我需要根据类别、子类别和值创建列，并分配它们的值。

列名是类别和子类别的连接，我需要在其中放入这些对的值：

cat1_sc1
cat1_sc2
cat1_sc3
cat1_blank
cat2_sc1
cat2_blank

这些列不需要自动生成，我可以硬编码它们。

问题是我不知道如何根据非唯一数据框填充这些列中的值。

我正在寻找的最终结果是这样的：

brand    model  column1  column2  column3  cat1_sc1  cat1_sc2  cat1_sc3  cat1_blank  cat2_sc1  cat2_blank
Dell     a      aa                bb       aaa       bbb               
Dell     a      aa                cc                                                 ccc   
Dell     b      aa       bb                ddd                                       eee   
Dell     b      aa       bb       cc                                                           fff
Asus     c      aa       cc       bb       ggg                           hhh       
Asus     c      aa                                   iii               
Asus     d      aa       cc       bb       jjj                    
Asus     d      aa       bb       bb                 kkk       lll     

我能够在我的解决方案最初开发的 PostrgreSQL 中执行此操作，对每个预定义列使用一个 UPDATE。大致如下：

#fill the cat1_sc1 column
UPDATE transposed_table
SET cat1_sc1 = subquery.value
FROM
    (SELECT ... FROM ... WHERE category = 'cat1' AND sub_category = 'sc1') subquery
WHERE brand = subquery.brand AND model = subquery.model etc

编辑：我的实际 CSV 有接近 50 万行

Answer 1

您可以执行以下操作：

noDupes['cat1_sc1'] = df[(df["category"] == "cat1") & (df["sub category"] == "sc1")]["value"]

您必须对所有类别和子类别执行此操作，但我想您明白了。

编辑：

完整的代码让它全部运行：

import pandas as pd

df = pd.read_csv("abhorrent.csv")

cats = df["category"].drop_duplicates().tolist()
sub_cats = df["sub category"].drop_duplicates().tolist()
cat_sc_s = []
for cat in cats:
    for sc in sub_cats:
        name = str(cat) + '_' + str(sc)
        cat_sc_s.append(name)
        df[name] = df[(df["category"] == cat) & (df["sub category"] == sc)]["value"]

noDupes = df[["brand", "model", "column1", "column2", "column3"] + cat_sc_s].drop_duplicates().copy()
print(noDupes)

编辑 2：

这对****来说有点痛苦：D但在某些时候它变成了个人的东西：D

import pandas as pd

df = pd.read_csv("abhorrent.csv")

df = df.fillna('')
cats = df["category"].drop_duplicates().tolist()
sub_cats = df["sub category"].drop_duplicates().tolist()
cat_sc_s = []
for cat in cats:
    for sc in sub_cats:
        if sc == '':
            sc = 'blanc'
        name = str(cat) + '_' + str(sc)
        cat_sc_s.append(name)
        df[name] = df[(df["category"] == cat) & (df["sub category"] == sc)]["value"]


df = df.fillna('')
df = df.groupby(["brand", "model", "column1", "column2", "column3"], as_index=False).agg(' '.join)
df = df.drop(['category', 'sub category', 'value'], axis = 1)
print(df)

测试一下并告诉我

然而，它改变了顺序。
结果：

  brand model column1 column2 column3 cat1_sc1 cat1_sc2 cat1_blanc cat1_sc3 cat2_sc1 cat2_sc2 cat2_blanc cat2_sc3
0  Asus     c      aa                               iii                                                          
1  Asus     c      aa      cc      bb     ggg                                                                    
2  Asus     d      aa      bb      bb              kkk                  lll                                      
3  Asus     d      aa      cc      bb      jjj                                                                   
4  Dell     a      aa              bb     aaa       bbb                                                          
5  Dell     a      aa              cc                                            ccc                             
6  Dell     b      aa      bb             ddd                                    eee                             
7  Dell     b      aa      bb      cc                                                                            

Answer 2

感谢我的同事...

import pandas as pd
import numpy as np

GROUP_BY = ["brand", "model", "column1", "column2", "column3"]
CATEGORY = "category"
SUB_CATEGORY = "sub category"
VALUE = "value"
GROUPING = "grouping"

def combine_model(group):
    def combine_value(value):
        return value.str.cat(sep=" || ") #in case of multiple values for one category / sub category combination
    value = group.groupby(GROUPING)[VALUE].apply(combine_value)
    group.loc[:, value.index.tolist()] = value.values
    return group

data = pd.read_csv("abhorrent.csv")

for col in GROUP_BY + [SUB_CATEGORY, VALUE]:
    data[col].fillna("N/A", inplace=True)
data[GROUPING] = data[CATEGORY] + "_" + data[SUB_CATEGORY]
columns = data[GROUPING].drop_duplicates().tolist()
for col in columns:
    data[col] = np.nan
data = data.groupby(GROUP_BY).apply(combine_model)
data.drop_duplicates(subset=GROUP_BY, inplace=True)

剩下的只是删除不必要的列...

感谢所有提出解决方案或发表评论的人！