Pandas 基于长格式的条件总和追加行

Question

所以，我有一些示例数据：

import pandas as pd
objs = [
    {'location':'US', 'fruit':'apple', 'time':'night', 'value': 1},
    {'location':'US', 'fruit':'orange', 'time':'night', 'value': 3},
    {'location':'US', 'fruit':'banana', 'time':'night', 'value': 1},
    {'location':'EU', 'fruit':'apple', 'time':'night', 'value': 4},
    {'location':'EU', 'fruit':'orange', 'time':'night', 'value': 1},
    {'location':'EU', 'fruit':'banana', 'time':'night', 'value': 2},
    {'location':'US', 'fruit':'apple', 'time':'day', 'value': 5},
    {'location':'US', 'fruit':'orange', 'time':'day', 'value': 2},
    {'location':'US', 'fruit':'banana', 'time':'day', 'value': 3},
    {'location':'EU', 'fruit':'apple', 'time':'day', 'value': 6},
    {'location':'EU', 'fruit':'orange', 'time':'day', 'value': 2},
    {'location':'EU', 'fruit':'banana', 'time':'day', 'value': 1},
]
df = pd.DataFrame.from_records(objs)

它给出了一个长格式的数据框，例如：

   location   fruit   time  value
0        US   apple  night      1
1        US  orange  night      3
2        US  banana  night      1
3        EU   apple  night      4
4        EU  orange  night      1
5        EU  banana  night      2
6        US   apple    day      5
7        US  orange    day      2
8        US  banana    day      3
9        EU   apple    day      6
10       EU  orange    day      2
11       EU  banana    day      1

---

对于location 和time 的每一对/分组，我想根据fruit 列中的值有条件地对value 列求和。 具体来说：

我想对每个分组的 apple 和 orange 求和，但不想对 banana 行求和。

生成以下数据框，并指定新行

   location      fruit   time  value
0        US      apple  night      1
1        US     orange  night      3
2        US     banana  night      1
3        US  NO_BANANA  night      4  <--
4        EU      apple  night      4
5        EU     orange  night      1
6        EU     banana  night      2
7        EU  NO_BANANA  night      5  <--
8        US      apple    day      5
9        US     orange    day      2
10       US     banana    day      3
11       US  NO_BANANA    day      7  <--
12       EU      apple    day      6
13       EU     orange    day      2
14       EU     banana    day      1
15       EU  NO_BANANA    day      8  <--

---

非常感谢任何帮助

Answer 1

如果每个组的条件相同，只需先过滤然后分组：

subdf = df[df['fruit']!='banana'].groupby(['location', 'time']).sum().reset_index()
subdf['fruit'] = 'NO_BANANA'
df = pd.concat([df, subdf]).sort_values(['time', 'location'], ascending = False).reset_index(drop=True)

location    fruit   time    value
0   US  apple   night   1
1   US  orange  night   3
2   US  banana  night   1
3   US  NO_BANANA   night   4
4   EU  apple   night   4
5   EU  orange  night   1
6   EU  banana  night   2
7   EU  NO_BANANA   night   5
8   US  apple   day 5
9   US  orange  day 2
10  US  banana  day 3
11  US  NO_BANANA   day 7
12  EU  apple   day 6
13  EU  orange  day 2
14  EU  banana  day 1
15  EU  NO_BANANA   day 8

Answer 2

• 创建您想要的组/聚合，不包括 banana
• concat()有原始数据
• 根据需要排序/索引

df = pd.read_csv(io.StringIO("""   location   fruit   time  value
0        US   apple  night      1
1        US  orange  night      3
2        US  banana  night      1
3        EU   apple  night      4
4        EU  orange  night      1
5        EU  banana  night      2
6        US   apple    day      5
7        US  orange    day      2
8        US  banana    day      3
9        EU   apple    day      6
10       EU  orange    day      2
11       EU  banana    day      1"""), sep="\s+")

df = (pd.concat([df, df.loc[df.fruit.ne("banana")].groupby(["location","time"],as_index=False).agg({"value":"sum"}).assign(fruit="NO_BANANA")])
 .sort_values(["time","location","fruit"], ascending=[0,0,1])
 .reset_index(drop=True)
)

输出

	location	fruit	time	value
0	US	NO_BANANA	night	4
1	US	apple	night	1
2	US	banana	night	1
3	US	orange	night	3
4	EU	NO_BANANA	night	5
5	EU	apple	night	4
6	EU	banana	night	2
7	EU	orange	night	1
8	US	NO_BANANA	day	7
9	US	apple	day	5
10	US	banana	day	3
11	US	orange	day	2
12	EU	NO_BANANA	day	8
13	EU	apple	day	6
14	EU	banana	day	1
15	EU	orange	day	2

Answer 3

另一种选择是创建一个数据透视表

df = df.pivot_table(index=['location', 'time', 'fruit'],
                    values=['value']).unstack()

               value              
fruit          apple banana orange
location time                     
EU       day       6      1      2
         night     4      2      1
US       day       5      3      2
         night     1      1      3

根据黑名单按列过滤水果：

# Mask Fruits To Keep/Exclude
fruits_blacklist = ['banana']
cm = np.where(df.columns.get_level_values(1).isin(fruits_blacklist), False, True)

然后对过滤后的列进行逐行求和：

df[('value', 'NO_BANANA')] = df.loc[:, cm].apply(np.sum, axis=1)

---

import pandas as pd
import numpy as np


# blacklist
fruits_blacklist = ['banana']

# Create Pivot Table
df = df.pivot_table(index=['location', 'time', 'fruit'],
                    values=['value']).unstack()

# Mask Fruits To Keep/Exclude
cm = np.where(df.columns.get_level_values(1).isin(fruits_blacklist), False, True)
# Sum Row-wise
df[('value', 'NO_BANANA')] = df.loc[:, cm].apply(np.sum, axis=1)
# Stack and Reset Index
df = df.stack('fruit') \
    .sort_values(["time", "location"],
                 ascending=False) \
    .reset_index()

# For Display
print(df.to_string())

输出：

   location   time      fruit  value
0        US  night      apple      1
1        US  night     banana      1
2        US  night     orange      3
3        US  night  NO_BANANA      4
4        EU  night      apple      4
5        EU  night     banana      2
6        EU  night     orange      1
7        EU  night  NO_BANANA      5
8        US    day      apple      5
9        US    day     banana      3
10       US    day     orange      2
11       US    day  NO_BANANA      7
12       EU    day      apple      6
13       EU    day     banana      1
14       EU    day     orange      2
15       EU    day  NO_BANANA      8