将数据框列字符串值转换为虚拟变量列

Question

我有以下数据框（不包括其余列）：

| customer_id | department                    |
| ----------- | ----------------------------- |
| 11          | ['nail', 'men_skincare']      |
| 23          | ['nail', 'fragrance']         |
| 25          | []                            |
| 45          | ['skincare', 'men_fragrance'] |

我正在对数据进行预处理以适应模型。我想将部门变量转换为每个独特部门类别的虚拟变量（无论可能有多少独特的部门，不仅限于此处的内容）。

想要得到这个结果：

| customer_id | department                    | nail | men_skincare | fragrance | skincare | men_fragrance |
| ----------- | ----------                    | ---- | ------------ | --------- | -------- | ------------- |
| 11          | ['nail', 'men_skincare']      | 1    | 1            | 0         | 0        | 0             |
| 23          | ['nail', 'fragrance']         | 1    | 0            | 1         | 0        | 0             |
| 25          | []                            | 0    | 0            | 0         | 0        | 0             |
| 45          | ['skincare', 'men_fragrance'] | 0    | 0            | 0         | 1        | 1             |

我试过这个link，但是当我拼接它时，它把它当作一个字符串来处理，并且只为字符串中的每个字符创建一个列；我用的是什么：

df['1st'] = df['department'].str[0]
df['2nd'] = df['department'].str[1]
df['3rd'] = df['department'].str[2]
df['4th'] = df['department'].str[3]
df['5th'] = df['department'].str[4]
df['6th'] = df['department'].str[5]
df['7th'] = df['department'].str[6]
df['8th'] = df['department'].str[7]
df['9th'] = df['department'].str[8]
df['10th'] = df['department'].str[9]

然后我尝试使用以下方法拆分字符串并变成一个列表：

df['new_column'] = df['department'].apply(lambda x: x.split(","))

然后再次尝试，仍然只为每个字符创建列。

有什么建议吗？

编辑：我使用 anky 发送的链接找到了答案，特别是我使用了这个：https://stackoverflow.com/a/29036042

什么对我有用：

df['department'] = df['department'].str.replace("'",'').str.replace("]",'').str.replace("[",'').str.replace(' ','')
df['department'] = df['department'].apply(lambda x: x.split(","))
s = df['department']
df1 = pd.get_dummies(s.apply(pd.Series).stack()).sum(level=0)
df = pd.merge(df, df1, right_index=True, left_index=True, how = 'left')

Answer 1

import pandas as pd

您可以通过explode()、value_counts() 和fillna() 方法做到这一点：

data=df.explode('department').fillna('empty')

现在使用crosstab() 方法：

data=pd.crosstab(data['customer_id'],data['department'])

由于concat() 方法给你一个错误，所以使用merge() 方法和drop() 方法：

data=pd.merge(df.set_index('customer_id'),data,left_index=True,right_index=True).drop(columns=['empty'])

现在，如果您打印 data，您将获得所需的输出：

Answer 2

这是一个 fast binarizer method 使用 sklearn 的 MultiLabelBinarizer 基于 anky 的链接：

from sklearn.preprocessing import MultiLabelBinarizer

df = pd.DataFrame({'customer_id':{0:11,1:23,2:25,3:45}, 'department':{0:["'nail'","'men_skincare'"], 1:["'nail'","'fragrance'"], 2:[''], 3:["'skincare'","'men_fragrance'"]}})
mlb = MultiLabelBinarizer()

df = df.join(pd.DataFrame(
    mlb.fit_transform(df.department),
    columns=[c.strip("'") for c in mlb.classes_],
    index=df.index,
)).drop(columns='')

#   customer_id                     department  fragrance  men_fragrance  men_skincare  nail  skincare
# 0          11       ['nail', 'men_skincare']          0              0             1     1         0
# 1          23          ['nail', 'fragrance']          1              0             0     1         0
# 2          25                             []          0              0             0     0         0
# 3          45  ['skincare', 'men_fragrance']          0              1             0     0         1

注意：这假设您的真实数据的department 列包含实际的python 列表，而不是看起来像列表的字符串。如果它们实际上是字符串（即type(df.department[0]) 输出str），则需要先进行此转换：

df.department = df.department.str.strip('[]').str.split(r'\s*,\s*')

Answer 3

试试：

df.merge(pd.get_dummies(df.set_index('customer_id')
                          .explode('department'), 
                        prefix='', 
                        prefix_sep='').sum(level=0),
        left_on='customer_id', right_index=True)

输出：

   customer_id                 department  fragrance  men_fragrance  men_skincare  nail  skincare
0           11       [nail, men_skincare]          0              0             1     1         0
1           23          [nail, fragrance]          1              0             0     1         0
2           25                         []          0              0             0     0         0
3           45  [skincare, men_fragrance]          0              1             0     0         1