当“代码”列超过四个时，在熊猫数据框中创建一个新行答案

【问题标题】：Creating a new row in pandas dataframe when there are more than four 'Code' columns当“代码”列超过四个时，在熊猫数据框中创建一个新行
【发布时间】：2020-11-20 10:42:34
【问题描述】：

我有一个数据框，每一行都有一个唯一的“组”和“成员”配对，以及与后续行中该配对相关的所有“代码”。下面是我编写的一段代码，用于创建其结构的一个小示例：

import pandas as pd
member = [1,1,2,2,2,3,3]
group = ['A', 'B', 'A', 'B', 'C', 'A', 'B']
dx1 = [11, 1232, 23, 346, 63, 346, 634]
dx2 = ["", 24, "", 2436, 346, 24, 97]
dx3 = ["", 2313, "", "", 987, 28, ""]
dx4 = ["", 234, "", "", "", "", ""]
dx5 = ["", 85, "", "", "", "", ""]
dx6 = ["", 4, "", "", "", "", ""]
dx7 = ["", 3, "", "", "", "", ""]
dx8 = ["", 9384, "", "", "", "", ""]
dx9 = ["", 38, "", "", "", "", ""]
dx0= ["", 3833, "", "", "", "", ""]

testdf = pd.DataFrame(data=[member, group, dx1, dx2, dx3, dx4, dx5, dx6, dx7, dx8, dx9, dx0])


cols=['group', 'member', 'code1', 'code2', 'code3', 'code4','code5','code6','code7','code8','code9','code0']

dft = testdf.T
dft.columns = cols
dft

该代码将生成此数据框：

Initial DF

实际的数据框有数千行。当一行/组成员配对中列出的“代码”超过四个时，我想为前四个之后的每四个代码创建一个新行。仅转换前两行的结果数据框应如下所示：

Final Df (only the first two rows of Initial DF)

我在想某种条件循环可以用来创建一个新的 df 并相应地追加新的行，但由于某种原因，我的大脑被困在如何做到这一点上。我很抱歉没有在这篇文章中将数据帧粘贴为文本，我对在 Stack 上发布相对较新（因此，如果有人对将 Pandas DF 作为文本粘贴到 Stack markdown 中提出建议，那也太棒了！）非常感谢任何帮助.

** 注意：在此示例中，第二行是唯一会被正确转换更改的行，因为它是唯一具有 > 4 个代码的行

【问题讨论】：

这里是关于在 Stack Overflow 上显示数据帧的信息：stackoverflow.com/help/minimal-reproducible-example
非常感谢@jsmart

标签： python pandas loops dataframe conditional-statements

【解决方案1】：

slicing multiple ranges 在链接中使用技巧

将数字列中的空字符串替换为np.nap
使用iloc 切出列和所需行
在对列进行适当重命名后，有效地为 pd.concat() 留下了三件事
清理排序顺序和索引

import pandas as pd
member = [1,1,2,2,2,3,3]
group = ['A', 'B', 'A', 'B', 'C', 'A', 'B']
dx1 = [11, 1232, 23, 346, 63, 346, 634]
dx2 = ["", 24, "", 2436, 346, 24, 97]
dx3 = ["", 2313, "", "", 987, 28, ""]
dx4 = ["", 234, "", "", "", "", ""]
dx5 = ["", 85, "", "", "", "", ""]
dx6 = ["", 4, "", "", "", "", ""]
dx7 = ["", 3, "", "", "", "", ""]
dx8 = ["", 9384, "", "", "", "", ""]
dx9 = ["", 38, "", "", "", "", ""]
dx0= ["", 3833, "", "", "", "", ""]

testdf = pd.DataFrame(data=[member, group, dx1, dx2, dx3, dx4, dx5, dx6, dx7, dx8, dx9, dx0])
cols=['group', 'member', 'code1', 'code2', 'code3', 'code4','code5','code6','code7','code8','code9','code0']
dft = testdf.T
dft.columns = cols

dft = dft.replace({"":np.nan}) # just better for masks
mask5 = dft[~dft["code5"].isna()].index  # rows with a value in code 5
mask9 = dft[~dft["code9"].isna()].index  # rows with a value in code 9
# just concat them all together
dft = pd.concat([
    dft.iloc[:,0:6],
    # rename columns so they concat correctly
    dft.iloc[mask5,np.r_[0:2, 6:10]].rename({f"code{i+5}":f"code{i+1}" for i in range(4)}, axis=1),
    dft.iloc[mask9,np.r_[0:2, 10:12]].rename({"code9":"code1","code0":"code2"}, axis=1),
]).sort_values(["group","member"]).reset_index(drop=True)

print(dft.to_string(index=False))

输出

group member   code1   code2   code3   code4
     1      A    11.0     NaN     NaN     NaN
     1      B  1232.0    24.0  2313.0   234.0
     1      B    85.0     4.0     3.0  9384.0
     1      B    38.0  3833.0     NaN     NaN
     2      A    23.0     NaN     NaN     NaN
     2      B   346.0  2436.0     NaN     NaN
     2      C    63.0   346.0   987.0     NaN
     3      A   346.0    24.0    28.0     NaN
     3      B   634.0    97.0     NaN     NaN

【讨论】：

这对我的真实数据集来说非常棒而且超级健壮，它有 44000 行，我必须分成 9 组而不是 4 组。谢谢！
@EddieMcGolrick 很高兴听到它运行良好。为了性能和代码稳定性，始终在数据集范式中工作，而不是循环

【解决方案2】：

import pandas as pd

member = [1,1,2,2,2,3,3]
group = ['A', 'B', 'A', 'B', 'C', 'A', 'B']
dx1 = [11, 1232, 23, 346, 63, 346, 634]
dx2 = ["", 24, "", 2436, 346, 24, 97]
dx3 = ["", 2313, "", "", 987, 28, ""]
dx4 = ["", 234, "", "", "", "", ""]
dx5 = ["", 85, "", "", "", "", ""]
dx6 = ["", 4, "", "", "", "", ""]
dx7 = ["", 3, "", "", "", "", ""]
dx8 = ["", 9384, "", "", "", "", ""]
dx9 = ["", 38, "", "", "", "", ""]
dx0= ["", 3833, "", "", "", "", ""]

testdf = pd.DataFrame(data=[member, group, dx1, dx2, dx3, dx4, dx5, dx6, dx7, dx8, dx9, dx0])

cols=['group', 'member', 'code1', 'code2', 'code3', 'code4','code5','code6','code7','code8','code9','code0']

dft = testdf.T
dft.columns = cols

dfnewrows = pd.DataFrame(columns=cols[:6]) # for added rows

for idx,row in dft.iterrows():
   if (row['code5']): # wrap 2nd four codes if needed
        dfnewrows.loc[len(dfnewrows)+1] = [row['group'],row['member'],row['code5'],row['code6'],row['code7'],row['code8']]
   if (row['code9']): # wrap last codes if needed
        dfnewrows.loc[len(dfnewrows)+1] = [row['group'],row['member'],row['code9'],row['code0'],"", ""]

dfnew = dft[['group', 'member', 'code1', 'code2', 'code3', 'code4']] # remove code5, code6,.....

dfnew = dfnew.append(dfnewrows) # add wrapped rows

dfnew = dfnew.sort_values(by=['group', 'member'])  # sort table

print(dfnew.to_string(index=False))

输出

group member code1 code2 code3 code4
    1      A    11
    1      B  1232    24  2313   234
    1      B    85     4     3  9384
    1      B    38  3833
    2      A    23
    2      B   346  2436
    2      C    63   346   987
    3      A   346    24    28
    3      B   634    97

【讨论】：