【问题标题】:pandas Optimizing many loops into onepandas 将多个循环优化为一个
【发布时间】:2021-11-27 16:27:57
【问题描述】:

我有多个具有相同列的 dfs。这是所有dfs的列表

dfs = [df_14, df_15, df_16, df_17]

例如,每个数据帧看起来像这样,df_14:

id Days
001 0
004 56
013 95
015 33

接下来,df_15:

Id Days
001 0
023 18
459 19
811 35

df_16:

Id Days
111 93
114 56
232 0

df_17:

Id Days
532 120
113 31
065 58
015 2

我的代码:



rows = [['532', 120],['113', 31], ['065', 58],['025', 2]]
for row in rows:
    df_14.loc[len(df_14)] = row
# and so on

任务是附加到每个月的列表 - 有 30-60 天的列表和另一个单独的列表,其中包含 60-100 天的客户 ID。



#The result should be like this:
14_1:  ['004', '015']
14_2:  ['013']
15_1:  ['811']

我尝试在上面使用 f'strings。比如:

abrreviations = ['14', '15','16', '17']
c = ['_1', '_2']
#Have wrote initializing loops like 
m_list=[]
for a in abrreviations:
    for cp in c:
        m_list.append(a+cp)

这个想法是在带有 f'string 或格式的循环中使用缩写。但是不知道怎么做?或者您能提供其他想法吗?

【问题讨论】:

    标签: python pandas list dataframe loops


    【解决方案1】:
    #let first create a list containing all the dataframe's
    
    all_df=[df_jan, df_feb, df_mar, df_apr, df_may, df_jun, df_jul, df_aug, df_sep, df_oct, df_nov, df_dec]
    
    #create 2 lists for storing the id values of 30-60 range and 90-120 range
    
    list_30,list_90=[],[]
    
    #1 nested for loop for handling all data frames
    
    for cur_df in all_df:
        for id,days in zip(cur_df['Id'],cur_df['Days']):
            if(30<=days<=60):
                list_30.append(id)
            elif(90<=days<=120):
                list_90.append(id)
    
    #Now list_30 and list_90 contains the corresponding id values in that range
    

    希望答案有帮助:)

    【讨论】:

    • 谢谢。但我需要将每个月数据框的 id 划分为相应的列表。比如 jan_30 = ['001', '015'],jan_90 = ['013'], feb_30 = ['811']
    【解决方案2】:

    由于您没有提供数据,所以我做了一个基本示例,它对我有用,所以这里是您描述的单个 for 循环:

    import numpy as np
    import pandas as pd
    dfs = [df_jan, df_feb, df_mar, df_apr, df_may, df_jun, df_jul, df_aug, df_sep, df_oct, df_nov, df_dec]
    df30 = []
    df90 = []
    dfsChained30 = []
    dfsChained90 = []
    for rowsForMonths, xForMonths in enumerate(dfs):
      # If January [don't consider chain];
      if rowsForMonths == 0:
        for dayN in range(dfs[rowsForMonths]):
          if dfs[rowsForMonths][dayN] in range(30, 61):
            df30.append(dfs[rowsForMonths][dayN])
          elif dfs[rowsForMonths][dayN] in range(90, 121):
            df90.append(dfs[rowsForMonths][dayN])
          else:
            pass
        dfsChained30.append(df30)
        dfsChained90.append(df90)
      # If not January [consider chain];
      else:
        for dayN in range(dfs[rowsForMonths]):
          if dfs[rowsForMonths][dayN] in range(30, 61) and dfs[rowsForMonths][dayN] not in set(dfsChained30):
            df30.append(dfs[rowsForMonths][dayN])
          elif dfs[rowsForMonths][dayN] in range(90, 121) and dfs[rowsForMonths][dayN] not in set(dfsChained90):
            df90.append(dfs[rowsForMonths][dayN])
          else:
            pass
        dfsChained30.append(df30)
        dfsChained90.append(df90)
    

    【讨论】:

    • 谢谢。我实际上添加了我的整个代码。这我看到返回整体结果,不除以月份和条件列表
    • 对不起,我看错了;如何将 df30 和 df90 变量定义为 np.zeros([1, numberOfMonthsInAYear], dtype = str) 并通过 df30[0, rowsForMonths] = dfs[rowsForMonths][dayN] 填充这些变量 第二次编辑:保持 df30 = [ ] 作为临时变量并用临时变量填充新的 df30 变量,然后每次迭代(即每个月)重置 temp 变量
    【解决方案3】:

    这可以帮助你

    import pandas as pd
    
    data = {'df_jan' : [['001', 0],['004', 56], ['013', 95],['015', 33]],
            'df_feb' : [['001', 0],['023', 18], ['459', 19],['811', 35]],
            'df_mar' : [['111', 93],['114', 56], ['232', 0]],
            'df_apr' : [['532', 120],['113', 31], ['065', 58],['025', 2]]}
    
    dfs = {}
    for df in data:
        dfs[df] = pd.DataFrame(data[df], columns=['id', 'days'])
    
    months = {}
    
    for df in dfs:
        months[df.replace('df_', '') + '_30'] = dfs[df][(dfs[df].days >= 30) & (dfs[df].days <= 60)].id.to_list()
        months[df.replace('df_', '') + '_90'] = dfs[df][(dfs[df].days >= 90) & (dfs[df].days <= 120)].id.to_list()
        
    months
    
    {'jan_30': ['004', '015'],
     'jan_90': ['013'],
     'feb_30': ['811'],
     'feb_90': [],
     'mar_30': ['114'],
     'mar_90': ['111'],
     'apr_30': ['113', '065'],
     'apr_90': ['532']}
    
    回应您的评论:

    我在字典中创建了 df 来简化测试数据的创建。

    您的代码可以以自己的方式创建 df ...

    df_jan = ...
    df_feb = ...
    df_mar = ...
    df_apr = ...
    

    要处理它们,您需要创建字典 ...

    dfs = {
        'df_jan' : df_jan,
        'df_feb' : df_feb,
        'df_mar' : df_mar,
        'df_apr' : df_apr
    }
    

    运行循环

    您可以将结果分配给您的变量

    并删除字典

    jan_30 = months['jan_30']
    jan_90 = months['jan_90']
    feb_30 = months['feb_30']
    feb_90 = months['feb_90']
    mar_30 = months['mar_30']
    mar_90 = months['mar_90']
    apr_30 = months['apr_30']
    apr_90 = months['apr_90']
    
    del dfs, months
    

    【讨论】:

    • 谢谢,效果很好。你能帮我吗,假设我的 dfs 有很多行(比如 10000 行),这种方法能很好地工作吗?我应该建立字典吗?
    • 您不需要在字典中构建 df。请参阅编辑后的答案。
    猜你喜欢
    • 2018-08-31
    • 2016-05-24
    • 2019-01-01
    • 2018-07-12
    • 2011-04-28
    • 1970-01-01
    • 1970-01-01
    • 2019-02-04
    • 1970-01-01
    相关资源
    最近更新 更多