根据列内的条件创建组/类

Question

我需要帮助转换我的数据，以便阅读交易数据。

商业案例

我正在尝试将一些相关事务组合在一起以创建一些事件组或类别。该数据集代表因各种缺勤事件外出的工人。我想根据请假事件类的 365 天内的任何交易创建一类请假。为了绘制趋势图表，我想对类进行编号，以便获得序列/模式。

我的代码允许我查看第一个事件发生的时间，并且它可以识别新类何时开始，但它不会将每个事务存储到一个类中。

要求：

• 根据所有行所属的休假类别标记所有行。
• 为每个唯一离开事件编号。使用此示例索引 0 将是唯一离开事件 2，索引 1 将是唯一离开事件 2，索引 3 将是唯一离开事件 2，并且索引 4 将是唯一离开事件 1，等等。

我在所需输出的列中添加了一个标记为“所需输出”的列。请注意，每人可以有更多的行/事件；而且可以有更多的人。

一些数据

import pandas as pd

data = {'Employee ID': ["100", "100", "100","100","200","200","200","300"],
        'Effective Date': ["2016-01-01","2015-06-05","2014-07-01","2013-01-01","2016-01-01","2015-01-01","2013-01-01","2014-01"],
        'Desired Output': ["Unique Leave Event 2","Unique Leave Event 2","Unique Leave Event 2","Unique Leave Event 1","Unique Leave Event 2","Unique Leave Event 2","Unique Leave Event 1","Unique Leave Event 1"]}
df = pd.DataFrame(data, columns=['Employee ID','Effective Date','Desired Output'])

我尝试过的一些代码

df['Effective Date'] = df['Effective Date'].astype('datetime64[ns]')
df['EmplidShift'] = df['Employee ID'].shift(-1)
df['Effdt-Shift'] = df['Effective Date'].shift(-1)
df['Prior Row in Same Emplid Class'] = "No"
df['Effdt Diff'] = df['Effdt-Shift'] - df['Effective Date']
df['Effdt Diff'] = (pd.to_timedelta(df['Effdt Diff'], unit='d') + pd.to_timedelta(1,unit='s')).astype('timedelta64[D]')
df['Cumul. Count'] = df.groupby('Employee ID').cumcount()


df['Groupby'] = df.groupby('Employee ID')['Cumul. Count'].transform('max')
df['First Row Appears?'] = ""
df['First Row Appears?'][df['Cumul. Count'] == df['Groupby']] = "First Row"
df['Prior Row in Same Emplid Class'][ df['Employee ID'] == df['EmplidShift']]  = "Yes"

df['Prior Row in Same Emplid Class'][ df['Employee ID'] == df['EmplidShift']]  = "Yes"

df['Effdt > 1 Yr?'] = ""                                        
df['Effdt > 1 Yr?'][ ((df['Prior Row in Same Emplid Class'] == "Yes" ) & (df['Effdt Diff'] < -365))  ] = "Yes"

df['Unique Leave Event'] = ""
df['Unique Leave Event'][ (df['Effdt > 1 Yr?'] == "Yes") | (df['First Row Appears?'] == "First Row") ] = "Unique Leave Event" 

df

Answer 1

您可以做到这一点，而无需循环或迭代您的数据框。根据Wes McKinney，您可以将.apply() 与 groupBy 对象一起使用，并定义一个函数以应用于 groupby 对象。如果您将其与 .shift() (like here) 一起使用，您无需使用任何循环即可获得结果。

简洁示例：

# Group by Employee ID
grouped = df.groupby("Employee ID")
# Define function 
def get_unique_events(group):
    # Convert to date and sort by date, like @Khris did
    group["Effective Date"] = pd.to_datetime(group["Effective Date"])
    group = group.sort_values("Effective Date")
    event_series = (group["Effective Date"] - group["Effective Date"].shift(1) > pd.Timedelta('365 days')).apply(lambda x: int(x)).cumsum()+1
    return event_series

event_df = pd.DataFrame(grouped.apply(get_unique_events).rename("Unique Event")).reset_index(level=0)
df = pd.merge(df, event_df[['Unique Event']], left_index=True, right_index=True)
df['Output'] = df['Unique Event'].apply(lambda x: "Unique Leave Event " + str(x))
df['Match'] = df['Desired Output'] == df['Output']

print(df)

输出：

  Employee ID Effective Date        Desired Output  Unique Event  \
3         100     2013-01-01  Unique Leave Event 1             1
2         100     2014-07-01  Unique Leave Event 2             2
1         100     2015-06-05  Unique Leave Event 2             2
0         100     2016-01-01  Unique Leave Event 2             2
6         200     2013-01-01  Unique Leave Event 1             1
5         200     2015-01-01  Unique Leave Event 2             2
4         200     2016-01-01  Unique Leave Event 2             2
7         300        2014-01  Unique Leave Event 1             1

                 Output Match
3  Unique Leave Event 1  True
2  Unique Leave Event 2  True
1  Unique Leave Event 2  True
0  Unique Leave Event 2  True
6  Unique Leave Event 1  True
5  Unique Leave Event 2  True
4  Unique Leave Event 2  True
7  Unique Leave Event 1  True

为了清楚起见，更详细的示例：

import pandas as pd

data = {'Employee ID': ["100", "100", "100","100","200","200","200","300"],
        'Effective Date': ["2016-01-01","2015-06-05","2014-07-01","2013-01-01","2016-01-01","2015-01-01","2013-01-01","2014-01"],
        'Desired Output': ["Unique Leave Event 2","Unique Leave Event 2","Unique Leave Event 2","Unique Leave Event 1","Unique Leave Event 2","Unique Leave Event 2","Unique Leave Event 1","Unique Leave Event 1"]}
df = pd.DataFrame(data, columns=['Employee ID','Effective Date','Desired Output'])

# Group by Employee ID
grouped = df.groupby("Employee ID")

# Define a function to get the unique events
def get_unique_events(group):
     # Convert to date and sort by date, like @Khris did
    group["Effective Date"] = pd.to_datetime(group["Effective Date"])
    group = group.sort_values("Effective Date")
    # Define a series of booleans to determine whether the time between dates is over 365 days
    # Use .shift(1) to look back one row
    is_year = group["Effective Date"] - group["Effective Date"].shift(1) > pd.Timedelta('365 days')
    # Convert booleans to integers (0 for False, 1 for True)
    is_year_int = is_year.apply(lambda x: int(x))    
    # Use the cumulative sum function in pandas to get the cumulative adjustment from the first date.
    # Add one to start the first event as 1 instead of 0
    event_series = is_year_int.cumsum() + 1
    return event_series

# Run function on df and put results into a new dataframe
# Convert Employee ID back from an index to a column with .reset_index(level=0)
event_df = pd.DataFrame(grouped.apply(get_unique_events).rename("Unique Event")).reset_index(level=0)

# Merge the dataframes
df = pd.merge(df, event_df[['Unique Event']], left_index=True, right_index=True)

# Add string to match desired format
df['Output'] = df['Unique Event'].apply(lambda x: "Unique Leave Event " + str(x))

# Check to see if output matches desired output
df['Match'] = df['Desired Output'] == df['Output']

print(df)

你得到相同的输出：

  Employee ID Effective Date        Desired Output  Unique Event  \
3         100     2013-01-01  Unique Leave Event 1             1
2         100     2014-07-01  Unique Leave Event 2             2
1         100     2015-06-05  Unique Leave Event 2             2
0         100     2016-01-01  Unique Leave Event 2             2
6         200     2013-01-01  Unique Leave Event 1             1
5         200     2015-01-01  Unique Leave Event 2             2
4         200     2016-01-01  Unique Leave Event 2             2
7         300        2014-01  Unique Leave Event 1             1

                 Output Match
3  Unique Leave Event 1  True
2  Unique Leave Event 2  True
1  Unique Leave Event 2  True
0  Unique Leave Event 2  True
6  Unique Leave Event 1  True
5  Unique Leave Event 2  True
4  Unique Leave Event 2  True
7  Unique Leave Event 1  True

Answer 2

这有点笨拙，但至少对于你的小例子来说它产生了正确的输出：

import pandas as pd

data = {'Employee ID': ["100", "100", "100","100","200","200","200","300"],
        'Effective Date': ["2016-01-01","2015-06-05","2014-07-01","2013-01-01","2016-01-01","2015-01-01","2013-01-01","2014-01-01"],
        'Desired Output': ["Unique Leave Event 2","Unique Leave Event 2","Unique Leave Event 2","Unique Leave Event 1","Unique Leave Event 2","Unique Leave Event 2","Unique Leave Event 1","Unique Leave Event 1"]}
df = pd.DataFrame(data, columns=['Employee ID','Effective Date','Desired Output'])

df["Effective Date"] = pd.to_datetime(df["Effective Date"])
df = df.sort_values(["Employee ID","Effective Date"]).reset_index(drop=True)

for i,_ in df.iterrows():
  df.ix[0,"Result"] = "Unique Leave Event 1"
  if i < len(df)-1:
    if df.ix[i+1,"Employee ID"] == df.ix[i,"Employee ID"]:
      if df.ix[i+1,"Effective Date"] - df.ix[i,"Effective Date"] > pd.Timedelta('365 days'):
        df.ix[i+1,"Result"] = "Unique Leave Event " + str(int(df.ix[i,"Result"].split()[-1])+1)
      else:
        df.ix[i+1,"Result"] = df.ix[i,"Result"]
    else:
      df.ix[i+1,"Result"] = "Unique Leave Event 1"

请注意，此代码假定第一行始终包含字符串 Unique Leave Event 1。

编辑：一些解释。

首先我将日期转换为日期时间格式，然后重新排序数据框，使每个员工 ID 的日期都升序。

然后我使用内置的迭代器iterrows 遍历帧的行。 for i,_ 中的 _ 只是我不使用的第二个变量的占位符，因为迭代器同时返回行号和行，我只需要这里的数字。

在迭代器中，我进行逐行比较，所以默认情况下我手动填写第一行，然后分配给i+1-th 行。我这样做是因为我知道第一行的值，但不知道最后一行的值。然后我将i+1-th 行与if-safeguard 中的i-th 行进行比较，因为i+1 会在最后一次迭代中给出索引错误。

在循环中，我首先检查Employee ID 是否在两行之间发生了变化。如果没有，那么我比较两行的日期，看看它们是否相隔超过 365 天。如果是这种情况，我从i-th 行读取字符串"Unique Leave Event X"，将数字加一并将其写入i+1-行。如果日期更近，我只需从上一行复制字符串。

另一方面，如果Employee ID 确实发生了变化，我只需写"Unique Leave Event 1" 重新开始。

注意 1：iterrows() 没有要设置的选项，所以我不能只迭代一个子集。

注意 2：始终使用内置迭代器之一进行迭代，并且仅在无法解决问题时进行迭代。

注意 3：在迭代中赋值时，请始终使用 ix、loc 或 iloc。