根据另一个数据框的值更新数据框

Question

我有一个由 ID 和日期组成的数据框。一个 ID 可能有多个日期 - ID 按照每个 ID 的日期排序。

AccidentDates

我的第二个数据框由 ID、开始日期、完成日期、布尔列 Accident（表示发生事故）和 Time to event 列组成。最后两列最初设置为 0。再次对 ID 以及每个 ID 的时间间隔进行排序。

PatientLog

我想根据第一个数据帧记录的事故更新第二个数据帧的两列。如果 ID 在两个数据帧上都存在（不是必须的），请检查是否在第二个数据帧的任何时间间隔内记录了任何事故。

如果有，找出它发生在哪个时间间隔内，将“事故”列更新为 1，时间 = df1.Date - df2.Start。 如果不是，则为患者的该条目设置 Accident = 0 和 Time = df2.Finish - df2.Start。

我设法通过列表和 for 循环做到了这一点。但是，我想知道是否有更聪明的方法，因为数据量很大，完成整个过程需要很多时间。提前致谢！

# Temporary lists
df1list = []
df2list = []

# Change format from dataframe to list
for row in df1.itertuples(index=True, name='Pandas'):

    # Get Patient ID and the date of the recorded accident
    df1list.append([getattr(row, "Patient"), getattr(row, "regdatum")])


# Change format from dataframe to list
for row in df2.itertuples(index=True, name='Pandas'):

    # Get Patient ID, info, occurrence of accident and time to event
    df2list.append([getattr(row, "Patient"), getattr(row, "Start"), getattr(row, "Finish"), getattr(row, "Gender"),
                   getattr(row, "Age"), getattr(row, "Accident"), getattr(row, "Time")])


#For each interval of each patient
for i in range(0, len(df2list)):

    #For each recorded accident of each patient
    for j in range(0, len(df1list)):

        #If there's a match in both lists
        if df2list[i][0] == df1list[j][0]:

            #If the recorded date is in between the time interval
            if (df1list[j][1] >= datetime.strptime(df2list[i][1], '%Y-%m-%d')) & (df1list[j][1] <= datetime.strptime(df2list[i][2], '%Y-%m-%d')):

                #Change the accident column to 1 and calculate the time to event
                #The extra if is to verify that this is the recorded accident is the first one to have happened within the time interval (if there are multiple, we only keep the first one)    
                if df2list[i][6] == 0 :
                    df2list[i][6] = 1
                    df2list[i][7] = df1list[j][1] - datetime.strptime(df2list[i][1], '%Y-%m-%d')

#Back to dfs
labels = ['Patient', 'Start', 'Finish', 'Gender', 'Age', 'Accident', 'Time']
df = pd.DataFrame.from_records(df2list, columns=labels)
```

Answer 1

我会这样做。

# Define a pair of functions that return the list of unique start and end dates for a given patient
def start_dates(patient):
    try:
        return df2.loc[df2['Patient'] == patient]['Start'].unique()
    except:
        return np.datetime64("NaT")

def finish_dates(patient):
    try:
        return df2.loc[df2['Patient'] == patient]['Finish'].unique()
    except:
        return np.datetime64("NaT")

# Add and fill 'Start' and 'Finish' columns to df1
df1['Start'] = list(zip(df1['Patient'], df1['Accident Date']))
df1['Start'] = df1['Start'].apply(lambda x: max([d for d in start_dates(x[0]) if d <= np.datetime64(x[1])]))
df1['Finish'] = list(zip(df1['Patient'], df1['Accident Date']))
df1['Finish'] = df1['Finish'].apply(lambda x: min([d for d in finish_dates(x[0]) if d >= np.datetime64(x[1])]))

# Merge the two DataFrames
df2 = df2.merge(df1, how='outer')

# Fill the 'Accident' column appropriately, and convert to int
df2['Accident'] = ~pd.isna(df2.iloc[:,5])
df2 = df2.astype({'Accident': int})

# Fill NaT fields in 'Accident Date' with 'Finish'
df2 = df2.fillna({'Accident Date': df2['Finish']})

# Fill 'Time' appropriately
df2['Time'] = df2['Accident Date'] - df2['Start']

# Drop the 'Accident Date' column
df2 = df2.drop(columns=['Accident Date'])

这适用于我创建的一些虚拟数据，我认为它应该适用于您的数据。我怀疑这是最有效的做事方式（我远不是 pandas 专家），但我认为它通常比使用循环更好。