从一列中获取时间戳并将其与 pandas 一起放入另一列

Question

我正在使用看门狗来监控一些文件夹并创建一个 csv 文件来查看文件的创建和修改时间。现在，我从 csv 中得到了这样的数据框：

       full_path             name             created           modified
0    C:\T1\1.txt            1.txt            14:04:30             NaN
1    C:\T1\1.txt            1.txt              NaN              14:04:30
2    C:\T1\T2\1.txt         1.txt            14:10:30              NaN
3    C:\T1\T2\1.txt         1.txt              NaN              14:10:30
4    C:\T1\T2\T3\1.txt      1.txt            14:15:30             NaN
5    C:\T1\T2\T3\1.txt      1.txt              NaN              14:15:30
6    C:\T1\T2\T3\T4\1.txt   1.txt            14:20:30             NaN
7    C:\T1\T2\T3\T4\1.txt   1.txt              NaN              14:20:30
8    C:\T1\2.txt            2.txt            14:25:30             NaN
9    C:\T1\2.txt            2.txt              NaN              14:25:30
10   C:\T1\T2\2.txt         2.txt            14:30:30             NaN
11   C:\T1\T2\2.txt         2.txt              NaN              14:30:30
12   C:\T1\T2\T3\2.txt      2.txt            14:35:30             NaN
13   C:\T1\T2\T3\2.txt      2.txt              NaN              14:35:30
14   C:\T1\T2\T3\T4\2.txt   2.txt              NaN              14:40:30

看门狗，通常在文件移动到另一个文件夹（创建和修改）时给出两个时间戳，但不知何故，当最后一个文件移动到 T4 文件夹时，它只给出修改后的时间戳。我使用此代码仅将此数据帧转换为两行，并将每个文件夹的时间戳放在其他列中：

m0 = (df["full_path"].ne(df["full_path"].shift(1, fill_value=df["full_path"].iloc[0])) & df["name"].eq(df["name"].shift(fill_value=df["name"].iloc[0])))
m1 = df["full_path"].eq(df.loc[df["full_path"].str.rsplit("\\", 2).str[-2] == 'T1', 'full_path'])
m2 = df["full_path"].eq(df.loc[df["full_path"].str.rsplit("\\", 2).str[-2] == 'T2', 'full_path'])
m3 = df["full_path"].eq(df.loc[df["full_path"].str.rsplit("\\", 2).str[-2] == 'T3', 'full_path'])
m4 = df["full_path"].eq(df.loc[df["full_path"].str.rsplit("\\", 2).str[-2] == 'T4', 'full_path'])

df['T1'] = np.where(m0 & m1, df['created'], "")
df['T2'] = np.where(m0 & m2, df['created'], "")
df['T3'] = np.where(m0 & m3, df['created'], "")
df['T4'] = np.where(m0 & m4, df['created'], "")

df = df.groupby(['name'], sort=False).agg({'full_path':'last','created':'first', 'modified':'last','T1':'first', 'T2':lambda x: ' '.join(set(x)), 'T3':lambda x: ' '.join(set(x)), 'T4':'last'}).reset_index()

它给了我一个像这样的数据框：

   full_path             name   created    modified      T1       T2        T3          T4
0  C:\T1\T2\T3\T4\1.txt  1.txt  14:04:30   14:20:30  14:04:30   14:10:30  14:15:30   14:20:30  
1  C:\T1\T2\T3\T4\2.txt  2.txt  14:25:30   14:40:30  14:25:30   14:30:30  14:35:30      NaN

如何修改我的代码以检测例如文件 2.txt 当它移动到文件夹 T4 时，如果 NaN 中的“创建”列，从“修改”列中获取时间戳，并具有类似的数据框这个：

   full_path             name   created    modified      T1       T2        T3          T4
0  C:\T1\T2\T3\T4\1.txt  1.txt  14:04:30   14:20:30  14:04:30   14:10:30  14:15:30   14:20:30  
1  C:\T1\T2\T3\T4\2.txt  2.txt  14:25:30   14:40:30  14:25:30   14:30:30  14:35:30   14:40:30

Answer 1

所以回答具体问题。使用 fillna 并在 groupby 之前传入来自“修改”的值，这将用修改的值填充 T4 中的 NaN 值：

df['T4'] = df['T4'].fillna(df['modified'])

---

作为替代方法，可以使用类似 pathlib 的东西来获取父文件夹的名称。

进口

from pathlib import PurePath

import numpy as np
import pandas as pd

示例帧

df = pd.DataFrame({
    'full_path': {0: 'C:\\T1\\1.txt', 1: 'C:\\T1\\1.txt',
                  2: 'C:\\T1\\T2\\1.txt', 3: 'C:\\T1\\T2\\1.txt',
                  4: 'C:\\T1\\T2\\T3\\1.txt',
                  5: 'C:\\T1\\T2\\T3\\1.txt',
                  6: 'C:\\T1\\T2\\T3\\T4\\1.txt',
                  7: 'C:\\T1\\T2\\T3\\T4\\1.txt',
                  8: 'C:\\T1\\2.txt', 9: 'C:\\T1\\2.txt',
                  10: 'C:\\T1\\T2\\2.txt',
                  11: 'C:\\T1\\T2\\2.txt',
                  12: 'C:\\T1\\T2\\T3\\2.txt',
                  13: 'C:\\T1\\T2\\T3\\2.txt',
                  14: 'C:\\T1\\T2\\T3\\T4\\2.txt'},
    'name': {0: '1.txt', 1: '1.txt', 2: '1.txt', 3: '1.txt',
             4: '1.txt', 5: '1.txt', 6: '1.txt', 7: '1.txt',
             8: '2.txt', 9: '2.txt', 10: '2.txt', 11: '2.txt',
             12: '2.txt', 13: '2.txt', 14: '2.txt'},
    'created': {0: '14:04:30', 1: np.nan, 2: '14:10:30', 3: np.nan,
                4: '14:15:30', 5: np.nan, 6: '14:20:30', 7: np.nan,
                8: '14:25:30', 9: np.nan, 10: '14:30:30', 11: np.nan,
                12: '14:35:30', 13: np.nan, 14: np.nan},
    'modified': {0: np.nan, 1: '14:04:30', 2: np.nan, 3: '14:10:30',
                 4: np.nan, 5: '14:15:30', 6: np.nan, 7: '14:20:30',
                 8: np.nan, 9: '14:25:30', 10: np.nan, 11: '14:30:30',
                 12: np.nan, 13: '14:35:30', 14: '14:40:30'}
})

获取文件夹名称

# Get Parent Folder Name From Each Path
df['folder'] = df['full_path'].apply(lambda x: PurePath(x).parent.name)
print(df.to_string())

               full_path   name   created  modified folder
0            C:\T1\1.txt  1.txt  14:04:30       NaN     T1
1            C:\T1\1.txt  1.txt       NaN  14:04:30     T1
2         C:\T1\T2\1.txt  1.txt  14:10:30       NaN     T2
3         C:\T1\T2\1.txt  1.txt       NaN  14:10:30     T2
4      C:\T1\T2\T3\1.txt  1.txt  14:15:30       NaN     T3
5      C:\T1\T2\T3\1.txt  1.txt       NaN  14:15:30     T3
6   C:\T1\T2\T3\T4\1.txt  1.txt  14:20:30       NaN     T4
7   C:\T1\T2\T3\T4\1.txt  1.txt       NaN  14:20:30     T4
8            C:\T1\2.txt  2.txt  14:25:30       NaN     T1
9            C:\T1\2.txt  2.txt       NaN  14:25:30     T1
10        C:\T1\T2\2.txt  2.txt  14:30:30       NaN     T2
11        C:\T1\T2\2.txt  2.txt       NaN  14:30:30     T2
12     C:\T1\T2\T3\2.txt  2.txt  14:35:30       NaN     T3
13     C:\T1\T2\T3\2.txt  2.txt       NaN  14:35:30     T3
14  C:\T1\T2\T3\T4\2.txt  2.txt       NaN  14:40:30     T4

那么最终的结果可以更容易完成：

# Get Parent Folder Name From Each Path
df['folder'] = df['full_path'].apply(lambda x: PurePath(x).parent.name)
# Keep Groupby Name handy for later
g = df.groupby('name')
# Transform each group to be the last path
df['full_path'] = g['full_path'].transform('last')
# combine_first created with modified (new column to not affect data)
df['c_m'] = df['created'].combine_first(df['modified'])

index_cols = ['full_path', 'name']
# Pivot to wide format
df = df.pivot_table(index=index_cols,
                    columns='folder',
                    values='c_m',
                    aggfunc='first')

# Add Summary Columns
summary_cols = ['created', 'modified']
# Merge on name
df = df.reset_index() \
    .merge(g[summary_cols].agg({'created': 'first', 'modified': 'last'}),
           on='name')

# Re-order Columns and axis names
df = df[[*index_cols,
         *summary_cols,
         *df.columns.difference(summary_cols + index_cols)]] \
    .rename_axis(None, axis=1)

# Output
print(df.to_string(index=False))

df:

           full_path  name  created modified       T1       T2       T3       T4
C:\T1\T2\T3\T4\1.txt 1.txt 14:04:30 14:20:30 14:04:30 14:10:30 14:15:30 14:20:30
C:\T1\T2\T3\T4\2.txt 2.txt 14:25:30 14:40:30 14:25:30 14:30:30 14:35:30 14:40:30