在数据框列中搜索字符串模式，返回每个出现并加入另一个数据框

Question

我正在尝试做类似于How to loop through pandas df column, finding if string contains any string from a separate pandas df column?的事情，特别是第二个问题

我的初始数据框logs 包含以下内容：

ID DB  USER  MDX_TEXT
1  DB1  JOE  SELECT [ATTR1].[ATTR1].[THE_ATTR1] ON COLUMNS,[ATTR2].[ATTR2].[THE_ATTR2] ON ROWS FROM [THE_DB] WHERE [ATTR3].[ATTR3].[THE_ATTR3]
2  DB1  JAY  SELECT [ATTR1].[ATTR1].[THE_ATTR1] ON COLUMNS, [ATTR3].[ATTR3].[THE_ATTR3] ON ROWS FROM [THE_DB] WHERE [ATTR3].[ATTR3].[THE_ATTR3]

使用正则表达式，然后我根据 ID 提取 MDX_TEXT 的唯一实例

# Step 1: Define a pattern to extract MDX regex pattern
import re
pattern = re.compile (r'(\[[\w ]+\]\.\[[\w ]+\](?:\.(?:Members|\[Q\d\]))?)')

# Step 2: Create a dataframe to store distinct list of MDX Query attributes, based on pattern
extrpat =  (
            logs['MDX_TEXT'].str.extractall(pattern)
                            .drop_duplicates()
                            .to_numpy()
           )
# Step 3: Create a dataframe to store distinct list of attributes used in a query
attr= pd.DataFrame(data=extrpat)
attr.rename({0: 'attrname'}, inplace=True, axis=1)

# Preview the first 5 lines of the attributes dataframe
attr.head()

结果：

attrname
[THE_ATTR1]
[THE_ATTR2]
[THE_ATTR3]
[THE_ATTR1]
[THE_ATTR3]

我想要的是，除了在步骤 2 中提取唯一属性之外，还要提取 ID 和 USER，如下所示：

ID  USER  attrname
1  JOE  [THE_ATTR1]
1  JOE  [THE_ATTR2]
1  JOE  [THE_ATTR3]
2  JAY  [THE_ATTR1]
2  JAY  [THE_ATTR3]

最后，在ID 上加入attr 和logs 数据帧。这个想法是引入第三个数据框users：

USER  LOC
JOE  NY
JIL  NJ
MAC  CA

...我将在USER 上加入上述内容以结束：

ID  USER  LOC  attrname
1  JOE  NY  [THE_ATTR1]
1  JOE  NY  [THE_ATTR2]
1  JOE  NY  [THE_ATTR3]
2  JAY  NJ  [THE_ATTR1]
2  JAY  NJ  [THE_ATTR3]

Answer 1

（已编辑）。

模式片是一个好的开始，但是你必须将它与原始数据框合并/加入：

df.index.name = "inx"
pattern = re.compile (r'(\[[\w ]+\]\.\[[\w ]+\])')

# extract the attributes. 
extracts = df.MDX_TEXT.str.extractall(pattern).rename(columns={0:"attrname"})

# join the result with the original dataframe. 
res = df.join(extracts).reset_index()[["ID", "USER", "attrname"]].drop_duplicates()

# take just the last part of each attribute name. 
res["attrname"] = res["attrname"].str.split(".", expand = True).iloc[:, -1]

结果是：

   ID USER attrname
0   1  JOE  [ATTR1]
1   1  JOE  [ATTR2]
2   1  JOE  [ATTR3]
3   2  JAY  [ATTR1]
4   2  JAY  [ATTR3]