【问题标题】:Find matching substrings in two dataframes在两个数据框中查找匹配的子字符串
【发布时间】:2017-08-02 19:49:40
【问题描述】:

我有两个这样的数据框:

[in]print(training_df.head(n=10))

[out]
                          product_id
transaction_id                      
0000001                   [P06, P09]
0000002         [P01, P05, P06, P09]
0000003                   [P01, P06]
0000004                   [P01, P09]
0000005                   [P06, P09]
0000006                   [P02, P09]
0000007         [P01, P06, P09, P10]
0000008                   [P03, P05]
0000009                   [P03, P09]
0000010         [P03, P05, P06, P09]

[in]print(testing_df.head(n=10))

[out]
                     product_id
transaction_id                 
001                       [P01]
002                  [P01, P02]
003             [P01, P02, P09]
004                  [P01, P03]
005             [P01, P03, P05]
006             [P01, P03, P07]
007             [P01, P03, P08]
008                  [P01, P04]
009             [P01, P04, P05]
010             [P01, P04, P08]

testing_df 中的每一行都是 training_df 中一行的可能“子字符串”。我想找到所有匹配项并为 testing_df 中的每个列表返回可能的 training_df 列表。如果我可以返回一个字典,其中键是 testing_df 中的 transaction_id 并且值是 training_df 中所有可能的“匹配项”,那将很有帮助。 (training_df 中的每个列表都应该比 test_df 中的相应列表长一个值)。

我试过了:

# Find the substrings that match
matches = []

for string in training_df:
    results = []
    for substring in testing_df:
        if substring in string:
            results.append(substring)
    if results:
        matches.append(results)  

但这不起作用,它只返回列名“product_id”。

我也试过了:

# Initialize a list to store the matches between incomplete testing_df and training_df
matches = {}

# Compare the "incomplete" testing lists to the training set
for line in testing_df.product_id:
    for line in training_df.product_id:
        if line in testing_df.product_id in line in training_df.product_id:
            matches[line] = training_df[training_df.product_id.str.contains(line)]

但是这会引发错误TypeError: unhashable type: 'list'

【问题讨论】:

  • 我认为问题在于括号。例如,“P01”是“[P01, P06]”的子字符串,但“[P01]”不是。您可以尝试 substring[1:-1] 而不是 substring 以摆脱括号。
  • @csander 我试过matches = [] for string in training_df[1:-1]: results = [] for substring in testing_df[1:-1]: if substring in string: results.append(substring) if results: matches.append(results),但也没有用
  • 不,你不想切片DataFrame,你想切片子字符串
  • 仍然只返回列名[['product_id']]

标签: python pandas string-matching


【解决方案1】:

我认为问题在于括号。问题是in 检查一个元素是否在一个列表中,而不是一个列表是否是另一个列表的子集。您可以将两个列表转换为集合,然后检查它们是否是彼此的子集。您还可以使用高级索引来保留transaction_id

training_df = pd.DataFrame([
    ['0000001', ['P06', 'P09']],
    ['0000002', ['P01', 'P05', 'P06', 'P09']],
    ['0000003', ['P01', 'P06']],
    ['0000004', ['P01', 'P09']],
    ['0000005', ['P06', 'P09']],
    ['0000006', ['P02', 'P09']],
    ['0000007', ['P01', 'P06', 'P09', 'P10']],
    ['0000008', ['P03', 'P05']],
    ['0000009', ['P03', 'P09']],
    ['0000010', ['P03', 'P05', 'P06', 'P09']],
], columns=['transaction_id', 'product_id'])

testing_df = pd.DataFrame([
    ['001', ['P01']],
    ['002', ['P01', 'P02']],
    ['003', ['P01', 'P02', 'P09']],
    ['004', ['P01', 'P03']],
    ['005', ['P01', 'P03', 'P05']],
    ['006', ['P01', 'P03', 'P07']],
    ['007', ['P01', 'P03', 'P08']],
    ['008', ['P01', 'P04']],
    ['009', ['P01', 'P04', 'P05']],
    ['010', ['P01', 'P04', 'P08']],
], columns=['transaction_id', 'product_id'])

matches = {}
for testing_id in testing_df.product_id:
    testing_id_set = set(testing_id)
    contains_id = training_df.product_id.apply(lambda id: testing_id_set.issubset(set(id)))
    matches[str(testing_id)] = contains_id

【讨论】:

  • testing_df 和 training_df 已经格式化为 pandas 数据帧。当我尝试matches = {} for testing_id in testing_df.product_id: matches[testing_id] = training_df[training_df.product_id.str.contains(testing_id[1:-1])] 时,它只会返回一个空字典
  • 我包含了定义以显示我正在测试的数据。如果它不适合您,您的数据框是什么样的?
  • 括号在外面所以例如第1行,transaction_id 001,product_id列如下:['P01','P02','P03']
  • 好的,我已经更新了答案以考虑到这一点
  • 我会试试这个但请看这个问题link我已经稍微更新了问题
猜你喜欢
  • 2018-12-07
  • 2015-01-11
  • 2019-03-16
  • 1970-01-01
  • 2014-05-07
  • 1970-01-01
  • 2019-11-27
  • 1970-01-01
  • 2014-02-11
相关资源
最近更新 更多