如何在熊猫数据框中对类似的新闻文本/文章进行分组

Question

我有一个新闻文章的熊猫数据框。假设

id	news title	keywords	publcation date	content
1	Congress Wants to Beef Up Army Effort to Develop Counter-Drone Weapons	USA,Congress,Drone,Army	2020-12-10	SOME NEWS CONTENT
2	Israel conflict: The range and scale of Hamas' weapons ...	Israel,Hamas,Conflict	2020-12-10	NEWS CONTENT
3	US Air Force progresses testing of anti-drone laser weapons	USA,Air Force,Weapon,Dron	2020-10-10	NEWS CONTENT
4	Hamas fighters display weapons in Gaza after truce with Israel	Hamas,Gaza,Israel,Weapon,Truce	2020-11-10	NEWS CONTENT

现在
如何根据新闻内容对相似数据进行分组并按发布日期排序
注意：内容可能是新闻摘要
使其显示为：

---

组 1

id	news title	keywords	publcation date	content
3	US Air Force progresses testing of anti-drone laser weapons	USA,Air Force,Weapon,Dron	2020-10-10	NEWS CONTENT
1	Congress Wants to Beef Up Army Effort to Develop Counter-Drone Weapons	USA,Congress,Drone,Army	2020-12-10	SOME NEWS CONTENT

---

组 2

id	news title	keywords	publcation date	content
4	Hamas fighters display weapons in Gaza after truce with Israel	Hamas,Gaza,Israel,Weapon,Truce	2020-11-10	NEWS CONTENT
2	Israel conflict: The range and scale of Hamas' weapons ...	Israel,Hamas,Conflict	2020-12-10	NEWS CONTENT

Answer 1

有点复杂，相似度我选择简单的方式，但你可以随意改变功能。

1. 您也可以将https://pypi.org/project/pyjarowinkler/ 用于is_similar 函数，而不是我所做的“set”。 *这个功能可能比我做的要复杂得多
   

2. 我使用了两个应用第一个是适合“grps”。没有第一次也可以，但第二次会更准确
   

3. 您还可以将范围（3，-1，-1）更改为更高的数字以提高准确性

    def is_similar(txt1,txt2,level=0):
        return len(set(txt1) & set(txt2))>level
   
    grps={}
    def get_grp_id(row):
        row_words = row['keywords'].split(',')
        if len(grps.keys())==0:
            grps[1]=set(row_words)
            return 1
        else:
            for level in range(3,-1,-1):
                for grp in grps:
                    if is_similar(grps[grp],row_words,level):
                        grps[grp]= grps[grp] | set(row_words)
                        return grp
   
            grp +=1
            grps[grp]=set(row_words)
            return grp
   
   
    df.apply(get_grp_id,axis=1)
    df['grp'] = df.apply(get_grp_id,axis=1)
    df = df.sort_values(['grp','publcation date'])
   

这是输出

如果您想将其拆分为单独的 df，请告诉我