在 for 循环中的计数器在输出中添加重复项，在 Python 中删除输出中的重复条目

Question

我想从我的函数输出中删除重复的条目。

我有一个函数可以在文本文件的每个段落中搜索关系。一个样本 metadata.csv 文件在段落中搜索关系如下：

Blister     Base Web    PVC/PVDC
Blister     Foil         Aluminium
Blister     Base Web    PVC/PVDC
Blister     Foil         Aluminium
Vial        Glass       Borosilicate Glass
Vial        Stopper     Bromobutyl Rubber
Vial        Cap         Aluminium

示例文本文件如下：

The tablets are filled into cylindrically shaped bottles made of white coloured
polyethylene. The volumes of the bottles depend on the tablet strength and amount of
tablets, ranging from 20 to 175 ml. The screw type cap is made of white coloured
polypropylene and is equipped with a tamper proof ring.

PVC/PVDC blister pack

Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tablet
is filled into a separate blister and a lidding foil of aluminium is welded on. The blisters
are opened by pressing the tablets through the lidding foil. PVDC foil is in contact with
the tablets.

Aluminium blister pack

Blisters are made in a cold-forming process from an aluminium base web. Each tablet is
filled into a separate blister and a lidding foil of aluminium is welded on. The blisters
are opened by pressing the tablets through the lidding foil.

所以这个文本文件将有 3 个组。

函数如下：

import csv
import re
import os         
def extractor(filepath):
    #pdb.set_trace()    
    TEXT_WITHOUT_COLOUR = 'Stage {counter} : Package Description: {sen} Values: {values}'
    TEXT_WITH_COLOUR = TEXT_WITHOUT_COLOUR + ','  ' Colour: {colour}'
    colours = ['White', 'Yellow', 'Blue', 'Red', 'Green', 'Black', 'Brown', 'Silver', 'Purple', 'Navy blue', 'Gray', 'Orange', 'Maroon', 'pink', 'colourless', 'blue']
    counter = 1
    result = [] 
    unique_desc = [] #every unique description is stored 
    output      = [] 
    with open(filepath, encoding='utf-8') as f:
        data=f.read()
        paragraphs=data.split("\n\n")
    inputfile = r"C:\metadata.csv"                
    inputm = []

    with open(inputfile, "r") as f:
        reader = csv.reader(f, delimiter="\t")
        for row in reader:
            #types = row.split(',')
            inputm.append(row)

    final_ref = [] 
    for lists in inputm:
        final_ref.append(str(lists[0]).split(','))
    def is_missing(words, sen):
        for w in words:
            if w.lower() not in sen.lower():
                return True
        return False




    for sen in paragraphs:
        for words in final_ref:
            if is_missing(words, sen):
                continue

            kwargs = {
                'counter': counter,
                'sen': sen,
                'values': str(words)
            }

            if (words[0] == 'Bottle') or (words[0]=='Vial') or (words[0] =='Container') or (words[0] =='Ampoules') or (words[0] =='Occlusive Dressing'):
                for wd in colours:
                    if wd.lower() in sen.lower():
                        kwargs['colour'] = wd
                        break
                text_const = TEXT_WITH_COLOUR
            else:
                text_const = TEXT_WITHOUT_COLOUR

            result.append(text_const.format(**kwargs).replace('\n', '').replace('\t', ''))




            for desc in result:

                compare = re.search(r'Package Description:(.*?)Values:',desc).group(1).replace(' ','') #clean spaces

                if compare in unique_desc:  

                    group = str(unique_desc.index(compare)+1) #index starts in 0 and group in 1     
                    desc = re.sub('Stage \d','Group '+group, desc)
                    output.append(desc)

                else: 

                    unique_desc.append(compare)     
                    group = str(len(unique_desc))    #new group

                    desc = re.sub('Stage \d','Group '+group, desc)
                    output.append(desc)
                    counter+=1
                    break








    return output      

作为输出返回

["Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cylindrically shaped Bottles', 'Polyethylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cylindrically shaped Bottles', 'Polyethylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Screw Type Cap', 'Polypropylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cylindrically shaped Bottles', 'Polyethylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Screw Type Cap', 'Polypropylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Tamper Proof Ring', ''], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cylindrically shaped Bottles', 'Polyethylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Screw Type Cap', 'Polypropylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Tamper Proof Ring', ''], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cap', 'Polypropylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cylindrically shaped Bottles', 'Polyethylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Screw Type Cap', 'Polypropylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Tamper Proof Ring', ''], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cap', 'Polypropylene'], Colour: White",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Foil', 'Aluminium']",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cylindrically shaped Bottles', 'Polyethylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Screw Type Cap', 'Polypropylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Tamper Proof Ring', ''], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cap', 'Polypropylene'], Colour: White",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Foil', 'Aluminium']",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Base Web', 'PVC/PVDC']",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cylindrically shaped Bottles', 'Polyethylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Screw Type Cap', 'Polypropylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Tamper Proof Ring', ''], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cap', 'Polypropylene'], Colour: White",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Foil', 'Aluminium']",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Base Web', 'PVC/PVDC']",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Base Web', 'PVC']",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cylindrically shaped Bottles', 'Polyethylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Screw Type Cap', 'Polypropylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Tamper Proof Ring', ''], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cap', 'Polypropylene'], Colour: White",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Foil', 'Aluminium']",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Base Web', 'PVC/PVDC']",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Base Web', 'PVC']",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Base Web', 'PVDC']",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cylindrically shaped Bottles', 'Polyethylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Screw Type Cap', 'Polypropylene'], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Tamper Proof Ring', ''], Colour: White",
 "Group 1 : Package Description: The tablets are filled into cylindrically shaped bottles made of white colouredpolyethylene. The volumes of the bottles depend on the tablet strength and amount oftablets, ranging from 20 to 175 ml. The screw type cap is made of white colouredpolypropylene and is equipped with a tamper proof ring. Values: ['Bottle', 'Cap', 'Polypropylene'], Colour: White",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Foil', 'Aluminium']",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Base Web', 'PVC/PVDC']",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Base Web', 'PVC']",
 "Group 2 : Package Description: Blisters are made in a thermo-forming process from a PVC/PVDC base web. Each tabletis filled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. PVDC foil is in contact withthe tablets. Values: ['Blister', 'Base Web', 'PVDC']",
 "Group 3 : Package Description: Blisters are made in a cold-forming process from an aluminium base web. Each tablet isfilled into a separate blister and a lidding foil of aluminium is welded on. The blistersare opened by pressing the tablets through the lidding foil. Values: ['Blister', 'Foil', 'Aluminium']"]

应从输出中忽略具有完全相同条目的“包描述和值”。我认为我的 break 语句和 counter 变量存在问题。有关此的任何帮助。

Answer 1

如果您愿意，可以在set 的帮助下实现这一目标。但请注意，虽然您可以实现唯一性，但不会保留顺序。

lst = ['foo', 'bar', 'foo', 'bar']
s = set(lst)
print(s) # this will print foo and bar