如何从字符串中生成唯一数据

Question

我有这样的数据。字符串用逗号分隔。

"India1,India2,myIndia     "
"Where,Here,Here   "
"Here,Where,India,uyete"
"AFD,TTT"

我想要做的是将它们全部放在一列中（彼此下方）所以它会变成这样

India1
India2
myIndia
Where
Here
Here
Here
Where
India
uyete
AFD
TTT

然后我保留导致这个的独特的那些

India1
India2
myIndia
Where
Here
India
uyete
AFD
TTT

所以我有第一个 .txt 格式的数据，我尝试使用 numpy 来处理这个

这是我的代码

#!/usr/bin/python
import numpy as np

# give a name to my data 
file_name = 'path to my data/test.txt'
# set my output 
with open ( 'output.txt' , 'w' ) as out:
    # read all the lines
    for n , line in enumerate ( open ( file_name ).readlines ( ) ):
        # split each stirg from another one by a comma
        item1 = file_name.split ( ',' )
    myList = ','.join ( map ( str , item1 ) )
    item2 = np.unique ( myList , return_inverse=True )
    # save the data into out
    out.write ( item2 )

我收到了TypeError: expected a character buffer object

我已经搜索过了，我发现了几个类似的帖子 TypeError: expected a character buffer object - while trying to save integer to textfile

如果我添加了out.seek ( 0 )，我仍然会遇到同样的错误

但是通过将其更改为out.write ( str(item2 )) 感谢TypeError: expected a character buffer object，我没有收到任何错误，但是输出显示了这个

(array(['/文件路径/test.txt'], dtype='|S29'), 数组([0]))

下面给出了我尝试使用的解决方案

import csv

data = []
def remove_quotes(file):
    for line in file:
        yield line.strip ( '"\n' )
with open ( 'test.txt' ) as f:
    reader = csv.reader ( remove_quotes ( f ) )
    for row in reader:
        data.extend ( row )

没有错误但也没有生成data

Answer 1

stack.txt 包含以下内容：

"India1,India2,myIndia"
"Where,Here,Here"
"Here,Where,India,uyete"
"AFD,TTT"

给你：

from collections import OrderedDict

with open("stack.txt", "r") as f:
    # read your data in from the gist site and strip off any new-line characters
    data = [eval(line.strip()) for line in f.readlines()]
    # get individual words into a list
    individual_elements = [word for row in data for word in row.split(",")]
    # remove duplicates and preserve order
    uniques = OrderedDict.fromkeys(individual_elements)   
    # convert from OrderedDict object to plain list
    final = [word for word in uniques]

print(final)

产生这个：

['India1', 'India2', 'myIndia', 'Where', 'Here', 'India', 'uyete', 'AFD', 'TTT']

编辑：要获得所需的输出，只需以所需的格式打印列表：

print("\n".join(final))

从输出的角度来看，这相当于：

for x in final:
    print(x)

产生这个：

India1
India2
myIndia
Where
Here
India
uyete
AFD
TTT

Answer 2

为什么要使用 numpy ？？？而且我不确定您是否要使用相同的文件作为输入和输出

#!/usr/bin/env python


# give a name to my data 
inputData = """India1,India2,myIndia
Where,Here,Here   
Here,Where,India,uyete
AFD,TTT"""

# if you want to read the data from a file
#inputData = open(fileName, 'r').readlines()

outputData = ""
tempData = list()
for line in inputData.split("\n"):
    lineStripped = line.strip()
    lineSplit = lineStripped.split(',')
    lineElementsStripped = [element.strip() for element in lineSplit]
    tempData.extend( lineElementsStripped )
tempData = set(tempData)
outputData = "\n".join(tempData)
print("\nInputdata: \n%s" % inputData)
print("\nOutputdata: \n%s" % outputData)

Answer 3

听起来您可能有一个 csv 文件。你不需要 numpy，随附的电池就是你所需要的。

 import csv

 data = []
 with open('test.txt') as f:
     reader = csv.reader(f)
     for row in reader:
         data.extend(row)

您可以.extend 列出而不是.append。基本上就像在说

for thing in row:
    data.append(thing)

不过，这仍然会留下重复项。如果您不关心订单，您可以将其设为 set 并调用 .update() 而不是扩展：

 data = set()
 with open('test.txt') as f:
     reader = csv.reader(f)
     for row in reader:
         data.extend(row)

现在一切都是独一无二的。但是，如果您关心订单，则必须稍微过滤一下：

unique_data = []
for thing in data:
    if thing not in unique_data:
        unique_data.append(thing)

如果您的test.txt 文件包含此文本：

"India1,India2,myIndia     "
"Where,Here,Here   "
"Here,Where,India,uyete"
"AFD,TTT"

不是

India1,India2,myIndia     
Where,Here,Here   
Here,Where,India,uyete
AFD,TTT

那么你还没有一个 csv。您可以修复生成 csv 的内容，也可以手动删除引号或即时修复它。

def remove_quotes(file):
    for line in file:
        yield line.strip('"\n')

reader = csv.reader(remove_quotes(f))