没有NLTK的python中的POS标记器

Question

我正在尝试为 Sorani Kurdish 的限定词和介词制作词性标注器。我正在使用以下代码将每个标签放在我的库尔德语文本中的每个命题或确定符之后。

import os
SOR = open("SOR-1.txt", "r+", encoding = 'utf-8')
old_text = SOR.read()
punkt = [".", "!", ",", ":", ";"]
text = ""
for i in old_text:
    if i in punkt:
        text+=" "+i
    else:
        text += i

d = {"DET":["ئێمە" , "ئێوە" , "ئەم" , "ئەو" , "ئەوان" , "ئەوەی", "چەند" ], "PREP":["بۆ","بێ","بێجگە","بە","بەبێ","بەدەم","بەردەم","بەرلە","بەرەوی","بەرەوە","بەلای","بەپێی","تۆ","تێ","جگە","دوای","دەگەڵ","سەر","لێ","لە","لەبابەت","لەباتی","لەبارەی","لەبرێتی","لەبن","لەبەینی","لەبەر","لەدەم","لەرێ","لەرێگا","لەرەوی","لەسەر","لەلایەن","لەناو","لەنێو","لەو","لەپێناوی","لەژێر","لەگەڵ","ناو","نێوان","وەک","وەک","پاش","پێش","" ], "punkt":[".", ",", "!"]}

text = text.split()
for w in text:
    for pos in d:
        if w in d[pos]:
            SOR.write(w+"/"+pos+" ")
SOR.close()

我想要做的是在定义的字典中每个单词之后的文本内添加 POS 标签，但结果是文件末尾的单词和 POS 标签的单独列表。

Answer 1

请记住，old_text 是一个字符串。因此，当您像在

中那样循环遍历它时

for i in old_text:
    if i in punkt:

你正在循环字符。我认为您打算改为循环遍历 old_text 行。如果是这种情况，您可以使用指定 read 和 write 模式的 with 语句打开文件。比如：

with open("SOR-1.txt", 'r+', encoding = 'utf-8') as f:
    old_text = f.readlines()
    for line in old_text:
        for punctuationMark in punct:
            if punctuationMark in line.strip('\n'):     #when you read the file, every line will be terminated with newline character `'\n'`
                #give more instructions