如何使用 nltk 计算文本中存在的单词的频率

Question

我有一个 Python 脚本，它可以读取文本并应用预处理函数来进行分析。
问题是我想统计单词出现的频率但是系统崩溃并显示如下错误。

文件“F:\AIenv\textAnalysis\setup.py”，第 208 行，在 tag_and_save 中 file.write(word+"/"+tag+" (frequency="+str(freq_tagged_data[word])+")\n") TypeError: tuple 索引必须是整数或切片，而不是 str

我正在尝试计算频率，然后写信给text file。

def get_freq(tagged):
    freqs = FreqDist(tagged)
    for word, freq in freqs.items():
        print(word, freq)
    result = word,freq
    return result

def tag_and_save(tagger,text,path):
    clt = clean_text(text)
    tagged_data = tagger.tag(clt)

    freq_tagged_data = get_freq(tagged_data)
    file = open(path,"w",encoding = "UTF8")
    for word,tag in tagged_data:
        file.write(word+"/"+tag+" (frequency="+str(freq_tagged_data[word])+")\n")
    file.close()

我希望输出是这样的：

('*****/DTNN') 3

---

基于

的回答

我将函数 get_freq() 更改为：

def get_freq(tagged):
    freq_dist = {}
    freqs = FreqDist(tagged)
    freq_dist = [(word, freq) for word ,freq in freqs.items()]
    return freq_dist

但现在它显示以下错误：

文件“F:\AIenv\textAnalysis\setup.py”，第 217 行，在 tag_and_save file.write(word+"/"+tag+" (frequency="+str(freq_tagged_data[word])+")\n")

类型错误：列表

索引必须是整数或切片，而不是 str

如何解决这个错误，我该怎么办？

Answer 1

也许这会有所帮助。

import nltk
text = "An an valley indeed so no wonder future nature vanity. Debating all she mistaken indulged believed provided declared. He many kept on draw lain song as same. Whether at dearest certain spirits is entered in to. Rich fine bred real use too many good. She compliment unaffected expression favourable any. Unknown chiefly showing to conduct no."
tokens = [t for t in text.split()]
freqs = nltk.FreqDist(tokens)
blah_list = [(k, v) for k, v in freqs.items()]
print(blah_list)

这个sn-p统计词频。

编辑：代码现在可以工作了。