Python - 以百分比计算词频

Question

我有一个文本，我计算了单词的数量和单词的频率。现在我必须按百分比显示前 7 个。我不知道该怎么做。我知道如何计算百分比、部分/整体，但不知道如何编写代码。我已经完成了下面的排序值。

def word_frequency():
"""
Function for word frequency
"""
d = dict()

with open(TEXT, "r") as f:
    for line in f:
        words = line.split()
        for w in words:
            if w in d:
                d[w] += 1
            else:
                d[w] = 1

    dict_list = sorted(d.items(), key = itemgetter(1), reverse = True)
    print(dict_list[0:7])

这给了我这个列表：

[('the', 12), ('to', 8), ('of', 6), ('and', 5), ('a', 4), ('in', 4), ('Phil', 3)]

但是如何用百分比而不是值来计算和呈现它们呢？ 文字字数为199。

问候

编辑：新修订的代码

def word_frequency():
"""
Function for word frequency
"""
d = dict()

with open(TEXT, "r") as f:
    for line in f:
        words = line.split()
        for w in words:
            if w in d:
                d[w] += round(1/1.99, 1)
            else:
                d[w] = round(1/1.99, 1)

    dict_list = sorted(d.items(), key = itemgetter(1), reverse = True)
    print(dict_list[0:7])

给我这份清单：

[('the', 6.0), ('to', 4.0), ('of', 3.0), ('and', 2.5), ('a', 2.0), ('in', 2.0), ('Phil', 1.5)]

我现在有百分比，但有没有办法以更好的方式呈现它？ 喜欢：

the 6%
to 4%
of 3%
and 2.5%
a 2%
in 2%
Phil 1.5%

Answer 1

您也可以使用collections 中的Counter 来计算单词的频率。

from operator import itemgetter
from collections import Counter

def most_common(instances):
    """Returns a list of (instance, count) sorted in total order and then from most to least common"""
    return sorted(sorted(Counter(instances).items(), key=itemgetter(0)), key=itemgetter(1), reverse=True)

利用most_common 函数，您可以像您说的那样“计算百分比，部分/整体”。您可以通过遍历单词及其频率并将其除以单词总数来完成。

# words = list of strings
frequencies = most_common(words)
percentages = [(instance, count / len(words)) for instance, count in frequencies]

根据您的用例，re.findall(r"\w+", text) 可能不是提取单词的最佳方法。

要获得前 7 个单词，您可以通过执行 percentages[:7] 来分割 percentages。

import re

text = "Alice opened the door and found that it led into a small passage, not much larger than a rat-hole: she knelt down and looked along the passage into the loveliest garden you ever saw."

words = re.findall(r"\w+", text)
frequencies = most_common(words)
percentages = [(instance, count / len(words)) for instance, count in frequencies]

for word, percentage in percentages[:7]:
    print("%s %.2f%%" % (word, percentage * 100))

哪些输出：

the 8.57%
a 5.71%
and 5.71%
into 5.71%
passage 5.71%
Alice 2.86%
along 2.86%

如果你想在不同的大小写相同的单词，算作相同。然后你可以在调用most_common之前对所有单词进行规范化。

import unicodedata

def normalize_caseless(text):
    return unicodedata.normalize("NFKD", text.casefold())

然后：

words = ...

变成：

words = list(map(normalize_caseless, ...))

然后是一个在不同大小写中包含相同单词的字符串，如下所示：

text = "Hello Test test TEST test TeSt"

结果：

test 83.33%
hello 16.67%

代替：

test 33.33%
Hello 16.67%
TEST 16.67%
TeSt 16.67%
Test 16.67%

Answer 2

您可以枚举字典中的项目

for k, v in dict_list.items():
    percent = str(v) + ' %'
    result = k + ' ' + percent
    print(result)