NLTK Python中的词义消歧

Question

我是 NLTK Python 的新手，我正在寻找一些可以进行词义消歧的示例应用程序。我在搜索结果中有很多算法，但没有示例应用程序。我只是想传一句话，想通过参考wordnet库来了解每个单词的意思。 谢谢

我在 PERL 中找到了一个类似的模块。 http://marimba.d.umn.edu/allwords/allwords.html NLTK Python 中是否存在这样的模块？

Answer 1

是的，可以使用 NLTK 中的 wordnet 模块。 您帖子中提到的工具中使用的相似度测量也存在于NLTK wordnet模块中。

Answer 2

参考http://jaganadhg.freeflux.net/blog/archive/2010/10/16/wordnet-sense-similarity-with-nltk-some-basics.html

Answer 3

NLTK 有访问 Wordnet 的 api。 Wordnet 将单词作为同义词集。这会给你一些关于这个词、它的上位词、下位词、词根等的信息。

“使用 NLTK 2.0 Cookbook 进行 Python 文本处理”是一本让您开始了解 NLTK 的各种功能的好书。它易于阅读、理解和实施。

此外，您还可以查看其他有关使用维基百科进行词义消歧的论文（NLTK 领域之外）。

Answer 4

是的，事实上，NLTK 团队编写了a book，其中有多个关于分类的章节，并且明确涵盖了how to use WordNet。您也可以从 Safari 购买实体版图书。

仅供参考：NLTK 由自然语言编程学者编写，用于他们的编程入门课程。

Answer 5

作为对 OP 请求的实际回答，这里是几个 WSD 方法的 python 实现，它以 NLTK 的 synset(s) 的形式返回意义，https://github.com/alvations/pywsd

包括

• Lesk算法（包括原始Lesk、改编的Lesk和简单的Lesk）
• 基线算法（随机感觉、第一感觉、最常见感觉）

可以这样使用：

#!/usr/bin/env python -*- coding: utf-8 -*-

bank_sents = ['I went to the bank to deposit my money',
'The river bank was full of dead fishes']

plant_sents = ['The workers at the industrial plant were overworked',
'The plant was no longer bearing flowers']

print "======== TESTING simple_lesk ===========\n"
from lesk import simple_lesk
print "#TESTING simple_lesk() ..."
print "Context:", bank_sents[0]
answer = simple_lesk(bank_sents[0],'bank')
print "Sense:", answer
print "Definition:",answer.definition
print

print "#TESTING simple_lesk() with POS ..."
print "Context:", bank_sents[1]
answer = simple_lesk(bank_sents[1],'bank','n')
print "Sense:", answer
print "Definition:",answer.definition
print

print "#TESTING simple_lesk() with POS and stems ..."
print "Context:", plant_sents[0]
answer = simple_lesk(plant_sents[0],'plant','n', True)
print "Sense:", answer
print "Definition:",answer.definition
print

print "======== TESTING baseline ===========\n"
from baseline import random_sense, first_sense
from baseline import max_lemma_count as most_frequent_sense

print "#TESTING random_sense() ..."
print "Context:", bank_sents[0]
answer = random_sense('bank')
print "Sense:", answer
print "Definition:",answer.definition
print

print "#TESTING first_sense() ..."
print "Context:", bank_sents[0]
answer = first_sense('bank')
print "Sense:", answer
print "Definition:",answer.definition
print

print "#TESTING most_frequent_sense() ..."
print "Context:", bank_sents[0]
answer = most_frequent_sense('bank')
print "Sense:", answer
print "Definition:",answer.definition
print

[出]：

======== TESTING simple_lesk ===========

#TESTING simple_lesk() ...
Context: I went to the bank to deposit my money
Sense: Synset('depository_financial_institution.n.01')
Definition: a financial institution that accepts deposits and channels the money into lending activities

#TESTING simple_lesk() with POS ...
Context: The river bank was full of dead fishes
Sense: Synset('bank.n.01')
Definition: sloping land (especially the slope beside a body of water)

#TESTING simple_lesk() with POS and stems ...
Context: The workers at the industrial plant were overworked
Sense: Synset('plant.n.01')
Definition: buildings for carrying on industrial labor

======== TESTING baseline ===========
#TESTING random_sense() ...
Context: I went to the bank to deposit my money
Sense: Synset('deposit.v.02')
Definition: put into a bank account

#TESTING first_sense() ...
Context: I went to the bank to deposit my money
Sense: Synset('bank.n.01')
Definition: sloping land (especially the slope beside a body of water)

#TESTING most_frequent_sense() ...
Context: I went to the bank to deposit my money
Sense: Synset('bank.n.01')
Definition: sloping land (especially the slope beside a body of water)

Answer 6

最近，wsd.py 模块中的部分pywsd 代码已移植到NLTK' 的前沿版本中，试试：

>>> from nltk.wsd import lesk
>>> sent = 'I went to the bank to deposit my money'
>>> ambiguous = 'bank'
>>> lesk(sent, ambiguous)
Synset('bank.v.04')
>>> lesk(sent, ambiguous).definition()
u'act as the banker in a game or in gambling'

为了获得更好的 WSD 性能，请使用 pywsd 库而不是 NLTK 模块。一般来说，来自pywsd 的simple_lesk() 比来自NLTK 的lesk 好。有空我会尽量更新NLTK模块。

---

在回应 Chris Spencer 的评论时，请注意 Lesk 算法的局限性。我只是给出算法的准确实现。这不是灵丹妙药，http://en.wikipedia.org/wiki/Lesk_algorithm

另外请注意，虽然：

lesk("My cat likes to eat mice.", "cat", "n")

不给你正确答案，可以用pywsd实现max_similarity()：

>>> from pywsd.similarity import max_similiarity
>>> max_similarity('my cat likes to eat mice', 'cat', 'wup', pos='n').definition 
'feline mammal usually having thick soft fur and no ability to roar: domestic cats; wildcats'
>>> max_similarity('my cat likes to eat mice', 'cat', 'lin', pos='n').definition 
'feline mammal usually having thick soft fur and no ability to roar: domestic cats; wildcats'

@Chris，如果你想要一个 python setup.py ，请礼貌地请求，我会写它...