如何让基本的倒排索引程序更 Pythonic

Question

我有如下的倒排索引代码。但是我对它不太满意，想知道如何使它更紧凑和 pythonic

class invertedIndex(object):


  def __init__(self,docs):
     self.docs,self.termList,self.docLists=docs,[],[]

     for index,doc in enumerate(docs):

        for term in doc.split(" "):
            if term in self.termList:
                i=self.termList.index(term)
                if index not in self.docLists[i]:
                    self.docLists[i].append(index)

            else:
                self.termList.append(term)
                self.docLists.append([index])  

  def search(self,term):
        try:
            i=self.termList.index(term)
            return self.docLists[i]
        except:
            return "No results"





docs=["new home sales top forecasts june june june",
                     "home sales rise in july june",
                     "increase in home sales in july",
                     "july new home sales rise"]

i=invertedIndex(docs)
print invertedIndex.search("sales")

Answer 1

将文档索引存储在 Python set 中，并使用 dict 来引用每个术语的“文档集”。

from collections import defaultdict

class invertedIndex(object):

  def __init__(self,docs):
      self.docSets = defaultdict(set)
      for index, doc in enumerate(docs):
          for term in doc.split():
              self.docSets[term].add(index)

  def search(self,term):
        return self.docSets[term]

docs=["new home sales top forecasts june june june",
                     "home sales rise in july june",
                     "increase in home sales in july",
                     "july new home sales rise"]

i=invertedIndex(docs)
print i.search("sales") # outputs: set([0, 1, 2, 3])

set 有点像列表，但它是无序的，不能包含重复的条目。

defaultdict 基本上是一个dict，当没有可用数据（在本例中为空集）时，它具有默认类型。

Answer 2

此解决方案与@Peter Gibson 的解决方案几乎相同。在这个版本中，索引是数据，不涉及委托docSets对象。这使得代码更短更清晰。

代码还保留了文档的原始顺序...这是一个错误，我更喜欢 Peter 的 set() 实现。

另请注意，对不存在的术语（如ix['garbage']）的引用会隐式修改索引。如果唯一的API是search，这很好，但这种情况值得注意。

来源

class InvertedIndex(dict):
    def __init__(self, docs):
        self.docs = docs

        for doc_index,doc in enumerate(docs):
            for term in doc.split(" "):
                self[term].append(doc_index)

    def __missing__(self, term):
        # operate like defaultdict(list)
        self[term] = []
        return self[term]

    def search(self, term):
        return self.get(term) or 'No results'


docs=["new home sales top forecasts june june june",
      "home sales rise in july june",
      "increase in home sales in july",
      "july new home sales rise",
      'beer',
      ]

ix = InvertedIndex(docs)
print ix.__dict__
print
print 'sales:',ix.search("sales")
print 'whiskey:', ix.search('whiskey')
print 'beer:', ix.search('beer')

print '\nTEST OF KEY SETTING'
print ix['garbage']
print 'garbage' in ix
print ix.search('garbage')

输出

{'docs': ['new home sales top forecasts june june june', 'home sales rise in july june', 'increase in home sales in july', 'july new home sales rise', 'beer']}

sales: [0, 1, 2, 3]
whiskey: No results
beer: [4]

TEST OF KEY SETTING
[]
True
No results