运行 gensim Word2Vec 时遇到问题答案

【问题标题】：Trouble running gensim Word2Vec运行 gensim Word2Vec 时遇到问题
【发布时间】：2018-03-12 06:34:35
【问题描述】：

我正在尝试使用 gensim 库在我自己的数据集上训练词嵌入 (word2vec)。

model = Word2Vec(sentences=alp[:20],size=100, window=6, min_count=5) 其中 alp 是我的语料库中包含单个句子标记的列表列表。

每当我尝试训练 w2v 模型时，我都会收到以下错误。请帮助。

`Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.5/dist-packages/gensim/models/word2vec.py", line 867, in worker_loop
    tally, raw_tally = self._do_train_job(sentences, alpha, (work, neu1))
  File "/usr/local/lib/python3.5/dist-packages/gensim/models/word2vec.py", line 785, in _do_train_job
    tally += train_batch_cbow(self, sentences, alpha, work, neu1, 
self.compute_loss)
  File "gensim/models/word2vec_inner.pyx", line 458, in gensim.models.word2vec_inner.train_batch_cbow (./gensim/models/word2vec_inner.c:5642)
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()`

`Exception in thread Thread-1:
    Traceback (most recent call last):
      File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.5/threading.py", line 862, in run
        self._target(*self._args, **self._kwargs)
      File "/usr/local/lib/python3.5/dist-packages/gensim/models/word2vec.py", line 867, in worker_loop
        tally, raw_tally = self._do_train_job(sentences, alpha, (work, neu1))
      File "/usr/local/lib/python3.5/dist-packages/gensim/models/word2vec.py", line 785, in _do_train_job
        tally += train_batch_cbow(self, sentences, alpha, work, neu1, self.compute_loss)
      File "gensim/models/word2vec_inner.pyx", line 458, in gensim.models.word2vec_inner.train_batch_cbow (./gensim/models/word2vec_inner.c:5642)
    ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()`

【问题讨论】：

标签： word2vec gensim

【解决方案1】：

问题已通过将 alp 类型转换为列表列表来解决。

【讨论】：

对于任何未来的访问者，我在传入 numpy 稀疏矩阵（而不是 gensim 语料库）时遇到了类似的错误。
我也遇到了同样的问题。我将列表列表转换为 numpy 数组以进行一些预处理。将其转换回列表并且它起作用了。

【解决方案2】：

上面的代码非常适合我。你能验证alp[:20]的类型吗。
工作代码（在gensim version 3.4.0 测试）：

from gensim.models.word2vec import Word2Vec  
model = Word2Vec(sentences=alp[0:20],size=100,window=6,min_count=5)

alp 如下所示：

    alp = [['this','is','first','sentence'],
          ['this','is','second','sentence'],
          [..],  
          [..],  
          [..]]

【讨论】：

alp 是一个列表列表。 alp[0] = ['i', 'think', 'we', 'could', 'import', 'the', 'old', 'comment', 'via', 'rsync', '', 'but', 'from', 'there', 'we', 'need', 'to', 'go', 'via', 'email', '', 'i', 'think', 'it', 'be', 'easier', 'than', 'cach', 'the', 'status', 'on', 'each', 'bug', 'and', 'than', 'import', 'bite', 'here', 'and', 'there', '__eou__', '__eot__', 'it', 'would', 'be', 'veri', 'easi', 'to', 'keep', 'a', 'hash', 'db', 'of', 'messageid', '__eou__', 'sound', 'good', '__eou__', '__eot__', 'ok', '__eou__', 'perhap', 'we', 'can',.... 'apt_preferec', '__eou__'...] 与 alp[1] 等相同。