导入 GoogleNews-vectors-negative300.bin

Question

我正在使用 gensim 编写代码，并且很难对代码中的 ValueError 进行故障排除。我终于能够压缩 GoogleNews-vectors-negative300.bin.gz 文件，这样我就可以在我的模型中实现它了。我也试过gzip，结果不成功。代码中的错误发生在最后一行。我想知道可以做些什么来修复错误。有什么解决方法吗？最后，有没有可以参考的网站？

非常感谢您的协助！

import gensim
from keras import backend
from keras.layers import Dense, Input, Lambda, LSTM, TimeDistributed
from keras.layers.merge import concatenate
from keras.layers.embeddings import Embedding
from keras.models import Mode

pretrained_embeddings_path = "GoogleNews-vectors-negative300.bin"
word2vec = 
gensim.models.KeyedVectors.load_word2vec_format(pretrained_embeddings_path, 
binary=True)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-23bd96c1d6ab> in <module>()
  1 pretrained_embeddings_path = "GoogleNews-vectors-negative300.bin"
----> 2 word2vec = 
gensim.models.KeyedVectors.load_word2vec_format(pretrained_embeddings_path, 
binary=True)

C:\Users\green\Anaconda3\envs\py35\lib\site-
packages\gensim\models\keyedvectors.py in load_word2vec_format(cls, fname, 
fvocab, binary, encoding, unicode_errors, limit, datatype)
244                             word.append(ch)
245                     word = utils.to_unicode(b''.join(word), 
encoding=encoding, errors=unicode_errors)
--> 246                     weights = fromstring(fin.read(binary_len), 
dtype=REAL)
247                     add_word(word, weights)
248             else:

ValueError: string size must be a multiple of element size

Answer 1

这对我有用。我加载了模型的一部分，而不是整个模型，因为它很大。

!pip install wget

import wget
url = 'https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz'
filename = wget.download(url)

f_in = gzip.open('GoogleNews-vectors-negative300.bin.gz', 'rb')
f_out = open('GoogleNews-vectors-negative300.bin', 'wb')
f_out.writelines(f_in)

import gensim
from gensim.models import Word2Vec, KeyedVectors
from sklearn.decomposition import PCA

model = KeyedVectors.load_word2vec_format('GoogleNews-vectors-negative300.bin', binary=True, limit=100000)

Answer 2

以下命令有效。

brew install wget

wget -c "https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz"

这会下载 GZIP 压缩文件，您可以使用以下方法解压缩：

gzip -d GoogleNews-vectors-negative300.bin.gz

然后您可以使用以下命令获取 wordVector。

from gensim import models

w = models.KeyedVectors.load_word2vec_format(
    '../GoogleNews-vectors-negative300.bin', binary=True)

Answer 3

试试这个 -

import gensim.downloader as api

wv = api.load('word2vec-google-news-300')

vec_king = wv['king']

另外，请访问此链接：https://radimrehurek.com/gensim/auto_examples/tutorials/run_word2vec.html#sphx-glr-auto-examples-tutorials-run-word2vec-py

Answer 4

你必须写完整的路径。

使用这条路径：

https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz