【问题标题】:How to log epochs in Gensim's LdaModel如何在 Gensim 的 LdaModel 中记录时代
【发布时间】:2019-04-15 17:34:08
【问题描述】:

我试图在我的 LdaModel 中显示学习进度,但我在网络上找到的每个示例都会引发异常:

l =  gensim.models.callbacks.CoherenceMetric(corpus=common_corpus, logger='shell')
lda = gensim.models.ldamodel.LdaModel(doc_term_matrix, num_topics=genres_count, id2word = common_corpus, passes=150, callbacks=[l])

投掷:

  File "<ipython-input-165-6ad0e2e8516c>", line 2, in <module>
    lda = gensim.models.ldamodel.LdaModel(doc_term_matrix, num_topics=genres_count, id2word = common_corpus, passes=150, callbacks=[l])

  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\gensim\models\ldamodel.py", line 371, in __init__
    self.update(corpus, chunks_as_numpy=use_numpy)

  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\gensim\models\ldamodel.py", line 750, in update
    current_metrics = callback.on_epoch_end(pass_)

  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\gensim\models\callbacks.py", line 288, in on_epoch_end
    value = metric.get_value(topics=topics, model=self.model, other_model=self.previous)

  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\gensim\models\callbacks.py", line 105, in get_value
    coherence=self.coherence, topn=self.topn

  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\gensim\models\coherencemodel.py", line 190, in __init__
    self.window_size = SLIDING_WINDOW_SIZES[self.coherence]

KeyError: None

此代码(找到here):

class EpochLogger(CallbackAny2Vec):
    '''Callback to log information about training'''

    def __init__(self):
        self.epoch = 0

    def on_epoch_begin(self, model):
        print("Epoch #{} start".format(self.epoch))

    def on_epoch_end(self, model):
        print("Epoch #{} end".format(self.epoch))
        self.epoch += 1

l = EpochLogger()
lda = gensim.models.ldamodel.LdaModel(doc_term_matrix, num_topics=genres_count, id2word = common_corpus, passes=150, callbacks=[l])

投掷:

Traceback (most recent call last):

  File "<ipython-input-167-e89e2bf41977>", line 1, in <module>
    lda = gensim.models.ldamodel.LdaModel(doc_term_matrix, num_topics=genres_count, id2word = common_corpus, passes=150, callbacks=[l])

  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\gensim\models\ldamodel.py", line 371, in __init__
    self.update(corpus, chunks_as_numpy=use_numpy)

  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\gensim\models\ldamodel.py", line 688, in update
    callback.set_model(self)

  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\gensim\models\callbacks.py", line 264, in set_model
    if any(metric.logger == "visdom" for metric in self.metrics):

  File "C:\Users\me\AppData\Local\Continuum\anaconda3\lib\site-packages\gensim\models\callbacks.py", line 264, in <genexpr>
    if any(metric.logger == "visdom" for metric in self.metrics):

AttributeError: 'EpochLogger' object has no attribute 'logger'

目前我最感兴趣的是监控学习进度(关注 ETA)。

设置回调的正确方法是什么?

【问题讨论】:

  • 您仍然必须在gensim 中手动编写回调的事实仍然让我感到困惑。

标签: python python-3.x gensim


【解决方案1】:

更新自:

l =  gensim.models.callbacks.CoherenceMetric(corpus=common_corpus, logger='shell')

到:

l =  gensim.models.callbacks.CoherenceMetric(corpus=common_corpus, coherence="u_mass", logger='shell')

“u_ma​​ss”只需要一个语料库。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2018-10-17
    • 2017-07-06
    • 1970-01-01
    • 2018-12-11
    • 2013-06-23
    • 1970-01-01
    相关资源
    最近更新 更多