【问题标题】:Gensim LDA giving output of Topic IDs but probabilities are not adding up to 1Gensim LDA 提供主题 ID 的输出,但概率加起来不等于 1
【发布时间】:2019-07-16 07:40:55
【问题描述】:

我已经训练了 LDA 模型来聚类 100 个主题,据我所知,每个主题都应该以一定的概率输出,加起来等于 1。

但是当我运行这段代码时,我只得到了 2 个主题。

请帮忙。

text = "A blood cell, also called a hematocyte, is a cell produced by hematopoiesis and normally found in blood."

# transform text into the bag-of-words space
bow_vector = dictionary.doc2bow(tokenize(text))
lda_vector = lda_model[bow_vector]
print("LDA Output: ", lda_vector)
print("\nTop Keywords from highest prob Topic: ",lda_model.print_topic(max(lda_vector, key=lambda item: item[1])[0]))
print("\n\nAddition of all the probabilities from LDA output:",functools.reduce(lambda x,y:x+y,[i[1] for i in lda_vector]))

LDA 输出:[(64, 0.6952628), (69, 0.18223721)]

来自最高概率主题的热门关键字:0.042*“健康”+ 0.032*“医疗”+ 0.017*“患者”+ 0.016*“癌症”+ 0.015*“医院”+ 0.015*“说”+ 0.015*“治疗” " + 0.012*"医生" + 0.012*"护理" + 0.012*"药物"

LDA 输出的所有概率相加:0.8775

【问题讨论】:

    标签: machine-learning nlp gensim topic-modeling unsupervised-learning


    【解决方案1】:

    如果将LdaModel 的参数minimum_probability 设置为0,则总和将为1(或接近1,由于近似错误)。它控制过滤为文档返回的主题。

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2017-02-01
      • 2017-11-18
      • 2017-08-08
      • 2015-06-27
      • 2017-02-19
      • 2017-07-06
      • 1970-01-01
      • 2022-01-14
      相关资源
      最近更新 更多