【问题标题】:Keep only certain bigrams in document term matrix R在文档术语矩阵 R 中只保留某些二元组
【发布时间】:2017-02-09 00:52:03
【问题描述】:

问题:我怎样才能只在文档术语矩阵或我想保留的二元组(术语)列表中保留二元组?

我想将此应用于一个非常大的文档术语矩阵。我尝试将术语矩阵转换为矩阵,但向量大小超过 1000 Gb。

代码:

dd <- data.frame(
id = 10:13,
text = c("No wonderful, then, that ever",
       "So that in many cases such a ",
       "But there were still other and",
       "Not even at the rationale"), stringsAsFactors = F)

library(tm)
library(RWeka)

myReader <- readTabular(mapping = list(content = "text", id = "id"))

#create v corpus
tm <- VCorpus(DataframeSource(dd), readerControl = list(reader = myReader))

#n-gram tokenizer
Tokenizer <- function(x) NGramTokenizer(x, Weka_control(min = 2, max = 2))

#create document term matrix using Tokenizer
       dtm <- TermDocumentMatrix(tm, control = list(tokenize = Tokenizer))
       inspect(dtm)

输出:

                             Docs
            Terms           10 11 12 13
            at the          0  0  0  1
            but there       0  0  1  0
            cases such      0  1  0  0
            even at         0  0  0  1
            in many         0  1  0  0
            many cases      0  1  0  0
            no wonderful    1  0  0  0
            not even        0  0  0  1
            other and       0  0  1  0
            so that         0  1  0  0
            still other     0  0  1  0
            such a          0  1  0  0
            that ever       1  0  0  0
            that in         0  1  0  0
            the rationale   0  0  0  1
            then that       1  0  0  0
            there were      0  0  1  0
            were still      0  0  1  0
            wonderful then  1  0  0  0

【问题讨论】:

    标签: r


    【解决方案1】:

    因为它是 DTM,所以认为它更复杂。

    问题解决了:

        d_sel <- dtm[c('no wonderful', 'there were'),]
        inspect(d_sel)
    
                    Docs
                    Terms          10 11 12 13
                    no wonderful    1  0  0  0
                    there were      0  0  1  0
    

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2015-05-19
      • 1970-01-01
      • 2015-05-05
      • 1970-01-01
      • 1970-01-01
      • 2018-11-26
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多