定制情感分析：根据单词及其各自的分数对文档进行评分 - R NLP

Question

我正在尝试根据文档中出现的单词对文档进行评分。对于语料库中出现的每个单词，我有两种类型的分数。它本质上类似于情感分析，但具有定制的字典和相应的分数。谢谢

#documents to be scored on 2 dimensions: score1 and score2
documents <- data.frame(textID = 1:3, text = c("Hello everybody, pleased to see everyone together", " DHL postmen have faced difficulties this year", "divorcees have trouble finding jobs in this country"), scored1 = rep(NA,3), scored2=rep(NA,3) )

#first scoring dimension
scores1 <- as.matrix(data.frame(words = c("hello", "everybody", "pleased", "to" ,"see", "everyone","together", "DHL", "postmen", "have", "faced","difficulties","this", "year", "divorcees", "trouble", "finding", "jobs", "in", "country" ), scores = 1:20))

#second scoring dimension
scores2 <- as.matrix(data.frame(words = c("hello", "everybody", "pleased", "to" ,"see", "everyone","together", "DHL", "postmen", "have", "faced","difficulties","this", "year", "divorcees", "trouble", "finding", "jobs", "in", "country" ), scores = 10:29))

#the result should look like this, where each text receives a score that represents the sum of #individual word scores: 

#textID                                                  text      scored1 scored2
#1      1   Hello everybody, pleased to see everyone together       28        91
#2      2       DHL postmen have faced difficulties this year       77        140
#3      3 divorcees have trouble finding jobs in this country       128       200

Answer 1

这可以通过

实现

1. tidytext::unnest_token将文档转换成单个单词
2. dplyr::left_join这个词得分
3. dplyr::summarise 计算每个文档的分数

library(dplyr)
library(tidytext)

#documents to be scored on 2 dimensions: score1 and score2
documents <- data.frame(textID = 1:3, text = c("Hello everybody, pleased to see everyone together", " DHL postmen have faced difficulties this year", "divorcees have trouble finding jobs in this country"), scored1 = rep(NA,3), scored2=rep(NA,3) )

# 1. Get rid of as.matrix

#first scoring dimension
scores1 <- data.frame(words = c("hello", "everybody", "pleased", "to" ,"see", "everyone","together", "DHL", "postmen", "have", "faced","difficulties","this", "year", "divorcees", "trouble", "finding", "jobs", "in", "country" ), scores = 1:20)

#second scoring dimension
scores2 <- data.frame(words = c("hello", "everybody", "pleased", "to" ,"see", "everyone","together", "DHL", "postmen", "have", "faced","difficulties","this", "year", "divorcees", "trouble", "finding", "jobs", "in", "country" ), scores = 10:29)       

# 2. Make words lowercase
scores1 <- mutate(scores1, words = tolower(words))
scores2 <- mutate(scores2, words = tolower(words))

# 3. Compute scores
documents %>% 
  select(-scored1, -scored2) %>% 
  tidytext::unnest_tokens(text, output = words, drop = FALSE) %>% 
  left_join(scores1, by = c("words" = "words")) %>% 
  left_join(scores2, by = c("words" = "words"), suffix = c("1", "2")) %>% 
  group_by(textID, text) %>% 
  summarise(across(starts_with("scores"), sum, na.rm = TRUE)) %>% 
  rename(scored1 = scores1, scored2 = scores2) %>% 
  ungroup()
#> `summarise()` regrouping output by 'textID' (override with `.groups` argument)
#> # A tibble: 3 x 4
#>   textID text                                                  scored1 scored2
#>    <int> <chr>                                                   <int>   <int>
#> 1      1 "Hello everybody, pleased to see everyone together"        28      91
#> 2      2 " DHL postmen have faced difficulties this year"           77     140
#> 3      3 "divorcees have trouble finding jobs in this country"     128     200