将列表中的单词和代码识别为 0 或 1，将不在列表代码中的单词识别为 1

Question

注意：关于此的更新/新问题始于 =====================

原帖：我正在处理儿童所说的话语和陈述。从每个话语中，如果语句中的一个或多个单词与多个“核心”单词（可能是 300 个单词）的预定义列表匹配，那么我想在“核心”中输入“1”（如果没有，则输入“0”进入“核心”）。

如果语句中有一个或多个词不是核心词，那么我想在'Fringe'中输入'1'（如果只有核心词而没有额外的，那么在'中输入'0'边缘'）。

基本上，现在我只有话语，从这些话语中，我需要确定是否有任何单词与核心单词匹配，如果有任何额外的单词，则将它们识别为边缘。这是我的数据的 sn-p。

  Core Fringe        Utterance
1   NA     NA            small
2   NA     NA            small
3   NA     NA  where's his bed
4   NA     NA  there's his bed
5   NA     NA  there's his bed
6   NA     NA is that a pillow

感谢来自原始帖子的 rjen，下面的代码将使我能够识别核心词和边缘词——假设我知道边缘词（这是我最初预期的）。然而，挑战已经改变，所以现在任何不是核心的东西都将被视为边缘。所以基本上我需要保持从列表中检测单词并将其定义为核心的能力，但我还需要能够搜索话语，如果话语中有任何不是核心的单词，将其识别为“1”因为我不会列出边缘词。

library(dplyr)
library(stringr)

coreWords <- c('small', 'bed')
fringeWords <- c('head', 'his')

CFdataNew <- CFdata %>%
  mutate(Core = + str_detect(Utterance, str_c(coreWords, collapse = '|')),
         Fringe = + str_detect(Utterance, str_c(fringeWords, collapse = '|')))

dput() 代码是：

    structure(list(Utterance = c("small", "small", "where's his bed", "there's his bed", "there's his bed", "is that a pillow", "what is that on his head", "hey he has his arm stuck here", "there there's it", "now you're gonna go night_night", "and that's the thing you can turn on", "yeah where's the music+box"), Core = c(NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA), Fringe = c(NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA)), class = "data.frame", row.names = c(NA, -12L))

====================

更新 rjen 的响应对示例数据非常有效。然而，我现在有一个核心单词和话语的“真实”列表，并不是所有额外的单词都被认为是边缘单词，即使它们不是，也有一些单词被识别为核心单词。我将这些词用双引号括起来，看看这是否有助于解决问题，以便核心词是明确的，但它没有。

coreWords <-c("I", "no", "nah", "nope", "yes", "yeah", "yea", "mmhmm", "yah",
              "ya", "un-huh", "uhhuh", "my", "the", "want", "is", "it", "that",
              "a", "go", "mine", "you", "what", "on", "in", "here", "more",
              "out", "off", "some", "help", "all done", "finished")

Df1 <- df %>%
  mutate(id = row_number()) %>%
  separate_rows(Utterance, sep = ' ') %>%
  mutate(Core = + str_detect(Utterance, str_c(coreWords, collapse = '|')),
         Fringe = + !Core) %>%
  group_by(id) %>%
  mutate(Core = + (sum(Core) > 0),
         Fringe = + (sum(Fringe) > 0)) %>%
  slice(1) %>%
  select(-Utterance) %>%
  left_join(df) %>% 
  ungroup() %>%
  select(Utterance, Core, Fringe, id)

上面脚本的输出和更长的核心词列表看起来像这样。

# A tibble: 98 x 4
   Utterance                        Core Fringe    id
   <chr>                           <int>  <int> <int>
 1 a baby                              1      0     1
 2 small                               1      0     2
 3 yes                                 1      0     3
 4 where's his bed                     1      1     4
 5 there's his bed                     1      1     5
 6 where's his pillow                  1      1     6
 7 what is that on his head            1      0     7
 8 hey he has his arm stuck here       1      1     8
 9 there there's it                    1      0     9
10 now you're gonna go night-night     1      1    10
# ... with 88 more rows

例如，在第 1 行中，“a”是核心词，因此“1”代表核心是正确的。但是，“婴儿”应该被选为边缘，所以边缘应该是“1”，而不是“0”。第 7 行和第 9 行也有应该被识别为边缘但不是的单词。

此外，如果话语中包含核心词的一部分，它似乎会被计算在内。例如，“small”被识别为核心词，即使它不是（但“all done”是一个核心词）。 “他的床在哪里”被识别为核心和边缘，尽管没有一个词是核心。非常感谢您对正在发生的事情以及如何纠正它提出任何建议。

Answer 1

这样做的一个小技巧是用空字符串"" 替换（gsub()）话语中的所有核心词。然后检查字符串 (nchar()) 的长度是否仍然大于零。如果大于零，则表示话语中有非核心词。通过在替换核心词后将trimws() 应用于字符串，我们确保没有多余的空格会被视为字符。

这是代码本身。

nchar(trimws(gsub(str_c(coreWords, collapse = '|'), "", CFdata$Utterance))) > 0
#>  [1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE

这是一个分步版本，让您检查正在发生的事情。

CFdata %>%
  mutate(
    core_words_removed = trimws(gsub(str_c(coreWords, collapse = '|'), "", Utterance)),
    no_core_words_included = as.numeric(nchar(core_words_removed) > 0)
  )
#>                               Utterance                   core_words_removed
#> 1                                 small                                     
#> 2                                 small                                     
#> 3                       where's his bed                          where's his
#> 4                       there's his bed                          there's his
#> 5                       there's his bed                          there's his
#> 6                      is that a pillow                     is that a pillow
#> 7              what is that on his head             what is that on his head
#> 8         hey he has his arm stuck here        hey he has his arm stuck here
#> 9                      there there's it                     there there's it
#> 10      now you're gonna go night_night      now you're gonna go night_night
#> 11 and that's the thing you can turn on and that's the thing you can turn on
#> 12           yeah where's the music+box           yeah where's the music+box
#>    no_core_words_included
#> 1                       0
#> 2                       0
#> 3                       1
#> 4                       1
#> 5                       1
#> 6                       1
#> 7                       1
#> 8                       1
#> 9                       1
#> 10                      1
#> 11                      1
#> 12                      1

这是一步，并集成到您的原始代码 sn-p 中。

CFdataNew <-
  CFdata %>%
  mutate(
    Core = as.numeric(str_detect(Utterance, str_c(coreWords, collapse = '|'))),
    no_core_words_included = as.numeric(nchar(gsub(
      str_c(coreWords, collapse = '|'), "", Utterance
    )) > 0),
    Fringe = as.numeric(str_detect(
      Utterance, str_c(fringeWords, collapse = '|')
    ))
  )

Answer 2

使用 separate_rows() 的 tidyverse 选项

library(dplyr)
library(stringr)
library(tidyr)

coreWords <- c('small', 'bed')

df1 <- df %>%
  transmute(id = row_number(),
            Utterance = Utterance)

df %>%
  mutate(id = row_number()) %>%
  separate_rows(Utterance, sep = ' ') %>%
  mutate(Core = + str_detect(Utterance, str_c(coreWords, collapse = '|')),
         Fringe = + !Core) %>%
  group_by(id) %>%
  mutate(Core = + (sum(Core) > 0),
         Fringe = + (sum(Fringe) > 0)) %>%
  slice(1) %>%
  select(-Utterance) %>%
  left_join(df1) %>%
  ungroup() %>%
  select(Utterance, Core, Fringe, -id)

# # A tibble: 12 x 3
#    Utterance                             Core Fringe
#    <chr>                                <int>  <int>
#  1 small                                    1      0
#  2 small                                    1      0
#  3 where's his bed                          1      1
#  4 there's his bed                          1      1
#  5 there's his bed                          1      1
#  6 is that a pillow                         0      1
#  7 what is that on his head                 0      1
#  8 hey he has his arm stuck here            0      1
#  9 there there's it                         0      1
# 10 now you're gonna go night_night          0      1
# 11 and that's the thing you can turn on     0      1
# 12 yeah where's the music+box               0      1