使用 R 从包含关键字的 PDF 中提取段落答案

【问题标题】：Extract the paragraphs from a PDF that contain a keyword using R使用 R 从包含关键字的 PDF 中提取段落
【发布时间】：2020-09-16 09:39:24
【问题描述】：

我需要从 pdf 文件中提取包含关键字的段落。尝试了各种代码，但没有得到任何东西。我从用户@Tyler Rinker (Extract before and after lines based on keyword in Pdf using R programming) 那里看到了这段代码，但它提取了关键字所在的行，之前和之后。

library(textreadr)
library(tidyverse)

loc <- function(var, regex, n = 1, ignore.case = TRUE){
    locs <- grep(regex, var, ignore.case = ignore.case)
    out <- sort(unique(c(locs - 1, locs, locs + 1)))
    out <- out[out > 0]
    out[out <= length(var)]
}

doc <- 'https://www.in.kpmg.com/pdf/Indian%20Pharma%20Outlook.pdf' %>%
    read_pdf() %>%
    slice(loc(text, 'cancer'))

但是，我需要获取段落并将每个段落连续存储在我的数据库中。你能帮帮我吗？

【问题讨论】：

我认为问题是在文档中，段落没有被任何特别的东西分隔。对于您想要做的工作，您必须能够将每页上的文本分成段落。例如，如果每个段落都以换行标记 \n 结尾并且该标记仅在段落末尾使用，这将起作用。但是，这里不是这样。
是的，每个句子都会以一个新的行标签\n结尾，但是我不知道如何获取整个段落。你知道@DaveArmstrong 是怎么做到的吗？
我认为这就是问题所在。没有什么一致的可以将一段与另一段分开，所以如果没有一些人工干预，我认为这是不可能的。也许其他人会有建议。

标签： r pdf text-mining

【解决方案1】：

段落中的文本行都会很长，除非它是段落的最后一行。我们可以计算每一行中的字符并做一个直方图来显示：

library(textreadr)

doc <- read_pdf('https://www.in.kpmg.com/pdf/Indian%20Pharma%20Outlook.pdf')

hist(nchar(doc$text), 20)

因此，少于大约 75 个字符的任何内容要么不在段落中，要么不在段落末尾。因此，我们可以在较短的行上添加换行符，将所有行粘贴在一起，然后在换行符处拆分：


doc$text[nchar(doc$text) < 75] <- paste0(doc$text[nchar(doc$text) < 75], "\n")
txt <- paste(doc$text, collapse = " ")
txt <- strsplit(txt, "\n")[[1]]

所以现在我们可以做我们的正则表达式并找到带有关键字的段落：

grep("cancer", txt, value = TRUE)
#> [1] " Ranjit Shahani applauds the National Pharmaceuticals Policy's proposal of public/private partnerships (PPPs) to tackle life-threatening diseases such as cancer and HIV/AIDS, but stresses that, in order for them to work, they should be voluntary, and the government should exempt all life-saving drugs from import duties and other taxes such as excise duty and VAT. He is, however, critical about a proposal for mandatory price negotiation of newly patented drugs. He feels this will erode India's credibility in implementing the Patent Act in © 2006 KPMG International. KPMG International is a Swiss cooperative that serves as a coordinating entity for a network of independent firms operating under the KPMG name. KPMG International provides no services to clients. Each member firm of KPMG International is a legally distinct and separate entity and each describes itself as such. All rights reserved. Collaboration for Growth 24"                                                                                                   
#> [2] " a fair and transparent manner. To deal with diabetes, medicines are not the only answer; awareness about the need for lifestyle changes needs to be increased, he adds. While industry leaders have long called for the development of PPPs for the provision of health care in India, particularly in rural areas, such initiatives are currently totally unexplored. However, the government's 2006 draft National Pharmaceuticals Policy proposes the introduction of PPPs with drug manufacturers and hospitals as a way of vastly increasing the availability of medicines to treat life-threatening diseases. It notes, for example, that while an average estimate of the value of drugs to treat the country's cancer patients is $1.11 billion, the market is in fact worth only $33.5 million. “The big gap indicates the near non-accessibility of the medicines to a vast majority of the affected population, mainly because of the high cost of these medicines,” says the Policy, which also calls for tax and excise exemptions for anti-cancer drugs."
#> [3] " 50.1 percent of Aventis Pharma is held by European drug major Sanofi-Aventis and, in early April 2006, it was reported that UB Holdings had sold its 10 percent holding in the firm to Variegate Trading, a UB subsidiary. The firm's major products are in the anti-infective, anti-inflammatory, cancer, diabetes and allergy market segments and, for the year ended December 31, 2005, it reported net sales (excluding excise duty) up 9.9 percent to $181.1 million, with domestic sales up 9.1 percent at $129.8 million and exports increasing 12 percent to $51.2 million. Sales were led by 83 percent annual growth for the diabetes treatment Lantus (insulin glargine), followed by the rabies vaccine Rabipur (+22 percent), the diabetes drug Amaryl (glimepiride) and epilepsy treatment Frisium (clobazam), both up 18 percent, the angiotensin-coverting enzyme inhibitor Cardace (ramipril +15 percent), Clexane (enoxaparin), an anticoagulant, growing 14 percent and Targocid (teicoplanin), an antibiotic, whose sales advanced 8 percent."

^{由reprex package (v0.3.0) 于 2020 年 9 月 16 日创建}

【讨论】：