正则表达式过滤所有标签文章

Question

我正在尝试从以下发布的文章中获取所有内容： https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-01975-8 我发现信息在标签里

<article><div...><..> information.... <></article>

我正在尝试这样的事情：

art_sections<-regexpr("<article (.*)?>(.[0-9]*)</article>",thepage)

但我无法检索信息..

请问你知道我该如何解决它。

Answer 1

这不是一个正则表达式问题，而是关于使用 R 使用库进行网络抓取的问题，例如rvest.

这里是一些示例代码，下面是一些帮助您入门的链接：

#Loading the rvest package
library('rvest')
#Specifying the url for desired website to be scraped
url <- 'https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-01975-8'
#Reading the HTML code from the website
webpage <- read_html(url)
article_html <- html_nodes(webpage,'article')
#Converting the ranking data to text
html_text(article_html)

• https://www.analyticsvidhya.com/blog/2017/03/beginners-guide-on-web-scraping-in-r-using-rvest-with-hands-on-knowledge/
• https://rvest.tidyverse.org/articles/harvesting-the-web.html
• https://bradleyboehmke.github.io/2015/12/scraping-html-text.html

最后，要清理您的文本，请查看 stringr 即

library(stringr)
str_replace_all(x, "[\r\n]" , "")

Answer 2

尝试使用rvest 包提取文章中的所有文本（仅文本）。但是，所有 HTML 标记（包括链接、图像等）都会被删除。

# install.packages("rvest")

library(rvest)
url <- "https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-01975-8"
article <- url %>% 
  read_html %>%
  html_node(css = 'article') %>%
  html_text

article

# Method\n    \n        \n            \n                Open Access\n            \n        \n    \n    \n\n                            Published: 19 March 2020\n                        VALOR2: characterization of large-scale structural variants using linked-reads\n                        Fatih Karaoğlanoğlu1 na1, Camir Ricketts2,4, Ezgi Ebren1, Marzieh Eslami Rasekh3, Iman Hajirasouliha4,5 & Can Alkan1,6 \n                            \n    Genome Biology\n\n                            volume 21, Article number: 72 (2020)\n            Cite this article\n                        \n                        \n    \n        \n            \n                        576 Accesses\n                    \n                \n                \n                    \n                        1 Citations\n                    \n                \n                \n                    \n                        \n                            12 Altmetric\n                        \n                    \n                \n                \n                    Metrics details\n                \n            \n    \n\n                        \n                        \n                        \n                    \n\n                    AbstractMost existing methods for structural variant detection focus on discovery and genotyping of deletions, insertions, and mobile elements. Detection of balanced structural variants with no gain or loss of genomic segments, for example, inversions and translocations, is a particularly challenging task. Furthermore, there are very few algorithms to predict the insertion locus of large interspersed segmental duplications and characterize translocations. Here, we propose novel algorithms to characterize large interspersed segmental duplications, inversions, deletions, and translocations using linked-read sequencing data. We redesign our earlier algorithm, VALOR, and implement our new algorithms in a new software package, called VALOR2.BackgroundAlterations of DNA content and organization larger than 50 bp, commonly referred to as genomic structural variations (SVs) [1], are among the major drivers of evolution [2, 3] and diseases of genomic origin [4]. Despite decades of research, they remain difficult to accurately characterize contributing to our lack of full understanding of the etiology of complex diseases, termed missing heritability [5].High-throughput sequencing

Regex解决方案提取<article>标签之间的所有内容（包括文本和其他HTML标签）

html <- paste(readLines(url), collapse = " ")
article <- sub(".*(<article.*?>.*</article>).*", "\\1", html)

article

# <article itemscope itemtype=\"http://schema.org/ScholarlyArticle\" lang=\"en\">                     <div class=\"c-article-header\">                                                   <ul class=\"c-article-identifiers\" data-test=\"article-identifier\">                                  <li class=\"c-article-identifiers__item\" data-test=\"article-category\">Method</li>                           <li class=\"c-article-identifiers__item\">                 <span class=\"c-article-identifiers__open\" data-test=\"open-access\">Open Access</span>             </li>                                                 <li class=\"c-article-identifiers__item\"><a href=\"#article-info\" data-track=\"click\" data-track-action=\"publication date\" data-track-category=\"article body\" data-track-label=\"link\">Published: <time datetime=\"2020-03-19\" itemprop=\"datePublished\">19 March 2020</time></a></li>                         </ul>                          <h1 class=\"c-article-title u-h1\" data-test=\"article-title\" data-article-title=\"\" itemprop=\"name headline\">VALOR2: characterization of large-scale structural variants using linked-reads</h1>                         <ul class=\"c-author-list js-list-authors js-etal-collapsed\" data-etal=\"25\" data-etal-small=\"3\" data-test=\"authors-list\"><li class=\"c-author-list__item\" itemprop=\"author\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Person\"><span itemprop=\"name\"><a data-test=\"author-name\" data-track=\"click\" data-track-action=\"open author\" data-track-category=\"article body\" data-track-label=\"link\" href=\"#auth-1\">Fatih KaraoÄŸlanoÄŸlu</a></span><sup class=\"u-js-hide\"><a href=\"#Aff1\">1</a><span itemprop=\"affiliation\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Organization\" class=\"u-visually-hidden\"><meta itemprop=\"name\" content=\"Bilkent University\" /><meta itemprop=\"address\" content=\"grid.18376.3b, 0000 0001 0723 2427, Department of Computer Engineering, Bilkent University, Ankara, 06800, Turkey\" /></span></sup><sup class=\"u-js-hide\">Â <a href=\"#na1\">na1</a></sup>, </li><li class=\"c-author-list__item\" itemprop=\"author\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Person\"><span itemprop=\"name\"><a data-test=\"author-name\" data-track=\"click\" data-track-action=\"open author\" data-track-category=\"article body\" data-track-label=\"link\" href=\"#auth-2\">Camir Ricketts</a></span><sup class=\"u-js-hide\"><a href=\"#Aff2\">2</a>,<a href=\"#Aff4\">4</a><span itemprop=\"affiliation\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Organization\" class=\"u-visually-hidden\"><meta itemprop=\"name\" content=\"Cornell University\" /><meta itemprop=\"address\" content=\"grid.5386.8, 000000041936877X, Tri-Institutional Computational Biology &amp; Medicine Program, Cornell University, 1300 York Ave, New York, 10065, NY, USA\" /></span><span itemprop=\"affiliation\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Organization\" class=\"u-visually-hidden\"><meta itemprop=\"name\" content=\"Weill Cornell Medicine\" /><meta itemprop=\"address\" content=\"grid.5386.8, 000000041936877X, Department of Physiology and Biophysics, Institute for Computational Biomedicine, Weill Cornell Medicine, 1300 York Ave, New York, 10065, NY, USA\" /></span></sup>, </li><li class=\"c-author-list__item\" itemprop=\"author\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Person\"><span itemprop=\"name\"><a data-test=\"author-name\" data-track=\"click\" data-track-action=\"open author\" data-track-category=\"article body\" data-track-label=\"link\" href=\"#auth-3\">Ezgi Ebren</a></span><sup class=\"u-js-hide\"><a href=\"#Aff1\">1</a><span itemprop=\"affiliation\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Organization\" class=\"u-visually-hidden\"><meta itemprop=\"name\" content=\"Bilkent University\" /><meta itemprop=\"address\" content=\"grid.18376.3b, 0000 0001 0723 2427, Department of Computer Engineering, Bilkent University, Ankara, 06800, Turkey\" /></span></sup>, </li><li class=\"c-author-list__item\" itemprop=\"author\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Person\"><span itemprop=\"name\"><a data-test=\"author-name\" data-track=\"click\" data-track-action=\"open author\" data-track-category=\"article body\" data-track-label=\"link\" href=\"#auth-4\">Marzieh Eslami Rasekh</a></span><sup class=\"u-js-hide\"><a href=\"#Aff3\">3</a><span itemprop=\"affiliation\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Organization\" class=\"u-visually-hidden\"><meta itemprop=\"name\" content=\"Boston University\" /><meta itemprop=\"address\" content=\"grid.189504.1, 0000 0004 1936 7558, Graduate Program in Bioinformatics, Boston University, 24 Cummington Mall, Boston, 02215, MA, USA\" /></span></sup>, </li><li class=\"c-author-list__item\" itemprop=\"author\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Person\"><span itemprop=\"name\"><a data-test=\"author-name\" data-track=\"click\" data-track-action=\"open author\" data-track-category=\"article body\" data-track-label=\"link\" href=\"#auth-5\" data-corresp-id=\"c1\">Iman Hajirasouliha<svg width=\"16\" height=\"16\" class=\"u-icon\"><use xmlns:xlink=\"http://www.w3.org/1999/xlink\" xlink:href=\"#global-icon-email\"></use></svg></a></span><sup class=\"u-js-hide\"><a href=\"#Aff4\">4</a>,<a href=\"#Aff5\">5</a><span itemprop=\"affiliation\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Organization\" class=\"u-visually-hidden\"><meta itemprop=\"name\" content=\"Weill Cornell Medicine\" /><meta itemprop=\"address\" content=\"grid.5386.8, 000000041936877X, Department of Physiology and Biophysics, Institute for Computational Biomedicine, Weill Cornell Medicine, 1300 York Ave, New York, 10065, NY, USA\" /></span><span itemprop=\"affiliation\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Organization\" class=\"u-visually-hidden\"><meta itemprop=\"name\" content=\"Weill Cornell Medicine\" /><meta itemprop=\"address\" content=\"grid.5386.8, 000000041936877X, Englander Institute for Precision Medicine, The Meyer Cancer Center, Weill Cornell Medicine, 1300 York Ave, New York, 10065, NY, USA\" /></span></sup> &amp; </li><li class=\"c-author-list__item\" itemprop=\"author\" itemscope=\"itemscope\" itemtype=\"http://schema.org/Person\"><span itemprop=\"name\"><a data-test=\"author-name\" data-track=\"click\" data-track-action=\"open author\" data-track-category=\"article body\" data-track-label=\"link\" href=\"#auth-6\" data-corresp-id=\"c2\">Can Alkan<svg width=\"16\" height=\"16\" class=\"u-icon\"><use xmlns:xlink=\"http://www.w3.org/1999/xlink\" xlink:href=\"#global-icon-email\"></use></svg></a>