运行代码从网站下载图片时出现“read_xml.raw 中的错误”

Question

我正在尝试根据特定人的姓名（在 R 中）从网站（网址）下载图片。 我收到以下错误

Error in read_xml.raw(raw, encoding = encoding, base_url = base_url, as_html = as_html,  : 
CHAR() can only be applied to a 'CHARSXP', not a 'NULL' 

这是回溯

19.read_xml.raw(raw, encoding = encoding, base_url = base_url, as_html = as_html, 
options = options) 
18.read_xml.connection(con, encoding = encoding, ..., as_html = as_html, 
base_url = x, options = options) 
17.read_xml.character(x, encoding = encoding, ..., as_html = TRUE, 
options = options) 
16.read_xml(x, encoding = encoding, ..., as_html = TRUE, options = options) 
15.withCallingHandlers(expr, warning = function(w) if (inherits(w, 
classes)) tryInvokeRestart("muffleWarning")) 
14.suppressWarnings(read_xml(x, encoding = encoding, ..., as_html = TRUE, 
options = options)) 
13.read_html.default(., image_page) 
12.read_html(., image_page) 
11.html_nodes(., "img") 
10.xml2::xml_attr(x, name, default = default) 
9.html_attr(., "src") 
8.handle_url(handle, url, ...) 
7.httr::GET(.) 
6.is.response(x) 
5.stopifnot(is.response(x)) 
4.httr::content(., "raw") 
3.writeBin(., paste0("~/", ceo_name, ".jpg")) 
2.paste0(site, image_page) %>% read_html(image_page) %>% html_nodes("img") %>% 
html_attr("src") %>% {
grep("gstatic", ., value = TRUE)
} %>% 1[] %>% httr::GET() %>% httr::content("raw") %>% writeBin(paste0("~/",  ... 
1.get_image(as.character("Mark Lloyd")) 

我不明白为什么。请问，有人可以启发我吗？ 非常感谢

代码

> library(rvest)
> library(httr)
> 
> get_image <- function(ceo_name)
+ {
+   site <- "https://www.icobench.com"
+   query <- paste0(site, "/ico/max-crowdfund/team", url_escape(ceo_name))
+   
+   image_page <- read_html(query)                          %>% 
+     html_nodes(xpath = "//a[contains(text(), 'Images')]") %>% 
+     html_attr("href")
+   
+   paste0(site, image_page)             %>%
+     read_html(image_page)              %>%
+     html_nodes("img")                  %>% 
+     html_attr("src")                   %>% 
+     {grep("gstatic", ., value = TRUE)} %>% 
+     `[`(1)                             %>%
+     httr::GET()                        %>%
+     httr::content("raw")               %>%
+     writeBin(paste0("~/", ceo_name, ".jpg"))
+ }
> 
> get_image(as.character("Mark Lloyd"))

Answer 1

我会确保我有？在 url 中指示查询字符串。然后我会使用一个较短的 css 选择器从页面中挑选出适当的图像。您应该真正检查状态以确保找到页面。

找到合适的节点后，我将使用正则表达式模式提取 url 并将其与协议 + 域从 url_absolute 直接传递给 httr。正则表达式，因为 url 在该页面的样式属性内。

理想情况下，在尝试从中提取属性值之前，还应该测试一个节点是否匹配。

由于您只需要单个图像，因此您可以将 html_node 用于 ceo 节点，从而获得在单个节点匹配后退出的效率。

---

这是假设实际的登陆网址，加上? 本来是：

https://icobench.com/ico/max-crowdfund/team?Mark%20Lloyd

对于该页面，您的 xpath 语句无效。

---

未与其他 CEO 一起测试。

---

library(rvest)
library(httr)
library(stringr)

get_image <- function(ceo_name) {
  site <- "https://www.icobench.com"
  query <- paste0(site, "/ico/max-crowdfund/team?", url_escape(ceo_name))

  image_page <- read_html(query) %>% 
  html_node(paste0('[title="', ceo_name, '"] .image_background')) %>%
    html_attr("style") %>%
    stringr::str_extract("/.*\\.jpg") %>%
    url_absolute(site)

  image_page %>%
    httr::GET() %>%
    httr::content("raw") %>%
    writeBin(paste0("~/", ceo_name, ".jpg"))
}

get_image(as.character("Mark Lloyd"))