【发布时间】:2020-12-14 08:57:14
【问题描述】:
我正在写一个用于抓取歌词的小包。现在的问题是它的行为非常笨拙,无法控制。我希望能够在执行抓取的 for 循环中指定要影响的范围。
我有两个功能,一个是显示歌曲列表,另一个是抓取它们:
songlist <- function(x) {
url <- paste0("https://www.azlyrics.com/", substring(x, 1, 1),"/",x, ".html")
page <- url
songs <- page %>%
xml2::read_html() %>%
rvest::html_nodes(xpath = "/html/body/div[2]/div/div[2]/div[4]/div/a") %>%
rvest::html_text() %>%
as.data.frame()
chart <- cbind(songs)
names(chart) <- c("Songs")
chart <- tibble::as_tibble(chart)
return(chart)
}
这给了我一个像这样的小标题:
Songs
<chr>
1 Meet Me In The Hallway
2 Sign Of The Times
3 Carolina
4 Two Ghosts
5 Sweet Creature
6 Only Angel
7 Kiwi
8 Ever Since New York
9 Woman
另一个名为songscrape() 的函数接受一个字符串参数,用于构建网址并从该艺术家那里抓取所有歌曲。它里面有一个songlist() 的副本,它会为 Like so 生成 url:
songscrape <- function(x) {
url <- paste0("https://www.azlyrics.com/", substring(x, 1, 1),"/",x, ".html")
artist <- x
SongsListScrapper <- function(x) {
page <- x
songs <- page %>%
xml2::read_html() %>%
rvest::html_nodes(xpath = "/html/body/div[2]/div/div[2]/div[4]/div/a") %>%
rvest::html_text() %>%
as.data.frame()
chart <- cbind(songs)
names(chart) <- c("Songs")
chart <- tibble::as_tibble(chart)
return(chart)
}
SongsList <- purrr::map_df(url, SongsListScrapper)
SongsList
SongsList %<>%
dplyr::mutate(
Songs = as.character(Songs)
,Songs = gsub("[[:punct:]]", "", Songs)
,Songs = tolower(Songs)
,Songs = gsub(" ", "", Songs)
)
SongsList$Songs
#Scrape Lyrics
wipe_html <- function(str_html) {
gsub("<.*?>", "", str_html)
}
lyrics <- c()
for(i in seq_along(SongsList$Songs)) {
for_url_name <- SongsList$Songs[i]
#clean name
for_url_name <- tolower(gsub("[[:punct:]]\\s", "", for_url_name))
#create url
paste_url <- paste0("https://www.azlyrics.com/lyrics/", artist,"/", for_url_name, ".html")
tryCatch( {
#open connection to url
for_html_code <- xml2::read_html(paste_url)
for_lyrics <- rvest::html_node(for_html_code, xpath = "/html/body/div[2]/div/div[2]/div[5]")
}
如您所见,for 循环显示for(i in seq_along(SongsList$Songs)),基本上表示所有歌曲。是否可以改为输入范围。就像我最多选择 5 首歌曲或从第 10 首歌曲到第 15 首歌曲(基于 tibble 中的数字)。
所以更新后的函数调用看起来像 songscrape("ironwine", from = 5, to = 15) 或 songscrape("ironwine", n = 20) #first 20 songs
我该怎么做?
【问题讨论】: