如何在 php 中使用 curl 从网页中获取特定内容？

【问题标题】：How to get specific content from a webpage using curl in php?如何在 php 中使用 curl 从网页中获取特定内容？
【发布时间】：2020-08-09 19:05:42
【问题描述】：

我想从作者的谷歌学者页面获取最新出版物的作者和共同作者姓名。为此，我尝试将 curl 与 php 一起使用。但由于 div 没有特定的 ID，并且多个 div 也有类似的 className，我无法通过网络抓取来跟踪数据。 到目前为止，我试过这个：

<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://scholar.google.com/citations?hl=en&user=mG4imMEAAAAJ&view_op=list_works&sortby=pubdate");
curl_setopt($ch, CURLOPT_HEADER, 0);

$data = curl_exec($ch);
curl_close($ch);
?>

// To Do:
// from $data variable need to get the author's name.
// <div class="gs_gray">R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz,...</div>

【问题讨论】：

标签： php curl web-scraping


【解决方案1】：

你可以用DOMDocument和DOMXPath解析html，你可以用XPath迭代文章

//*[@id="gsc_a_b"]/tr

一旦你有了一篇文章，你就可以使用 XPath 获取作者

.//div[@class="gs_gray"][1]

以文章为参考节点，所以这个

$content=file_get_contents('https://scholar.google.com/citations?hl=en&user=mG4imMEAAAAJ&view_op=list_works&sortby=pubdate');
$domd=@DOMDocument::loadHTML($content);
$xp=new DOMXPath($domd);
foreach($xp->query('//*[@id="gsc_a_b"]/tr') as $article){
//    var_dump($domd->saveHTML($article));
    foreach($xp->query('.//div[@class="gs_gray"][1]',$article) as $authors){
      var_dump($authors->textContent);

    }
}

输出：

string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"
string(53) "Y Shen, M Voisin, A Aliamiri, A Avati, A Hannun, A Ng"
string(74) "J Irvin, P Rajpurkar, M Ko, Y Yu, S Ciurea-Ilcus, C Chute, H Marklund, ..."
string(70) "D Rolnick, PL Donti, LH Kaack, K Kochanski, A Lacoste, K Sankaran, ..."
string(73) "F Maksimovic, B Wheeler, DC Burnett, O Khan, S Mesri, I Suciu, L Lee, ..."
string(63) "A Park, C Chute, P Rajpurkar, J Lou, RL Ball, K Shpanskaya, ..."
string(48) "J Lee, Z Xie, C Wang, M Drach, D Jurafsky, AY Ng"
string(75) "R Pranav, A Park, J Irvin, C Chute, M Bereket, M Domenico, CP Langlotz, ..."
string(73) "MT Chiu, X Xu, K Wang, J Hobbs, N Hovakimyan, TS Huang, H Shi, Y Wei, ..."
string(56) "A Smit, S Jain, P Rajpurkar, A Pareek, AY Ng, MP Lungren"
string(69) "P Rajpurkar, A Park, J Irvin, C Chute, M Bereket, D Mastrodicasa, ..."
string(68) "P Rajpurkar, A Joshi, A Pareek, P Chen, A Kiani, J Irvin, AY Ng, ..."
string(69) "A Kiani, B Uyumazturk, P Rajpurkar, A Wang, R Gao, E Jones, Y Yu, ..."
string(40) "S Zhou, J Zhang, H Jiang, T Lundh, AY Ng"
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G Diamos, E Elsen, ..."
string(63) "A Hannun, C Case, J Casper, B Catanzaro, G DIAMOS, E Elsen, ..."
string(40) "G Yuan, S Yang, S Yang, A Ng, MJ Oursler"
string(77) "G McNicol, E Fluet-chouinard, Z Zhang, J Irvin, S Zhou, F Lu, A Kondrich, ..."
string(69) "B Uyumazturk, A Kiani, P Rajpurkar, A Wang, RL Ball, R Gao, Y Yu, ..."
string(52) "B Wheeler, A Ng, B Kilberg, F Maksimovic, KSJ Pister"
string(50) "T Duan, A Avati, DY Ding, S Basu, AY Ng, A Schuler"

（作为优化说明，如果您将 curl 与 CURLOPT_ENCODING 一起使用，它将比使用 file_get_contents() 更快 - curl 支持 gzip，file_get_contents() 不支持 - curl 支持读取直到“Content-Length”并停止读取，而 file_get_contents() 只会读取直到远程服务器关闭套接字，这在某些网站上使得 file_get_contents() 比 curl 慢得多。）

【讨论】：

  • 谢谢！还有一个问题，这个过程并不总是有效。有时它会给出“HTTP ERROR 500”。谷歌会阻止这种自动报废吗？
  • @mnur 您的意思是您的代码从 google 获得 500 响应，还是您的意思是当您尝试运行代码时从您自己的网络服务器收到 500 错误？如果是后者，请检查您的 php 错误日志，但如果 500 来自 google，我猜 google 在某处有错误，您应该再试一次，500 不用于速率限制，它用于当错误/服务器时-errors 使网站崩溃。
  • 那次我试了好几次，都遇到同样的错误。但是今天我再次尝试并获得了预期的输出。由于我正在对数千个 url 执行此报废操作，因此我认为 google 在同时从 IP 地址发出过多请求时会阻塞。我需要找到一个技巧来在每次请求后暂停。你能帮我解决这个问题吗？
  • @mnur 可以在获取页面后使用 sleep() 命令 - 或者如果这对您来说太慢，您可以使用代理/多个 IP 来进行呼叫。如果您有预算，我可以提供帮助，联系信息here
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2014-07-20
  • 2013-02-16
  • 1970-01-01
  • 2014-05-15
  • 2012-04-21
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode