【发布时间】:2020-09-14 11:04:01
【问题描述】:
谁能告诉我在下面的代码中做错了什么?
我正在编写 php 脚本以从 https://beams.us.yazaki.com/BEAMSLogin/Login.aspx?ReturnUrl=%2FBeams%2FDefault.aspx 登录和抓取数据
我已尝试使用以下 curl 代码。
<?php
define('USERNAME', 'XXX');
define('PASSWORD', 'XXX');
define('DOMAIN', 'XXX');
define('USER_AGENT', 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.2309.372 Safari/537.36');
define('COOKIE_FILE', 'cookie.txt');
define('LOGIN_FORM_URL', 'https://beams.us.yazaki.com/BEAMSLogin/Login.aspx?ReturnUrl=%2fBeams%2fDefault.aspx');
define('LOGIN_ACTION_URL', 'https://beams.us.yazaki.com/BEAMSLogin/Login.aspx?ReturnUrl=%2fBeams%2fDefault.aspx');
$postValues = array(
'txtUserName' => USERNAME,
'txtPassword' => PASSWORD,
'lstDomain' => DOMAIN
);
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, LOGIN_ACTION_URL);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, http_build_query($postValues));
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt($curl, CURLOPT_USERAGENT, USER_AGENT);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_REFERER, LOGIN_FORM_URL);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, false);
curl_exec($curl);
if(curl_errno($curl)){
throw new Exception(curl_error($curl));
}
curl_setopt($curl, CURLOPT_URL, 'https://beams.us.yazaki.com/BEAMS/SearchAndResults.aspx?topic=component');
curl_setopt($curl, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt($curl, CURLOPT_USERAGENT, USER_AGENT);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
echo curl_exec($curl);
?>
我得到如下页面的输出。
Object not found!
The requested URL was not found on this server. The link on the referring page seems to be wrong or outdated. Please inform the author of that page about the error.
If you think this is a server error, please contact the webmaster.
Error 404
localhost
Apache/2.4.43 (Win64) OpenSSL/1.1.1g PHP/7.4.8 *
浏览器被重定向到 http://localhost/BeamsLogin/AuthenticationSelect.aspx?ReturnUrl=%2fBEAMS%2fSearchAndResults.aspx%3ftopic%3dcomponent&topic=component
我正在认真等待回复。
谢谢。
编辑 1 @Shibon 我认为这个网站不使用任何 csrf 令牌。我正在提供 HTTP 标头实时内容。请检查一次。
https://beams.us.yazaki.com/BEAMSLogin/Login.aspx?ReturnURL=%2fBeams%2fDefault.aspx
Host: beams.us.yazaki.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://beams.us.yazaki.com/
Connection: keep-alive
Upgrade-Insecure-Requests: 1
GET: HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
Vary: Accept-Encoding
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Tue, 15 Sep 2020 06:19:19 GMT
Content-Length: 3376
【问题讨论】:
-
我认为这与 cookie 有关。首先尝试获取网站,然后尝试登录,您还应该查看表单中的 __VIEWSTATE。
-
我尝试使用
$cookies = curl_getinfo($curl, CURLINFO_COOKIELIST); print_r($cookies);获取并得到 Array() 作为输出。我认为你是对的。你能建议如何解决这个问题,因为我对此有点陌生。 -
谢谢@nguaman,我应该发送 __VIEWSTATE、__EVENTVALIDATION、__VIEWSTATEGENERATOR 以及用户名和密码。
标签: php curl web-scraping