【发布时间】:2021-12-01 11:48:35
【问题描述】:
我正在尝试从这个网站获取数据:https://www.illibraio.it/librerie/,数据似乎专门存储在多个不同长度的https://www.illibraio.it/wp-admin/admin-ajax.php 页面中。
我按照教程编写了这个简短的代码,但我不明白为什么它不起作用。我不断收到此错误消息
ERROR: Spider error processing <POST https://www.illibraio.it/wp-admin/admin-ajax.php> (referer: https://www.illibraio.it/librerie/)
这是我当前的参考代码,感谢任何帮助。这东西我是初学者,但我需要网站上的数据...
import scrapy
import json
class Librerie(scrapy.Spider):
name = "Librerie"
start_urls = ['https://www.illibraio.it/librerie/']
headers = {
"Accept": "application/json, text/javascript, */*; q=0.01",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9",
"Referer": "https://www.illibraio.it/librerie/",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Site": "same-origin",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36",
"X-Requested-With": "XMLHttpRequest",
}
data = {
'nome': 'Mondadori Bookstore - Melegnano',
'indirizzo': 'Via Zuavi, 8 20077 Melegnano MI',
'latlong': '{"lat":"45.356655","lng":"9.322607"'
}
def parse(self, response):
url = 'https://www.illibraio.it/wp-admin/admin-ajax.php'
meta = {'handle_httpstatus_all': True}
yield scrapy.FormRequest(url,
method='POST',
formdata=self.data,
meta=meta,
callback=self.parse_api,
headers=self.headers)
def parse_api(self, response):
raw_data = response.body
data = json.loads(raw_data)
yield {
'nome' : data['title'],
'indirizzo' : data['location'],
'latlong' : data['coords'],
}
【问题讨论】:
标签: python web-scraping scrapy