刮这个网站答案

【问题标题】：Scraping this site刮这个网站
【发布时间】：2020-06-05 13:04:13
【问题描述】：

我试图用这个payload从这个站点模仿这个POST request：

这是我尝试mimic 的请求的图像。

这是我当前在 python 中的代码：

import requests
headers = {'content-type': 'application/json; charset=UTF-8'}
url = 'https://surviv.io/api/user_stats'
payload = {"slug":"gert1","interval":"all","mapIdFilter":"-1"}
r = requests.post(url=url, headers=headers, data=payload)
print(r.content)

这会返回：

b'<html>\r\n<head><title>500 Internal Server Error</title></head>\r\n<body bgcolor="white">\r\n<center><h1>500 Internal Server Error</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n'

这不是我想要它返回的。我希望它返回 user_stats 请求的响应选项卡中显示的确切响应，其中包含玩家的统计信息。

这是我希望它返回的内容：

{"slug":"gert1","username":"GERT","player_icon":"","banned":false,"wins":61,"kills":2830,"games":2034,"kpg":"1.4","modes":[{"teamMode":1,"games":1512,"wins":46,"kills":2230,"winPct":"3.0","mostKills":21,"mostDamage":1872,"kpg":"1.5","avgDamage":169,"avgTimeAlive":92},{"teamMode":2,"games":255,"wins":4,"kills":234,"winPct":"1.6","mostKills":8,"mostDamage":861,"kpg":"0.9","avgDamage":162,"avgTimeAlive":102},{"teamMode":4,"games":267,"wins":11,"kills":366,"winPct":"4.1","mostKills":17,"mostDamage":2225,"kpg":"1.4","avgDamage":246,"avgTimeAlive":125}]}

【问题讨论】：

标签： python post web-scraping beautifulsoup python-requests

【解决方案1】：

您应该在 post 方法中使用 json 属性而不是数据。 r = requests.post(url=url, headers=headers, json=payload)

【讨论】：

【解决方案2】：

将您的代码更改为您忘记使用json ：

import json
import requests
headers = {'content-type': 'application/json; charset=UTF-8'}
url = 'https://surviv.io/api/user_stats'
payload = {"slug":"gert1","interval":"all","mapIdFilter":"-1"}
r = requests.post(url=url, headers=headers, data=json.dumps(payload))
print(r.content)

【讨论】：

谢谢。如果数据不是 json，为什么我的代码不起作用。数据参数是否总是必须在json中
是的，因为您提供了 content-type 作为 application/json。