如何在 requests.post() 中生成 Python XHR 请求

Question

我正在尝试使用 AJAX 从网站获取信息。 Website 显示不同尺寸的香水，基本上，选择不同尺寸时价格会发生变化。 我检查了 chrome Network Tab，发现它是一个 XHR 请求，但是查看请求头我不知道如何使用 Requests 包生成相同的标头和数据。

这是我的代码目前的样子：

import requests    
url = "https://www.beautyfresh.com/uc_aac"
session = requests.Session()
data = {"attributes[Size]":"100ml"} # I want to get the price for 100ml
headers = {"Referer": "https://www.beautyfresh.com/product/fragrance/men/perfume-fragrance/women/perfume-men/fragrance/perfume/jo-malone-orange-blossom-cologne",}
r = session.post(url,headers=headers,data=data)
print(r.text)

Chrome 网络标签下的常规信息是

Request URL: https://www.beautyfresh.com/uc_aac
Request Method: POST
Status Code: 200 OK
Remote Address: 103.255.250.100:443
Referrer Policy: no-referrer-when-downgrade

响应标头是

HTTP/1.1 200 OK
date: Fri, 18 Dec 2020 02:01:05 GMT
expires: Sun, 19 Nov 1978 05:00:00 GMT
x-site: beautyfresh
x-url: /uc_aac
last-modified: Fri, 18 Dec 2020 02:01:05 GMT
x-backend-server: web4
content-type: application/json
x-varnish: 700512226
age: 0
via: 1.1 varnish (Varnish/6.0)
x-cache: MISS
cache-control: Cache-Control: store, no-cache, must-revalidate
accept-ranges: bytes
content-length: 2193

请求标头是

POST /uc_aac HTTP/1.1
Host: www.beautyfresh.com
Connection: keep-alive
Content-Length: 164
Accept: application/json, text/javascript, */*; q=0.01
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_1_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Origin: https://www.beautyfresh.com
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://www.beautyfresh.com/product/fragrance/men/perfume-fragrance/women/perfume-men/fragrance/perfume/jo-malone-orange-blossom-cologne
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9
Cookie: has_js=1; SESSc5f2026dce40de323b60d32130e6ce0b=n7tr5e2fsragf1js6garc6u06n; _ga=GA1.2.959938064.1608216963; _gid=GA1.2.1979056032.1608216963; _v1EmaticSolutionsUTMData=%7B%22utm_source%22%3A%22%22%2C%22utm_medium%22%3A%22%22%2C%22utm_campaign%22%3A%22%22%7D; _fbp=fb.1.1608216963288.1600789170; _v1EmaticSolutionsBye=%7B%2228732%22%3A%7B%2230038%22%3A%7B%22dont_show_till%22%3A%222020-12-20%22%2C%22loop%22%3A1%7D%7D%2C%2228739%22%3A%7B%2230045%22%3A%7B%22dont_show_till%22%3A%222020-12-20%22%2C%22loop%22%3A1%7D%7D%7D; _v1EmaticSolutionsEI=%7B%22c_28739_1%22%3A%5B1%2C1608216997180%2C33181%5D%2C%22c_28732_2%22%3A%5B1%2C1608216969616%2C0%5D%7D; _v1EmaticSolutions=%5B%22fc5d18b3-4077-11eb-970e-0242ac160003%22%2C1608217230870%2C%5B%22IMG%22%2C%22%22%2C1%2C%22glasshouse_fragrances_amalfi_coast_sea_candle_350gr.jpg%22%5D%5D; __atuvc=8%7C51; __atuvs=5fdc0d5961186ea4000; _gat=1; _dc_gtm_UA-63339192-1=1

而表单数据显示：

attributes%5BSize%5D=100ml&nid=2905&qty=1&form_build_id=form-991e88780c30fdf883375a36a986b550&form_id=uc_product_add_to_cart_form_2905&product-nid=2905&aac_nid=2905

我实际上不知道如何构造我的请求，以便可以成功地将其发布到服务器以获得正确的响应。它应该返回“100ml”的价格，但目前我的代码一无所获。

非常感谢您的帮助！

Answer 1

您的data 中缺少一些信息。要接收响应，请在发送 post 请求时尝试将 "aac_nid": "2905" 添加到您的 data：

import requests


headers = {
    "Referer": "https://www.beautyfresh.com/product/fragrance/men/perfume-fragrance/women/perfume-men/fragrance/perfume/jo-malone-orange-blossom-cologne",
}

data = {"attributes[Size]": "30ml", "aac_nid": "2905"}

response = requests.post(
    "https://www.beautyfresh.com/uc_aac", headers=headers, data=data
)

>>> print(response.content)
b'{"nid":"2905","model":"690251006564","replacements":{"sellprice":"<div class=\\"product-info sellprice\\"><div class=\'retail_price\'>City Price: <div class=\'product-info retail_price\'><span class=\'uc-price\'>S$110.00<\\/span><\\/div><\\/div>\\r\\n                <div class=\'promoprice\'>Our Price: <div class=\'product-info sellprice1\'><span class=\'uc-price\'>S$89.00<\\/span><\\/div><\\/div><\\/div>","model":"<div class=\\"model\\"><span class=\\"label\\">Product Code: <\\/span>690251006564<\\/div>"},"form":"<form action=\\"\\/uc_aac\\"  accept-charset=\\"UTF-8\\" method=\\"post\\" id=\\"uc-product-add-to-cart-form\\" class=\\" uc-aac-cart\\">\\n<div><div class=\'attributes\'><div class=\\"form-item\\" id=\\"edit-attributes-Size-wrapper\\">\\n <label for=\\"edit-attributes-Size\\">Size <span class=\\"form-required\\" title=\\"This field is required.\\">*<\\/span><\\/label>\\n <select name=\\"attributes[Size]\\" class=\\"form-select required chosen-widget\\" data-name=\\"Size\\" id=\\"edit-attributes-Size\\" ><option value=\\"30ml\\" selected=\\"selected\\">30ml<\\/option><option value=\\"100ml\\">100ml<\\/option><\\/select>\\n<\\/div>\\n<\\/div><input type=\\"hidden\\" name=\\"nid\\" id=\\"edit-nid\\" value=\\"2905\\"  \\/>\\n<div class=\\"form-item\\" id=\\"edit-qty-wrapper\\">\\n <label for=\\"edit-qty\\">Qty <\\/label>\\n <input type=\\"text\\" maxlength=\\"3\\" name=\\"qty\\" id=\\"edit-qty\\" size=\\"5\\" value=\\"1\\" class=\\"form-text textfield\\" \\/>\\n<\\/div>\\n<input type=\\"hidden\\" name=\\"form_build_id\\" id=\\"form-b7adf002178f04ed96377894057352a2\\" value=\\"form-b7adf002178f04ed96377894057352a2\\"  \\/>\\n<input type=\\"hidden\\" name=\\"form_id\\" id=\\"edit-uc-product-add-to-cart-form\\" value=\\"uc_product_add_to_cart_form\\"  \\/>\\n<input type=\\"hidden\\" name=\\"aac_nid\\" id=\\"edit-aac-nid\\" value=\\"2905\\"  \\/>\\n<div class=\'leadtime_message\' style=\'margin-bottom:1em;\'><p>Delivers in 1-3 working days<\\/p><\\/div><input type=\\"submit\\" name=\\"op\\" id=\\"edit-submit-2905\\" value=\\"Add to Cart\\"  class=\\"notranslate form-submit node-add-to-cart primary\\" \\/>\\n\\n<\\/div><\\/form>\\n"}'

要获取价格，请尝试借助内置的re（正则表达式）模块进行搜索：

import re

prices = re.findall(r"'uc-price\\'>S(\$\d.*?)<", str(response.content))

print("Original Price:", prices[0])
print("Our Price:", prices[1])

输出：

Original Price: $110.00
Our Price: $89.00