【问题标题】:In this context, is there anyway I can use requests.post() without also using BeautifulSoup?在这种情况下,无论如何我可以在不使用 BeautifulSoup 的情况下使用 requests.post() 吗?
【发布时间】:2016-05-04 09:36:41
【问题描述】:

我花了一天半的时间(我是初学者)试图弄清楚如何做一些简单的事情:填写 ID 代码并点击“搜索”。我已经阅读了请求文档并搜索了堆栈溢出示例,但似乎没有任何帮助。

我想要做的是遍历我拥有的 ID 列表并检查哪些是有效的,哪些是无效的。 (这是为了学术研究) 这是我的代码:

import requests
url = 'https://ndber.seai.ie/pass/ber/search.aspx' 

payload = {'ctl00$DefaultContent$BERSearch$dfSearch$txtBERNumber':'100000000'
}
#the above dictionary key corresponds to the name , found using Firefox inspector in the source code. 
#I have tried using input ID too with no difference in outcome
#enter in arbitrary number of '100000000'- expect to see 'Invalid ID Number' printed


r = requests.post(url, data = payload)
print(r.status_code, r.reason)
print(r.text)
if 'Number.'in r.text: #'Number' is an arbitrary word that happens to be in the page if a correct ID code is entered
    print('Valid ID Number')
elif 'No results found.' in r.text: #as above, 'No results found' is in the page if an incorrect ID is entered
    print('Invalid ID Number') #with the aboove number entered, I would expect this line to be printed
else:
    print('Code Failure') #this line gets printed

我已尝试尽可能多地对其发表评论,以便您了解我的意思。 我得到的输出是Code Failure

BeautifulSoup 之所以出现这一切,是因为我昨天在 Reddit 的优秀“learn python”subreddit 上寻求帮助。一位 Redditor 慷慨地花时间试图解释我哪里出错了,以及我应该做些什么不同的事情。他建议我这样做: enter image description here,涉及BS4。

那么,Redditor 是对的,还是可以使用 requests 库以简单且“轻松”的方式完成?

干杯!

【问题讨论】:

  • 如果有人能解释为什么我没有得到任何回应,我将不胜感激。问题是否以某种方式格式不正确?我是否包含太多/太少的信息?是不是太长了?

标签: python-3.x beautifulsoup python-requests


【解决方案1】:

确保使用参数模拟完全相同的帖子,您可以使用 firefox 的 firebug 来查看正在发送的参数。 下面你可以看到参数,我看过。

__EVENTARGUMENT=
__EVENTTARGET=
__EVENTVALIDATION=/wEdAAUFK13s0OHvBcOmXYSKCCYAA5nYY19nD30S2PPC/bgb6yltaY6CwJw6+g9S5X73j5ccPYOXbBJVHSWhVCIjHWYlwa4c5fa9Qdw0vorP2Y9f7xVxOMmbQICW5xMXXERtE7U18lHqXqDZVqJ8R/A9h66g
__SCROLLPOSITIONX=0
__SCROLLPOSITIONY=114
__VIEWSTATE=/wEPDwULLTE2MDEwODU4NjAPFgIeE1ZhbGlkYXRlUmVxdWVzdE1vZGUCARYCZg9kFgICAw9kFgICAw8WAh4FY2xhc3MFC21haW53cmFwcGVyFgQCBQ8PFgIeB1Zpc2libGVnZGQCCQ9kFgICAQ9kFgxmD2QWAgIBD2QWAgIBD2QWAmYPZBYCZg9kFgQCAQ8WAh4JaW5uZXJodG1sZWQCAw9kFgICAg9kFgJmD2QWBAIBD2QWAgIDDw8WCh4EXyFTQgKAAh4MRGVmYXVsdFdpZHRoHB4EVGV4dAUFY3RsMDAeB1Rvb2xUaXAFPlBsZWFzZSBlbnRlciBhIHZhbHVlLCB3aXRoIG5vIHNwZWNpYWwgY2hhcmFjdGVycywgd2l0aCBubyB0ZXh0HgVXaWR0aBxkZAIDD2QWAgIDDw8WCh8EAoACHwUcHwYFCTEwMDAwMDAwMB8HBT5QbGVhc2UgZW50ZXIgYSB2YWx1ZSwgd2l0aCBubyBzcGVjaWFsIGNoYXJhY3RlcnMsIHdpdGggbm8gdGV4dB8IHGRkAgQPFCsAAmQQFgAWABYAFgJmD2QWAgICD2QWAmYPPCsAEQIBEBYAFgAWAAwUKwAAZAIGDxYCHwJoFgQCAQ8WAh8CaGQCAw9kFgJmD2QWAmYPZBYCAgMPZBYCZg9kFgJmD2QWAgIBDxYCHwJoZAIIDxYCHwJoFgQCAQ8WAh8CaGQCAw9kFgJmD2QWAmYPZBYCAgMPZBYCZg9kFgJmD2QWAgIBDxYCHwJoZAIKDxYCHwJoFgQCAQ8WAh8CaGQCAw9kFgJmD2QWAmYPZBYCAgMPZBYCZg9kFgJmD2QWAgIBDxYCHwJoZAIMDxYCHwJoFgQCAQ8WAh8CaGQCAw9kFgJmD2QWAmYPZBYCAgMPZBYCZg9kFgJmD2QWAgIBDxYCHwJoZBgBBTNjdGwwMCREZWZhdWx0Q29udGVudCRCRVJTZWFyY2gkZ3JpZFJhdGluZ3MkZ3JpZHZpZXcPZ2QPHVcTs3MrN3SZqli/KiHgWASd1DRvCLPuS7Rs+GRQAQ==
__VIEWSTATEGENERATOR=1F9CCB97
ctl00$DefaultContent$BERSearch$dfSearch$Bottomsearch=Search
ctl00$DefaultContent$BERSearch$dfSearch$txtBERNumber=2313131
ctl00$DefaultContent$BERSearch$dfSearch$txtMPRN=11111111111

下面的代码刚刚工作,

from __future__ import unicode_literals

import requests
url = 'https://ndber.seai.ie/pass/ber/search.aspx' 

payload = {'__EVENTVALIDATION':'/wEdAAUFK13s0OHvBcOmXYSKCCYAA5nYY19nD30S2PPC/bgb6yltaY6CwJw6+g9S5X73j5ccPYOXbBJVHSWhVCIjHWYlwa4c5fa9Qdw0vorP2Y9f7xVxOMmbQICW5xMXXERtE7U18lHqXqDZVqJ8R/A9h66g',
           '__VIEWSTATE':'/wEPDwULLTE2MDEwODU4NjAPFgIeE1ZhbGlkYXRlUmVxdWVzdE1vZGUCARYCZg9kFgICAw9kFgICAw8WAh4FY2xhc3MFC21haW53cmFwcGVyFgQCBQ8PFgIeB1Zpc2libGVnZGQCCQ9kFgICAQ9kFgxmD2QWAgIBD2QWAgIBD2QWAmYPZBYCZg9kFgQCAQ8WAh4JaW5uZXJodG1sZWQCAw9kFgICAg9kFgJmD2QWBAIBD2QWAgIDDw8WCh4EXyFTQgKAAh4MRGVmYXVsdFdpZHRoHB4EVGV4dAUFY3RsMDAeB1Rvb2xUaXAFPlBsZWFzZSBlbnRlciBhIHZhbHVlLCB3aXRoIG5vIHNwZWNpYWwgY2hhcmFjdGVycywgd2l0aCBubyB0ZXh0HgVXaWR0aBxkZAIDD2QWAgIDDw8WCh8EAoACHwUcHwYFCTEwMDAwMDAwMB8HBT5QbGVhc2UgZW50ZXIgYSB2YWx1ZSwgd2l0aCBubyBzcGVjaWFsIGNoYXJhY3RlcnMsIHdpdGggbm8gdGV4dB8IHGRkAgQPFCsAAmQQFgAWABYAFgJmD2QWAgICD2QWAmYPPCsAEQIBEBYAFgAWAAwUKwAAZAIGDxYCHwJoFgQCAQ8WAh8CaGQCAw9kFgJmD2QWAmYPZBYCAgMPZBYCZg9kFgJmD2QWAgIBDxYCHwJoZAIIDxYCHwJoFgQCAQ8WAh8CaGQCAw9kFgJmD2QWAmYPZBYCAgMPZBYCZg9kFgJmD2QWAgIBDxYCHwJoZAIKDxYCHwJoFgQCAQ8WAh8CaGQCAw9kFgJmD2QWAmYPZBYCAgMPZBYCZg9kFgJmD2QWAgIBDxYCHwJoZAIMDxYCHwJoFgQCAQ8WAh8CaGQCAw9kFgJmD2QWAmYPZBYCAgMPZBYCZg9kFgJmD2QWAgIBDxYCHwJoZBgBBTNjdGwwMCREZWZhdWx0Q29udGVudCRCRVJTZWFyY2gkZ3JpZFJhdGluZ3MkZ3JpZHZpZXcPZ2QPHVcTs3MrN3SZqli/KiHgWASd1DRvCLPuS7Rs+GRQAQ==',
           '__VIEWSTATEGENERATOR':'1F9CCB97',
           'ctl00$DefaultContent$BERSearch$dfSearch$Bottomsearch':'Search',
           'ctl00$DefaultContent$BERSearch$dfSearch$txtBERNumber':'2313131',
           'ctl00$DefaultContent$BERSearch$dfSearch$txtMPRN':'11111111111'
}
session = requests.session()
r = requests.post(url, data=payload)
if 'Number.'in r.text: #'Number' is an arbitrary word that happens to be in the page if a correct ID code is entered
    print('Valid ID Number')
elif 'No results found.' in r.text: #as above, 'No results found' is in the page if an incorrect ID is entered
    print('Invalid ID Number') #with the aboove number entered, I would expect this line to be printed
else:
    print('Code Failure') #this line gets printed"""

只要把你的参数放在payload字典里,其余的参数我认为可以保留

BER/DEC 编号 > ctl00$DefaultContent$BERSearch$dfSearch$txtBERNumber

MPRN > ctl00$DefaultContent$BERSearch$dfSearch$txtMPRN

【讨论】:

    猜你喜欢
    • 2012-03-28
    • 2018-05-05
    • 1970-01-01
    • 2022-11-14
    • 2016-09-12
    • 2016-11-07
    • 2022-08-05
    • 2013-11-14
    • 2012-09-09
    相关资源
    最近更新 更多