在标签 BeautifulSoup 中显示文本

Question

我试图只显示标签内的文本，例如：

<span class="listing-row__price ">$71,996</span>

我只想显示

“71,996 美元”

我的代码是：

import requests
from bs4 import BeautifulSoup
from csv import writer

response = requests.get('https://www.cars.com/for-sale/searchresults.action/?mdId=21811&mkId=20024&page=1&perPage=100&rd=99999&searchSource=PAGINATION&showMore=false&sort=relevance&stkTypId=28880&zc=11209')

soup = BeautifulSoup(response.text, 'html.parser')

cars = soup.find_all('span', attrs={'class': 'listing-row__price'})
print(cars)

如何从标签中提取文本？

Answer 1

要获取标签内的文本，有两种方法，

a) 使用标签的.text属性。

cars = soup.find_all('span', attrs={'class': 'listing-row__price'})
for tag in cars:
    print(tag.text.strip())

输出

$71,996
$75,831
$71,412
$75,476
....

b) 使用get_text()

for tag in cars:
    print(tag.get_text().strip())

c) 如果标签内只有那个字符串，你也可以使用这些选项

• .string
• .contents[0]
• next(tag.children)
• next(tag.strings)
• next(tag.stripped_strings)

即。

for tag in cars:
    print(tag.string.strip()) #or uncomment any of the below lines
    #print(tag.contents[0].strip())
    #print(next(tag.children).strip())
    #print(next(tag.strings).strip())
    #print(next(tag.stripped_strings))

输出：

$71,996
$75,831
$71,412
$75,476
$77,001
...

注意：

.text 和 .string 不一样。如果标签中还有其他元素，.string 返回None，而.text 将返回标签内的文本。

from bs4 import BeautifulSoup
html="""
<p>hello <b>there</b></p>
"""
soup = BeautifulSoup(html, 'html.parser')
p = soup.find('p')
print(p.string)
print(p.text)

输出

None
hello there

Answer 2

print( [x.text for x in cars] )

Answer 3

实际上request 没有返回任何response。如我所见，响应代码是500，这意味着网络问题，您没有收到任何数据。

您缺少的是user-agent，您需要将其发送到headers 和request。

import requests
import re #regex library
from bs4 import BeautifulSoup

headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36"
}

crawl_url = 'https://www.cars.com/for-sale/searchresults.action/?mdId=21811&mkId=20024&page=1&perPage=100&rd=99999&searchSource=PAGINATION&showMore=false&sort=relevance&stkTypId=28880&zc=11209'
response = requests.get(crawl_url, headers=headers )


cars = soup.find_all('span', attrs={'class': 'listing-row__price'})

for car in cars:
    print(re.sub(r'\s+', '', ''.join([car.text])))

输出

$71,412  
$75,476  
$77,001  
$77,822  
$107,271 
...