用 Python 在一个类中包含一个列表中的 Web 抓取内容

Question

所以我有一个结构如下的网站：

请看图：https://i.stack.imgur.com/LZIzy.png

这是网站的链接：https://parramattamg.com.au/up4053-961230-mg-hs-2020.html

我正在尝试提取列表中的所有项目。

import bs4, requests
import pandas as PD

wagon_stock_url = 'https://parramattamg.com.au/up4053-961230-mg-hs-2020.html'

headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like 
Gecko) Chrome/96.0.4664.45 Safari/537.36'
}


response = requests.get(wagon_stock_url, headers = headers)
soup = bs4.BeautifulSoup(response.text, 'html.parser')  

name = soup.select(".stockItemInfo").

我知道汤。选择只打印类项目。之后需要帮助。

感谢您的帮助。

Answer 1

您接近解决方案 - 只需将 li 添加到您的 css selector，将为您提供所有列表元素的结果集：

name = soup.select(".stockItemInfo li")

--> [<li>    <span><strong>Vehicle</strong></span>: 2020 MG HS      </li>, <li>    <span><strong>Series</strong></span>: SAS23 MY20     </li>, <li>    <span><strong>Badge</strong></span>: Vibe DCT FWD        </li>, <li>    <span><strong>Colour</strong></span>: White      </li>, <li>    <span><strong>Odometer</strong></span>: 11,213kms        </li>, <li>    <span><strong>Body</strong></span>: Wagon        </li>, <li>    <span><strong>Engine</strong></span>: 1.5 litre, 4-cylinder      </li>, <li>    <span><strong>Fuel Type</strong></span>: Petrol      </li>, <li>    <span><strong>Transmission</strong></span>: 7-speed Automatic        </li>, <li>    <span><strong>Doors</strong></span>: 5-door      </li>, <li>    <span><strong>Seats</strong></span>: 5       </li>, <li>    <span><strong>Trim</strong></span>: Black        </li>, <li>    <span><strong>VIN</strong></span>: LSJA24U92LN012249     </li>, <li>    <span><strong>Registration</strong></span>: EIT61T       </li>, <li>    <span><strong>Stock Number</strong></span>: UP4053       </li>, <li>    <span><strong>MY</strong></span>: 20     </li>]

或仅获取名称作为列表：

names = [x.text for x in soup.select(".stockItemInfo li strong")]

--> ['Vehicle', 'Series', 'Badge', 'Colour', 'Odometer', 'Body', 'Engine', 'Fuel Type', 'Transmission', 'Doors', 'Seats', 'Trim', 'VIN', 'Registration', 'Stock Number', 'MY']

获取包含名称和值的字典列表

如果您想发布流程，请推送到pd.DataFrame(data)，...

data = []
for x in soup.select(".stockItemInfo li"):
    item = x.text.strip().split(':')
    data.append({
        'name': item[0],
        'value': item[1]
    })
    
data

输出

 [{'name': 'Vehicle', 'value': ' 2020 MG HS'},
 {'name': 'Series', 'value': ' SAS23 MY20'},
 {'name': 'Badge', 'value': ' Vibe DCT FWD'},
 {'name': 'Colour', 'value': ' White'},
 {'name': 'Odometer', 'value': ' 11,213kms'},
 {'name': 'Body', 'value': ' Wagon'},
 {'name': 'Engine', 'value': ' 1.5 litre, 4-cylinder'},
 {'name': 'Fuel Type', 'value': ' Petrol'},
 {'name': 'Transmission', 'value': ' 7-speed Automatic'},
 {'name': 'Doors', 'value': ' 5-door'},
 {'name': 'Seats', 'value': ' 5'},
 {'name': 'Trim', 'value': ' Black'},
 {'name': 'VIN', 'value': ' LSJA24U92LN012249'},
 {'name': 'Registration', 'value': ' EIT61T'},
 {'name': 'Stock Number', 'value': ' UP4053'},
 {'name': 'MY', 'value': ' 20'}]