遍历页面但它没有发生答案

【问题标题】：Iterate through pages but its not happening遍历页面但它没有发生
【发布时间】：2021-08-21 16:53:40
【问题描述】：

我需要遍历页面并获得结果。请看一下代码，是不是我做错了什么。

import requests
import pandas as pd
from bs4 import BeautifulSoup

url = "https://www.pakwheels.com/used-cars/search/-/?page="

for page in range(10):

    print('---', page, '---')

    r = requests.get(url + str(page))


soup = BeautifulSoup(r.content, "html.parser")

all_data = []
for ul in soup.select("ul.search-vehicle-info-2"):
    car_name = ul.find_previous("h3").get_text(strip=True)
    info = [li.get_text(strip=True) for li in ul.select("li")]
    all_data.append([car_name, *info])

df = pd.DataFrame(
    all_data, columns=["Car Name", "Year", "KM", "Type", "CC", "Type 2"]
)
print(df)
df.to_csv("data.csv", index=False)

它需要从 page=1 开始并在给定的范围内结束。

【问题讨论】：

标签： python pandas web-scraping

【解决方案1】：

要更正代码，请移动 all_data 顶部并修复缩进：

import requests
import pandas as pd
from bs4 import BeautifulSoup

url = "https://www.pakwheels.com/used-cars/search/-/?page="

all_data = []
for page in range(1, 11):
    print("---", page, "---")
    r = requests.get(url + str(page))

    soup = BeautifulSoup(r.content, "html.parser")

    for ul in soup.select("ul.search-vehicle-info-2"):
        car_name = ul.find_previous("h3").get_text(strip=True)
        info = [li.get_text(strip=True) for li in ul.select("li")][:5]
        all_data.append([car_name, *info])

df = pd.DataFrame(
    all_data, columns=["Car Name", "Year", "KM", "Type", "CC", "Type 2"]
)
print(df)
df.to_csv("data.csv", index=False)

保存data.csv（来自 LibreOffice 的屏幕截图）：

【讨论】：