【发布时间】:2020-07-21 06:53:09
【问题描述】:
import pandas as pd
import requests
import time
from time import ctime
from bs4 import BeautifulSoup
count=0
while(1):
page =requests.get("https://www.flipkart.com/apple-iphone-11-black-64-gb/p/itm0f37c2240b217?pid=MOBFKCTSVZAXUHGR&lid=LSTMOBFKCTSVZAXUHGREPBFGI&marketplace=FLIPKART&srno=s_1_1&otracker=AS_QueryStore_OrganicAutoSuggest_4_5_na_na_na&otracker1=AS_QueryStore_OrganicAutoSuggest_4_5_na_na_na&fm=SEARCH&iid=7152a9e3-bb03-43e4-883d-b366d856ad66.MOBFKCTSVZAXUHGR.SEARCH&ppt=sp&ppn=sp&ssid=tn7ys3sre80000001595289428691&qH=f6cdfdaa9f3c23f3")
soup = BeautifulSoup(page.content, 'html.parser')
product_name = soup.find(class_='_35KyD6').get_text()
price = soup.find(class_='_1vC4OE _3qQ9m1').get_text()
product_details = pd.DataFrame({
'Name':product_name,
'Price':price,
'Time':time.ctime()},
index=[count])
product_details.to_csv('flipkart_scrapper.csv')
count+=1
print(count)
time.sleep(1)
当我打印 product_details 时,我得到的价格是每秒递增的索引和时间,但是当我尝试将其写入 CSV 时,只出现一行。
【问题讨论】:
标签: python python-3.x pandas web-scraping