【发布时间】:2021-09-12 08:31:07
【问题描述】:
我正在抓取的网页是 books.toscrape.com 我的代码是:
import requests
from bs4 import BeautifulSoup
import csv
import pandas as pd
import os
URL = 'https://books.toscrape.com/catalogue/page-'
books=[]
Price=[]
Stock_availability=[]
Books_url=[]
# For loop to crawl and get information from multiple pages.
for page in range(1,6):
Source = requests.get(URL + str(page) + '.html')
Scrape = BeautifulSoup(Source.text,'html.parser')
#print(Scrape.prettify())
# Grabbing book titles
for article in Scrape.find_all('article'):
books.append(article.h3.a.text)
#print(books)
#Grabbing book price
Price.append(article.find('div', class_ = 'product_price').p.text)
#print(Price)
#Grabbing stock availability of each book
Stock = Scrape.find('p', class_ = 'instock availability').text.strip()
Stock_availability.append(Stock)
#print(Stock_availability)
#Grabbing the link of each book
for link in article.find_all('a',href = True):
url = link['href']
Books_url.append('https://books.toscrape.com/catalogue/' + url)
#print(Books_url)
Scraped_Data = {'TITLE':books,'PRICE':Price,'STOCK AVAILABILTY':Stock_availability,'URL':Books_url}
Scraped_Books = pd.DataFrame(Scraped_Data)
Scraped_Books[:5]
Scraped_Books.to_csv('SCB.csv',index = None)
#--------- #我的输出显示价格前的特殊字符“”
#0 £51.77
#1 £53.74
#2 £50.10
#3 £47.82
#4 £54.23
我的 CSV 文件导出再次显示相同的特殊字符,后跟一个逗号 Â, Â £51.77 -- 但网页中除了货币符号外没有特殊字符。
请问我的代码有什么问题?感谢您的宝贵时间!
【问题讨论】:
-
这是一个编码问题。试试
Scrape = BeautifulSoup(Source.content,'html.parser') -
这几乎可以肯定是 UTF-8 被解释为 Windows 代码页 1252。可能在其周围寻找重复;这是一个非常常见的常见问题解答。
标签: python python-3.x web-scraping