【问题标题】:Webscraping in Python - I am getting special characters in price column but there is no special character in the webpagePython 中的网页抓取 - 我在价格列中收到特殊字符,但网页中没有特殊字符
【发布时间】:2021-09-12 08:31:07
【问题描述】:

我正在抓取的网页是 books.toscrape.com 我的代码是:

import requests
from bs4 import BeautifulSoup
import csv
import pandas as pd
import os
URL = 'https://books.toscrape.com/catalogue/page-'

books=[]
Price=[]
Stock_availability=[]
Books_url=[]
# For loop to crawl and get information from multiple pages.
for page in range(1,6):
    Source = requests.get(URL + str(page) + '.html')
    Scrape = BeautifulSoup(Source.text,'html.parser')
    #print(Scrape.prettify()) 

    # Grabbing book titles
    for article in Scrape.find_all('article'):
        books.append(article.h3.a.text)
        #print(books)
    #Grabbing book price
        Price.append(article.find('div', class_ = 'product_price').p.text)
        #print(Price)
    #Grabbing stock availability of each book
        Stock = Scrape.find('p', class_ = 'instock availability').text.strip()
        Stock_availability.append(Stock)
        #print(Stock_availability)
        
    #Grabbing the link of each book
        for link in article.find_all('a',href = True):
            url = link['href']
        Books_url.append('https://books.toscrape.com/catalogue/' + url)
        #print(Books_url)   
         
Scraped_Data = {'TITLE':books,'PRICE':Price,'STOCK AVAILABILTY':Stock_availability,'URL':Books_url}
Scraped_Books = pd.DataFrame(Scraped_Data)
Scraped_Books[:5]
Scraped_Books.to_csv('SCB.csv',index = None)

#--------- #我的输出显示价格前的特殊字符“”

#0     £51.77
#1     £53.74
#2     £50.10
#3     £47.82
#4     £54.23 

我的 CSV 文件导出再次显示相同的特殊字符,后跟一个逗号 Â, Â £51.77 -- 但网页中除了货币符号外没有特殊字符。

请问我的代码有什么问题?感谢您的宝贵时间!

【问题讨论】:

  • 这是一个编码问题。试试Scrape = BeautifulSoup(Source.content,'html.parser')
  • 这几乎可以肯定是 UTF-8 被解释为 Windows 代码页 1252。可能在其周围寻找重复;这是一个非常常见的常见问题解答。

标签: python python-3.x web-scraping


【解决方案1】:

这是一个编码问题。这应该可以解决它:

Price.append(article.find('div', class_ = 'product_price').p.text.encode('latin-1').decode('utf-8'))

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2014-12-17
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2018-10-10
    • 1970-01-01
    相关资源
    最近更新 更多