在不同的列中分离 Python Web 抓取的数据 (Excel)

Question

尊敬的 Stackoverflow 社区，

最近我开始玩 Python。通过观看 YouTube 视频和浏览这个平台，我学到了很多东西。但我无法解决我的问题。

希望你们能帮帮我。

所以我尝试使用 Python(Anaconda) 从网站上抓取信息。并将这些信息放在一个 CSV 文件中。我试图通过在我的脚本中添加“，”来分隔列。但是当我打开我的 CSV 文件时，所有数据都放在 1 列（A）中。相反，我希望将数据分隔在不同的列中（A 和 B（以及 C、D、E、F 等，当我想添加信息时））。

我必须在这段代码中添加什么：

filename = "brands.csv"
f = open(filename, "w")

headers = "brand, shipping\n"

f.write(headers)

for container in containers:
    brand_container = container.findAll("h2",{"class":"product-name"})
    brand = brand_container[0].a.text

    shipping_container = container.findAll("p",{"class":"availability in-stock"})
    shipping = shipping_container[0].text.strip()

    print("brand: " + brand)
    print("shipping: " + shipping)

    f.write(brand + "," + shipping +  "," + "\n")

f.close()

感谢您的帮助！

亲切的问候，

---

根据Game0ver的建议完成脚本：

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = 'https://www.scraped-website.com'

# opening up connection, grabbing the page
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

# html parsing
page_soup = soup(page_html, "html.parser")   

# grabs each product
containers = page_soup.findAll("li",{"class":"item last"})
container = containers[0]

import csv

filename = "brands.csv"
with open(filename, 'w') as csvfile:
    fieldnames = ['brand', 'shipping']
    # define your delimiter
    writer = csv.DictWriter(csvfile, delimiter=',', fieldnames=fieldnames)
    writer.writeheader()

for container in containers:
    brand_container = container.findAll("h2",{"class":"product-name"})
    brand = brand_container[0].a.text

    shipping_container = container.findAll("p",{"class":"availability in-stock"})
    shipping = shipping_container[0].text.strip()

    print("brand: " + brand)
    print("shipping: " + shipping)

正如我提到的，这段代码不起作用。我一定是做错了什么？

Answer 1

您最好使用python's csv module 来做到这一点：

import csv

filename = "brands.csv"
with open(filename, 'w') as csvfile:
    fieldnames = ['brand', 'shipping']
    # define your delimiter
    writer = csv.DictWriter(csvfile, delimiter=',', fieldnames=fieldnames)
    writer.writeheader()
    # write rows...

Answer 2

尝试用双引号将您的值括起来，例如

f.write('"'+brand + '","' + shipping +  '"\n')

虽然，有更多更好的方法来处理这个通用任务和这个功能。

Answer 3

您可以选择我在下面显示的任何一种方式。由于您的脚本中可用的 url 无法访问，因此我提供了一个有效的。

import csv
import requests
from bs4 import BeautifulSoup

url = "https://yts.am/browse-movies"

response = requests.get(url)
soup = BeautifulSoup(response.content, 'lxml')

with open("movieinfo.csv", 'w', newline="") as f:
    writer = csv.DictWriter(f, ['name', 'year'])
    writer.writeheader()

    for row in soup.select(".browse-movie-bottom"):
        d = {}
        d['name'] = row.select_one(".browse-movie-title").text
        d['year'] = row.select_one(".browse-movie-year").text
        writer.writerow(d)

或者你可以尝试如下：

soup = BeautifulSoup(response.content, 'lxml')

with open("movieinfo.csv", 'w', newline="") as f:
    writer = csv.writer(f)
    writer.writerow(['name','year'])

    for row in soup.select(".browse-movie-bottom"):
        name = row.select_one(".browse-movie-title").text
        year = row.select_one(".browse-movie-year").text
        writer.writerow([name,year])