使用 BeautifulSoup 解析数据并使用 pandas DataFrame to_csv 对数据进行排序

Question

我的目标是解析来自网站的数据，并将这些数据存储在文本文件中，格式化后可以在 Excel 中打开。

代码如下：

from bs4 import BeautifulSoup
import requests
import pprint
import re
import pyperclip
import json
import pandas as pd
import csv

pag = range (2, 126)
out_file = open('bestumbrellasoffi.txt', 'w', encoding='utf-8')
with open('bestumbrellasoffi.txt', 'w', encoding='utf-8') as file:
    for x in pag:

    # Iterate pages
        url = 'https://www.paginegialle.it/ricerca/lidi%20balneari/italia/p-' + str(x) + '?mr=50'
        response = requests.get(url)
        soup = BeautifulSoup(response.content, "html.parser")

        # Parse data
        for i, j, k, p, z in zip(soup.find_all('span', attrs=
{'itemprop': 'name'}), soup.find_all('span', attrs=
{'itemprop': 'longitude'}), soup.find_all('span', attrs=
{'itemprop': 'latitude'}), soup.find_all('span', attrs = {'class': 'street-
address'}), soup.find_all('div', attrs = {'class': 'tel elementPhone'})):
            info = i.text, j.text, k.text, p.text, z.text

            # Check if data is good
            print(url)
            print (info)

            # Create dataframe
            raw_data = { 'nome': [i], 'longitudine': [j], 'latitudine':
            [k], 'indirizzo': [p], 'telefono': [z]}
            print(raw_data)
            df = pd.DataFrame(raw_data, columns =
               ['nome', 'longitudine', 'latitudine', 'indirizzo', 'telefono'])
            df.to_csv('bestumbrellasoffi.txt')

out_file.close()

所有这些模块都是因为我尝试了很多次。

所以输出

print(info)  [is][1]

输出

打印（原始数据）is

---

这是经过审查且运行良好的代码。

from bs4 import BeautifulSoup
import requests
import pprint
import re
import pyperclip
import json
import pandas as pd
import csv

pag = range (2, 126)
with open('bestumbrellasoffia.txt', 'a', encoding='utf-8') as file:

    for x in pag:

    # Iterate pages
        url = 'https://www.paginegialle.it/ricerca/lidi%20balneari/italia/p-' + str(x) + '?mr=50'
        response = requests.get(url)
        soup = BeautifulSoup(response.content, "html.parser")
        raw_data = { 'nome': [], 'longitudine': [], 'latitudine': [], 'indirizzo': [], 'telefono': []}
        df = pd.DataFrame(raw_data, columns = ['nome', 'longitudine', 'latitudine', 'indirizzo', 'telefono'])

        # Parse data
        for i, j, k, p, z in zip(soup.find_all('span', attrs = {'itemprop': 'name'}), soup.find_all('span', attrs = {'itemprop': 'longitude'}), soup.find_all('span', attrs = {'itemprop': 'latitude'}), soup.find_all('span', attrs = {'class': 'street-address'}), soup.find_all('div', attrs = {'class': 'tel elementPhone'})):
            inno = i.text.lstrip()
            ye = inno.rstrip()
            info = ye, j.text, k.text, p.text, z.text

            # Check if data is good
            print(info)

            # Create dataframe
            raw_data = { 'nome': [i], 'longitudine': [j], 'latitudine': [k], 'indirizzo': [p], 'telefono': [z]}

            # Try dataframe
            #print(raw_data)
            file.write(str(info) + "\n")

Answer 1

您在 for 循环的每次迭代中都初始化一个新的数据框。

您应该在循环外对其进行初始化，然后在循环中使用 append() 来添加每一行数据。