【问题标题】:Scrape Table Data into .csv将表数据刮入 .csv
【发布时间】:2017-09-20 13:12:21
【问题描述】:

我能够打印出表格,但无法将其提取到 .csv 文件中。我每天都是抓取和学习的新手。

如何将这些数据截屏到 CSV 文件?

标准库模块 导入操作系统 导入系统

# The wget module
import wget

# The BeautifulSoup module
from bs4 import BeautifulSoup

# The selenium module
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

driver = webdriver.Chrome() # if you want to use chrome, replace Firefox()    with Chrome()
driver.get("https://www.arcountydata.com/county.asp?county=Benton") # load the web page
search_begin = driver.find_element_by_xpath("//*[@id='Assessor']/div/div[2]/a/i").click()
# for websites that need you to login to access the information
elem = driver.find_element_by_id("OwnerName") # Find the email input field of the login form
elem.send_keys("Roth Family Inc") # Send the users email

search_exeute = driver.find_element_by_xpath("//*[@id='Search']").click()


src = driver.page_source # gets the html source of the page

parser = BeautifulSoup(src,"lxml") # initialize the parser and parse the source "src"


table = parser.find("table", attrs={"class" : "table table-striped-yellow     table-hover table-bordered table-condensed table-responsive"}) # A list of attributes that you want to check in a tag)

f = open('/output.csv', 'w')

parcel=""
owner=""
ptype=""
site_address=""
location=""
acres=""

summons =[]
#print table

list_of_rows = []
for row in table.findAll('tr')[1:]:
    list_of_cells = []
    for cell in row.findAll('td'):
        text = cell.text.replace(" ", "")
        list_of_cells.append(text)
    list_of_rows.append(list_of_cells)
print list_of_rows

driver.close() # closes the driver ?>

【问题讨论】:

    标签: python python-2.7 selenium web-scraping beautifulsoup


    【解决方案1】:

    很高兴见到新成员,祝你好运,先生! 你做得很棒,你很好地评论了你的代码,这意味着你理解它,干得好!

    import csv
    

    这是python中的一个模块,可以轻松读取/写入CSV文件,所以让我们先导入它。

      with open(name_csv+'.csv', 'w+') as csvfile:
          spamwriter = csv.writer(csvfile, delimiter=',')
          spamwriter.writerow(list_of_rows)
    #Used to write 1 row, each element in the array will be seperated by a comma
    

    编辑:

    with open('somecsv.csv','w+') as csvfile:
        spamwriter = csv.writer(csvfile, delimiter='|') # Changed the delimiter (Way of separating )
        # This opens the CSV file and we set some additional parameters
        for row in table.findAll('tr')[::2]:
            list_of_cell = []
            for cell in row.findAll('td')[:5]:
                text = cell.text.replace(" ", "").strip()
                text = text.replace('''...\n\n\n'''," |") #This one is added so it replaces string before Lot with comma
                text = text.replace(''':\n''',':') #This one is added so it doesn't interfere
                text = text.replace('''\n''','|') #Adds a comma before block
                list_of_cell.append(text)
            print(list_of_cell)
            spamwriter.writerow(list_of_cell)
    

    【讨论】:

    • 感谢错误.. 它有帮助,因为我现在能够将数据解析为 csv。然而,整个数据被解析为 1 行。我应该怎么做才能逐行解析,如 html 表中所示...
    • 这应该可以,别忘了导入 csv。我也会考虑添加选择 Selenium。
    • 非常感谢先生..它帮助我今天学到了一些新东西..我会尝试进一步清理代码...因为某些数据是一个被解析为多行的列..
    • 我错过了什么.. csv 中的输出文件为 html 表中的“位置”字段中的值引入了空白行
    • 哦,有问题,我发现TR里面还有一个TR所以有空格
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2019-07-13
    • 2013-12-21
    • 1970-01-01
    • 1970-01-01
    • 2020-06-10
    • 2011-06-29
    相关资源
    最近更新 更多