【问题标题】:How can I convert all percentages into decimals, and write that to CSV?如何将所有百分比转换为小数,并将其写入 CSV?
【发布时间】:2017-11-03 03:02:01
【问题描述】:

目标是将所有值从百分比转换为小数形式。代码如下:

import requests
from bs4 import BeautifulSoup
import lxml


FIU = open('C://Users//joey//Desktop//response.txt','r').read()
#soup = BeautifulSoup(FIU, "html.parser")


soup = BeautifulSoup(FIU, "lxml")

tables = soup.find_all('table')

for table in tables:
    rows = table.find_all("tr")
    for row in rows:
        cells = row.find_all("td")
        if len(cells) == 7:  # this filters out rows with 'Term', 'Instructor Name' etc.
            for cell in cells:
                print(cell.text + "\t", end="")  # \t is a Tab character, and end="" prevents a newline between cells
            print("")  # newline after each row



def p2f(x): return float(x.strip('%'))/100
percentage_list = []
for cell in cells:
    if '%' in cell.text:
        percentage_list.append(p2f(cell.text))

在最底部,您会看到我尝试去除百分比然后除以 100 以获得每个数字的小数的函数。但是,它并没有影响输出:

Description of course objectives and assignments    0.0%    68.4%   10.5%   15.8%   5.3%    0.0%    
Communication of ideas and information  0.0%    52.6%   26.3%   10.5%   10.5%   0.0%    
Expression of expectations for performance in this class    0.0%    68.4%   15.8%   10.5%   0.0%    5.3%    
Availability to assist students in or out of class  0.0%    57.9%   31.6%   10.5%   0.0%    0.0%    
Respect and concern for students    0.0%    47.4%   42.1%   10.5%   0.0%    0.0%    
Stimulation of interest in course   0.0%    47.4%   26.3%   21.1%   0.0%    5.3%    
Facilitation of learning    0.0%    52.6%   26.3%   10.5%   10.5%   0.0%    
Overall assessment of instructor    0.0%    52.6%   31.6%   10.5%   0.0%    5.3%

我可以实现什么代码来解决这个问题?

【问题讨论】:

  • 可以打印percentage_list吗?
  • @flamelite 我更新了我的帖子,所以它有输出,如果这就是你的意思?代码本身已经将其“打印”到控制台。

标签: python python-3.x function web-scraping


【解决方案1】:

在此处使用您的 p2f 函数:

def p2f(x): 
    return float(x.strip('%'))/100    
for table in tables:
    rows = table.find_all("tr")
    for row in rows:
        cells = row.find_all("td")
        if len(cells) == 7:
            for cell in cells:
                if '%' in cell.text:
                    print(str(p2f(cell.text)) + "\t", end="")
                else:
                    print(cell.text + "\t", end="")
                print("")  # newline after each row

【讨论】:

  • 这没有按预期工作。 imgur.com/a/2u5wd 不知道是什么问题,但它只是删除了所有数字,然后留下了一个小块。
  • 在调试报告中查看错误:NameError: name 'p2f' is not defined。
  • 首先定义你的 p2f 函数,然后在下面调用它。
  • 啊哎呀。忘记了。上面已经注释掉了。我把它带回来了,现在我有一个类型错误:imgur.com/a/p0niA No undefined function error, but a type error, and still empty output.
  • 再次看到错误,告诉您无法添加浮点类型数据和字符串。所以你需要将浮点类型数据类型转换为字符串类型数据。
【解决方案2】:

我想出了一种将它转换成字典的方法,但结果并没有我希望的那么干净

 one_table = {}
for row in rows:
    cells = row.find_all("td")
    name = cells[0].text
    one_table[name] = []
    if all('%' in cell.text for cell in cells[1:]):
        one_table[name] = [] #Create dictionary entry if this is a percentage row
    else:
        continue  #Otherwise, move on to the next row
    for cell in cells[1:]:
            one_table[name].append(p2f(cell.text))


with open('dict.csv', 'w') as csv_file:
    writer = csv.writer(csv_file)
    for key, value in one_table.items():
       writer.writerow([key, value])

print(one_table)

结果是:

{'Term: 1171 - Spring 2017': [], 'Instructor Name: Austin, Lathan Craig': [], 'Course: TRA   4721  ': [], 'Enrolled: 27': [], '\xa0': [], 'Question': [], 'Description of course objectives and assignments': [0.0, 0.684, 0.105, 0.158, 0.053, 0.0], 'Communication of ideas and information': [0.0, 0.526, 0.263, 0.105, 0.105, 0.0], 'Expression of expectations for performance in this class': [0.0, 0.684, 0.158, 0.105, 0.0, 0.053], 'Availability to assist students in or out of class': [0.0, 0.579, 0.316, 0.105, 0.0, 0.0], 'Respect and concern for students': [0.0, 0.474, 0.42100000000000004, 0.105, 0.0, 0.0], 'Stimulation of interest in course': [0.0, 0.474, 0.263, 0.21100000000000002, 0.0, 0.053], 'Facilitation of learning': [0.0, 0.526, 0.263, 0.105, 0.105, 0.0], 'Overall assessment of instructor': [0.0, 0.526, 0.316, 0.105, 0.0, 0.053]}

所以它确实写入了 CSV,但我在将其转换为字典时不知何故丢失了 99% 的数据。只有一张桌子,我以前有数百张。 CSV 中的输出也不太理想:

因此,如果我能以某种方式找到一种方法来包含我的所有数据,然后按照预期的方式用逗号分隔它,那么这可能就足够了。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2017-06-19
    • 2011-11-02
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2019-12-09
    • 1970-01-01
    相关资源
    最近更新 更多