在python中解析html到csv数据

Question

我正在使用 python 将 html 表格转换为 CSV 格式。

代码：

from BeautifulSoup import BeautifulSoup
import sys
import csv
import argparse

reload(sys)
sys.setdefaultencoding('utf-8')
parser = argparse.ArgumentParser(description='Reads in an HTML and attempts to convert all tables into CSV files.')
parser.add_argument('--delimiter', '-d', action='store', default=',',help="Character with which to separate CSV columns")
parser.add_argument('--quotechar', '-q', action='store', default='"',help="Character within which to nest CSV text")
parser.add_argument('filename',nargs="?",help="HTML file from which to extract tables")
args = parser.parse_args()

if sys.stdin.isatty() and not args.filename:
  parser.print_help()
  sys.exit(-1)
elif not sys.stdin.isatty():
  args.filename = sys.stdin
else:
  args.filename = open(sys.argv[1],'r')

print "Opening file"
fin  = args.filename.read()

print "Parsing file"
soup = BeautifulSoup(fin,convertEntities=BeautifulSoup.HTML_ENTITIES)

print "Preemptively removing unnecessary tags"
[s.extract() for s in soup('script')]

print "CSVing file"
tablecount = -1
for table in soup.findAll("table"):
  tablecount += 1
  print "Processing Table #%d" % (tablecount)
  with open(sys.argv[1]+str(tablecount)+'.csv', 'wb') as csvfile:
        fout = csv.writer(csvfile, delimiter=args.delimiter, quotechar=args.quotechar, quoting=csv.QUOTE_MINIMAL)
        for row in table.findAll('tr'):
          cols = row.findAll(['td','th'])
          if cols:
            cols = [x.text for x in cols]
            fout.writerow(cols)

在这里，我不想显式提供 sys 参数，而是希望在脚本中使用文件名对其进行硬编码。 目前的用法是 - python html2csv.py test.html。

有什么办法可以做到吗？

错误：

File "html2csv.py", line 17, in <module>
if sys.stdin.isatty() and not args.filename:
AttributeError: 'Namespace' object has no attribute 'filename'

Answer 1

看来，打开的文件对象被写入变量args.filename

我会尝试（快速且非常肮脏）在

之前添加

print "Opening file"
fin  = args.filename.read()

一行

args.filename = open('test.html', 'r')

解析器可能会抱怨您没有将文件名传递给命令，但可能不是，请尝试:)