如何将抓取的多行内容转换为列表？

Question

我试图将抓取的内容转换为列表以进行数据操作，但出现以下错误：TypeError: 'NoneType' object is not callable

#! /usr/bin/python

from urllib import urlopen
from BeautifulSoup import BeautifulSoup
import os
import re

# Copy all of the content from the provided web page
webpage = urlopen("http://www.optionstrategist.com/calculators/free-volatility-    data").read()

# Grab everything that lies between the title tags using a REGEX
preBegin = webpage.find('<pre>') # Locate the pre provided
preEnd = webpage.find('</pre>') # Locate the /pre provided

# Copy the content between the pre tags
voltable = webpage[preBegin:preEnd] 

# Pass the content to the Beautiful Soup Module
raw_data = BeautifulSoup(voltable).splitline()

Answer 1

代码很简单。这是 BeautifulSoup4 的代码：

# Find all <pre> tag in the HTML page
preTags = webpage.find_all('pre')

for tag in preTags:
    # Get the text inside the tag
    print(tag.get_text())

参考：

• find_all()
• Kinds of filters to put into name field of find()/findall()
• get_text()

Answer 2

从第一个 pre 元素中获取文本：

#!/usr/bin/env python
from urllib2 import urlopen
from BeautifulSoup import BeautifulSoup

url = "http://www.optionstrategist.com/calculators/free-volatility-data"
soup = BeautifulSoup(urlopen(url))
print soup.pre.string

提取带有数据的行：

from itertools import dropwhile

lines = soup.pre.string.splitlines()
# drop lines before the data table header
lines = dropwhile(lambda line: not line.startswith("Symbol"), lines)
# extract lines with data
lines = (line for line in lines if '%ile' in line)

现在每一行都包含固定列格式的数据。您可以使用切片和/或正则表达式来解析/验证每行中的各个字段。