如何从 html 表格元素中解析文本答案

【问题标题】：How to parse text from a html table element如何从 html 表格元素中解析文本
【发布时间】：2016-12-05 21:38:27
【问题描述】：

我目前正在使用 python 请求和 lxml 库编写一个小型测试网络爬虫。我正在尝试使用 xpaths 从this site 的表格行中提取文本来唯一标识表格。由于表本身只能通过其类名来标识，并且考虑到类名不是唯一的，我不得不使用父 div 元素来指定表。有问题的表格列出了《权力的游戏》节目的季节顺序、拍摄和播出日期，我试图通过以下路径选择：

tree.xpath('//div[@id = "mw-content-text"]//table[@class = "wikitable"]//text()')

由于某种原因，当我在 shell 中打印此路径时，它返回一个空列表。我相信打印这条路径只会显示我试图做的表格中的所有文本，以确保我能真正获得内容；但是，我实际上需要打印表格的每一行。

这个 xpath 有问题吗？如果是这样，打印表格内容的正确方法是什么？

【问题讨论】：

标签： python html xpath python-requests lxml

【解决方案1】：

wikitable 的类别过于宽泛，无法区分 Wiki 页面上的表格。

我会改为依赖前面的Adaptation schedule 标签：

import requests
from lxml.html import fromstring

url = "https://en.wikipedia.org/wiki/Game_of_Thrones"
response = requests.get(url)
root = fromstring(response.content)

table = root.xpath(".//h3[span = 'Adaptation schedule']/following-sibling::table")[0]
for row in table.xpath(".//tr")[1:]:
    print([cell.text_content() for cell in row.xpath(".//td")])

打印：

['Season 1', 'March 2, 2010[52]', 'Second half of 2010', 'April 17, 2011', 'June 19, 2011', 'A Game of Thrones']
['Season 2', 'April 19, 2011[53]', 'Second half of 2011', 'April 1, 2012', 'June 3, 2012', 'A Clash of Kings and some early chapters from A Storm of Swords[54]']
['Season 3', 'April 10, 2012[55]', 'Second half of 2012', 'March 31, 2013', 'June 9, 2013', 'About the first two-thirds of A Storm of Swords[56][57]']
['Season 4', 'April 2, 2013[58]', 'Second half of 2013', 'April 6, 2014', 'June 15, 2014', 'The remaining one-third of A Storm of Swords and some elements from A Feast for Crows and A Dance with Dragons[59]']
['Season 5', 'April 8, 2014[60]', 'Second half of 2014', 'April 12, 2015', 'June 14, 2015', 'A Feast for Crows, A Dance with Dragons and original content,[61] with some late chapters from A Storm of Swords[62] and elements from The Winds of Winter[63][64]']
['Season 6', 'April 8, 2014[60]', 'Second half of 2015', 'April 24, 2016', 'June 26, 2016', 'Original content and outlined from The Winds of Winter,[65][66] with some late elements from A Feast for Crows and A Dance with Dragons[67]']
['Season 7', 'April 21, 2016[50]', 'Second half of 2016[49]', 'Mid-2017[5]', 'Mid-2017[5]', 'Original content and outlined from The Winds of Winter and A Dream of Spring[66]']

【讨论】：