使用python请求和bs4获取表中的所有超链接

Question

这个网站上有一张桌子https://www.investing.com/commodities/real-time-futures。表类 id 是 cross_rate_1。

我正在尝试获取与位于 //*[@id="cross_rate_1"]/tbody/tr[2]/td[2]/a 的表格标题中的每个项目相关联的所有超链接。

每个项目的标签位置是td class="bold left plusIconTd noWrap elp"，其中位于a title 和href。

我尝试了以下代码

urlheader = {
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36",
    "X-Requested-With": "XMLHttpRequest"
}

url = "https://www.investing.com/commodities/real-time-futures"
req = requests.get(url, headers=urlheader)
soup = BeautifulSoup(req.content, "lxml")
table = soup.find('table', id="cross_rate_1")

但我只拿到了桌子本身，我也试过了，

links= soup.findAll("td", { "class" : "href" },)

但它是空白的。

如何创建如下所示的表：

Commodity  Hyperlink 
Gold       https://www.investing.com/commodities/gold
XAU/USD    https://www.investing.com/currencies/xau-usd
.....

Answer 1

很简单：

import requests
from bs4 import BeautifulSoup

urlheader = {
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36",
    "X-Requested-With": "XMLHttpRequest"
}

url = "https://www.investing.com/commodities/real-time-futures"
req = requests.get(url, headers=urlheader)
soup = BeautifulSoup(req.content, "lxml")
table = soup.find('table', id="cross_rate_1")

for a in table.findAll('a'):
    text = a.text
    url = a.get("href")
    print(text, url)
    # Or do what you want