从 Coindix 抓取答案

【问题标题】：Scraping from Coindix从 Coindix 抓取
【发布时间】：2022-01-20 06:09:12
【问题描述】：

我对 Python 还很陌生，所以需要一些指导。我想通过抓取从https://coindix.com/?sort=-tvl 中提取“名称、协议、APY、TVL”数据（因为我相信没有 API），但是有一些问题。当我在下面执行时：

import requests
from bs4 import BeautifulSoup

url = "https://coindix.com/?sort=-tvl"
r = requests.get(url)
soup = BeautifulSoup(r.content, "html.parser")

返回的数据不包含我想获取的信息。有人可以帮忙吗？

【问题讨论】：

请编辑问题以将其限制为具有足够详细信息的特定问题，以确定适当的答案。

标签： python web-scraping

【解决方案1】：

有一个 API。在 Dev Tools -> Network -> XHR -> Headers 中找到 url

import requests
import pandas as pd

url = 'https://apiv2.coindix.com/search'
payload = {
    'sort': '-tvl',
    'first': 'true',
    'screen': '1114'}  

data = requests.get(url, params=payload).json()['data']  
df = pd.DataFrame(data)

输出：

print(df.head(5).to_string())
      id           name                                               icon     chain protocol      base  reward          rewards       apy  apy_7_day         tvl  risk                                            link  is_new
0  17419            UST            https://apiv2.coindix.com/icons/UST.png     Terra   Anchor  0.193600  0.0000               {}  0.193600    0.19570  5977961341     2  https://apiv2.coindix.com/vault/17419/redirect   False
1  17206  DAI-USDC-USDT  https://apiv2.coindix.com/icons/DAI-USDC-USDT.png  Ethereum    Curve  0.002800  0.0087  {'CRV': 0.0087}  0.011500    0.01210  5952854016     1  https://apiv2.coindix.com/vault/17206/redirect   False
2  17174           LUNA           https://apiv2.coindix.com/icons/LUNA.png     Terra     Lido  0.079000  0.0000               {}  0.079000    0.07900  5534798290     1  https://apiv2.coindix.com/vault/17174/redirect   False
3  15940            ETH            https://apiv2.coindix.com/icons/ETH.png  Ethereum     Lido  0.047000  0.0000               {}  0.047000    0.04700  5347746431     1  https://apiv2.coindix.com/vault/15940/redirect   False
4  13517      cUSD-cEUR      https://apiv2.coindix.com/icons/cUSD-cEUR.png      Celo    Sushi  0.002466  0.0000               {}  0.002466    0.01058  4609514119     2  https://apiv2.coindix.com/vault/13517/redirect   False

[100 rows x 14 columns]

【讨论】：