从 url 下载 csv 有时会失败

Question

我使用下面的代码从nasdaq.com 中提取历史数据，但失败了。

import pandas as pd
link = "https://www.nasdaq.com/api/v1/historical/BFC/stocks/2015-07-12/2020-07-12"
data = pd.read_csv(link, skiprows=2)
print(data)

然后我尝试了下面的另一个代码，但也失败了。

import csv
import requests

csv_url = 'https://www.nasdaq.com/api/v1/historical/BFC/stocks/2015-07-12/2020-07-12/'
req = requests.get(csv_url)
url_content = req.content
csv_file = open('downloaded.csv', 'wb')
csv_file.write(url_content)
csv_file.close()

但上面的代码适用于其他一些 url 链接，例如 https://www.ishares.com/de/professionelle-anleger/de/produkte/270048/ishares-msci-world-value-factor-ucits-etf/1478358465952.ajax?fileType=csv&fileName=IS3S_holdings&dataType=fund&asOfDate=20180731

谁能给点建议？

Answer 1

您需要为请求设置一个带有用户代理的标头 标题应该是dict格式

headers = {'User-Agent': 'Mozilla/5.0'}

所以你的请求代码将是：

import csv
import requests
headers = {'User-Agent': 'Mozilla/5.0'}
csv_url = 'https://www.nasdaq.com/api/v1/historical/BFC/stocks/2015-07-12/2020-07-12/'
req = requests.get(csv_url, headers=headers)
url_content = req.content
csv_file = open('downloaded.csv', 'wb')
csv_file.write(url_content)
csv_file.close()