使用python从nse网站抓取股票期货数据答案

【问题标题】：scraping stock futures data from nse website using python使用python从nse网站抓取股票期货数据
【发布时间】：2021-10-10 16:25:12
【问题描述】：

这是我的代码

url_oc = "https://www.nseindia.com/get-quotes"
url = f"https://www.nseindia.com/get-quotes/derivatives?symbol=WIPRO"
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) 
             AppleWebKit/537.36 (KHTML, '
                     'like Gecko) '
                     'Chrome/80.0.3987.149 Safari/537.36',
       'accept-language': 'en,gu;q=0.9,hi;q=0.8', 'accept-encoding': 
          'gzip, deflate, br'}
session = requests.Session()
request = session.get(url_oc, headers=headers, timeout=5)
cookies = dict(request.cookies)
response = session.get(url, headers=headers, timeout=5, cookies=cookies).json()

enter image description here

**我无法获取数据

出现错误**

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

【问题讨论】：

标签： python-3.x web-scraping

【解决方案1】：

这是从 api 获取数据的完整示例。

import requests
import pandas as pd
import json
data=[]

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36',}
with requests.session() as req:
    req.get('https://www.nseindia.com/get-quotes/derivatives?symbol=WIPRO',headers = headers)

    api_req=req.get('https://www.nseindia.com/api/quote-derivative?symbol=WIPRO',headers = headers).json()
    for item in api_req['stocks']:
        data.append([
            item['metadata']['instrumentType'],
            item['metadata']['openPrice']])


cols=['instrumentType','openPrice']

df = pd.DataFrame(data, columns=cols)
print(df)
#df.to_csv('info.csv',index = False)

输出：

     instrumentType   openPrice
0    Stock Futures     646.15
1    Stock Options      12.05
2    Stock Options      23.00
3    Stock Options      29.00
4    Stock Options      20.40
..             ...        ...
170  Stock Options       0.00
171  Stock Options       0.00
172  Stock Options       0.00
173  Stock Options       0.00
174  Stock Options       0.00

[175 rows x 2 columns]

【讨论】：

如何只提取所有到期的期货数据

【解决方案2】：

要从该页面获取 JSON 数据，请使用正确的 API URL：

import json
import requests


api_url = "https://www.nseindia.com/api/quote-derivative?symbol=WIPRO"
headers = {
    "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:93.0) Gecko/20100101 Firefox/93.0",
}

with requests.session() as s:
    # load cookies
    s.get(
        "https://www.nseindia.com/get-quotes/derivatives?symbol=WIPRO",
        headers=headers,
    )
    data = s.get(api_url, headers=headers).json()

# pretty print:
print(json.dumps(data, indent=4))

打印：

{
    "info": {
        "symbol": "WIPRO",
        "companyName": "Wipro Limited",
        "industry": "COMPUTERS - SOFTWARE",
        "activeSeries": [
            "EQ"
        ],
        "debtSeries": [],
        "tempSuspendedSeries": [],
        "isFNOSec": true,
        "isCASec": false,
        "isSLBSec": true,
        "isDebtSec": false,
        "isSuspended": false,
        "isETFSec": false,
        "isDelisted": false,
        "isin": "INE075A01022"
    },
    "underlyingValue": 661.95,
    "vfq": 32001,
    "fut_timestamp": "08-Oct-2021 15:30:24",
    "opt_timestamp": "08-Oct-2021 15:30:13",
    "stocks": [
        {
            "metadata": {
                "instrumentType": "Stock Futures",
                "expiryDate": "28-Oct-2021",
                "optionType": "-",
                "strikePrice": 0,
                "identifier": "FUTSTKWIPRO28-10-2021XX0.00",
                "openPrice": 646.15,
                "highPrice": 669,
                "lowPrice": 645.95,
                "closePrice": 662.25,
                "prevClose": 645.65,
                "lastPrice": 662.75,
                "change": 17.100000000000023,
                "pChange": 2.6484937659722796,
                "numberOfContractsTraded": 11655,
                "totalTurnover": 123179.36
            },

...

【讨论】：

如何只提取所有到期的期货数据