从 JSON 中提取数据并使用 python 进行迭代

Question

所以我正在使用 HubSpot API 来跟踪系统中的“交易”，但没有一种强大的方法来搜索/过滤它返回的数据。因此，它只是将系统中的所有“交易”转储到一个大 JSON 中，然后为您提供一些分页信息，以帮助您在后端将它们重新粘合在一起。

这是一个演示/源 API URL： https://api.hubapi.com/deals/v1/deal/all?hapikey=demo

这返回的 JSON 基本上看起来像这样（缩短）：

{  
   "deals":[  
      {  
         "portalId":62515,
         "dealId":17886969,
         "isDeleted":false,
         "associations":{
            "associatedCompanyIds":[  
               113448746
            ]
         },
         "properties":{
            "dealname":{  
               "value":"Google Website"
            },
            "amount":{  
               "value":"150000"
            },
            "hubspot_owner_id":{  
               "value":"72"
            },
            "dealstage":{  
               "value":"qualifiedtobuy"
            },
            "dealtype":{  
               "value":"existingbusiness"
            }
         },
         "imports":[  

         ]
      }
   ],
   "hasMore":false,
   "offset":28692358
}

但是……

1) 有很多交易返回，我想用dealtype 过滤它，这是一个可以改变的变量。例如，我只想返回值为“qualifiedtobuy”的交易

2) 然后我需要根据dealId 对每笔交易运行一个查询并“做一些事情”。我想这意味着我需要将以上所有内容放入dict 并以某种方式进行迭代？我不确定。

这就是我目前所拥有的，但这实际上只是抓取 JSON 并将其转换为 dict（我认为）。

import requests
import json

apikey = "demo"
url = 'https://api.hubapi.com/deals/v1/deal/all?hapikey=' + apikey
response = requests.get(url)
response.raise_for_status()
jsonDeals = response.json()
dict_object = dict(jsonDeals)

我不确定接下来要采取什么步骤来获取 dealId 值，然后对返回的每个值进行“处理”。

非常感谢任何帮助。

更新：这里是原始字典，没有简化为一次性交易的属性：

{u'hs_lastmodifieddate': {u'sourceId': None, u'timestamp': 1457479099306, u'versions': [{u'timestamp': 1457479099306, u'sourceVid': [], u'name': u'hs_lastmodifieddate', u'value': u'1457479099306', u'source': u'CALCULATED'}], u'value': u'1457479099306', u'source': u'CALCULATED'}, u'pipeline': {u'sourceId': None, u'timestamp': 1457479063182, u'versions': [{u'timestamp': 1457479063182, u'name': u'pipeline', u'value': u'default', u'sourceVid': []}], u'value': u'default', u'source': None}, u'num_associated_contacts': {u'sourceId': None, u'timestamp': 0, u'versions': [{u'source': u'CALCULATED', u'name': u'num_associated_contacts', u'value': u'0', u'sourceVid': []}], u'value': u'0', u'source': u'CALCULATED'}, u'dealstage': {u'sourceId': None, u'timestamp': 1457479063157, u'versions': [{u'timestamp': 1457479063157, u'sourceVid': [], u'name': u'dealstage', u'value': u'qualifiedtobuy', u'source': u'API'}], u'value': u'qualifiedtobuy', u'source': u'API'}, u'createdate': {u'sourceId': None, u'timestamp': 1457479063181, u'versions': [{u'timestamp': 1457479063181, u'name': u'createdate', u'value': u'1457479063181', u'sourceVid': []}], u'value': u'1457479063181', u'source': None}, u'hs_salesforceopportunityid': {u'sourceId': None, u'timestamp': 1457479097680, u'versions': [{u'timestamp': 1457479097680, u'sourceVid': [], u'name': u'hs_salesforceopportunityid', u'value': u'00628000007nhqFAAQ', u'source': u'SALESFORCE'}], u'value': u'00628000007nhqFAAQ', u'source': u'SALESFORCE'}, u'hubspot_owner_assigneddate': {u'sourceId': None, u'timestamp': 1457479097680, u'versions': [{u'timestamp': 1457479097680, u'sourceVid': [], u'name': u'hubspot_owner_assigneddate', u'value': u'1457479097680', u'source': u'SALESFORCE'}], u'value': u'1457479097680', u'source': u'SALESFORCE'}, u'hubspot_owner_id': {u'sourceId': None, u'timestamp': 1457479097680, u'versions': [{u'timestamp': 1457479097680, u'sourceVid': [], u'name': u'hubspot_owner_id', u'value': u'11626092', u'source': u'SALESFORCE'}], u'value': u'11626092', u'source': u'SALESFORCE'}, u'amount': {u'sourceId': None, u'timestamp': 1457479063157, u'versions': [{u'timestamp': 1457479063157, u'sourceVid': [], u'name': u'amount', u'value': u'150000', u'source': u'API'}], u'value': u'150000', u'source': u'API'}, u'hs_createdate': {u'sourceId': None, u'timestamp': 1457479063181, u'versions': [{u'timestamp': 1457479063181, u'name': u'hs_createdate', u'value': u'1457479063181', u'sourceVid': []}], u'value': u'1457479063181', u'source': None}, u'salesforcelastsynctime': {u'sourceId': None, u'timestamp': 1457479099298, u'versions': [{u'timestamp': 1457479099298, u'sourceVid': [], u'name': u'salesforcelastsynctime', u'value': u'1457479070904', u'source': u'SALESFORCE'}], u'value': u'1457479070904', u'source': u'SALESFORCE'}, u'closedate': {u'sourceId': None, u'timestamp': 1457479099298, u'versions': [{u'timestamp': 1457479099298, u'sourceVid': [], u'name': u'closedate', u'value': u'1461013200000', u'source': u'SALESFORCE'}], u'value': u'1461013200000', u'source': u'SALESFORCE'}, u'dealtype': {u'sourceId': None, u'timestamp': 1457479063157, u'versions': [{u'timestamp': 1457479063157, u'sourceVid': [], u'name': u'dealtype', u'value': u'existingbusiness', u'source': u'API'}], u'value': u'existingbusiness', u'source': u'API'}, u'dealname': {u'sourceId': None, u'timestamp': 1457479063157, u'versions': [{u'timestamp': 1457479063157, u'sourceVid': [], u'name': u'dealname', u'value': u'Google Website', u'source': u'API'}], u'value': u'Google Website', u'source': u'API'}}

Answer 1

response.json() 返回的对象已经转换为dict，因此您无需对其进行任何进一步的处理。要获取所有qualifiedtobuy 交易的列表，请尝试以下操作：

jsonDeals = response.json()

deals = []
for deal in jsonDeals['deals']:
    properties = deal['properties']
    if ('dealstage' in properties and
        properties['dealstage']['value'] == 'qualifiedtobuy'):
        deals.append(deal)

if deals:
    print(deals[0]['dealId'])
else:
    print('found no "qualifiedtobuy" deals')

Answer 2

Python 无疑是一种简单且更具可读性的方法。有时我很懒，所以我只使用jq（文档：https://stedolan.github.io/jq/）。

对于您的示例，它可以是Bash 中的单行：

curl -s https://api.hubapi.com/deals/v1/deal/all?hapikey=demo | \
    jq '.deals[] | select(.properties.dealstage.value=="qualifiedtobuy")'

这将为您提供您需要的格式良好的一行。将--compact-output 标志添加到jq 将在一行中打印每个对象。然后，如果您需要更复杂的操作，可以将输出重定向到文件。

对于您的测试 url，使用标志它只输出一条记录：

{"portalId":62515,"dealId":17886969,"isDeleted":false,"associations":{"associatedVids":[],"associatedCompanyIds":[113448746],"associatedDealIds":[]},"proper
ties":{"pipeline":{"value":"default","timestamp":1456622756943,"source":null,"sourceId":null,"versions":[{"name":"pipeline","value":"default","timestamp":14
56622756943,"sourceVid":[]}]},"dealname":{"value":"Google Website","timestamp":1456622756908,"source":"API","sourceId":null,"versions":[{"name":"dealname","
value":"Google Website","timestamp":1456622756908,"source":"API","sourceVid":[]}]},"amount":{"value":"150000","timestamp":1456622756908,"source":"API","sour
ceId":null,"versions":[{"name":"amount","value":"150000","timestamp":1456622756908,"source":"API","sourceVid":[]}]},"closedate":{"value":"1461042000000","ti
mestamp":1456622756908,"source":"API","sourceId":null,"versions":[{"name":"closedate","value":"1461042000000","timestamp":1456622756908,"source":"API","sour
ceVid":[]}]},"hubspot_owner_id":{"value":"72","timestamp":1456622756908,"source":"API","sourceId":null,"versions":[{"name":"hubspot_owner_id","value":"72","
timestamp":1456622756908,"source":"API","sourceVid":[]}]},"hs_lastmodifieddate":{"value":"1456622756943","timestamp":1456622756943,"source":"CALCULATED","so
urceId":null,"versions":[{"name":"hs_lastmodifieddate","value":"1456622756943","timestamp":1456622756943,"source":"CALCULATED","sourceVid":[]}]},"hubspot_ow
ner_assigneddate":{"value":"1456622756908","timestamp":1456622756908,"source":"API","sourceId":null,"versions":[{"name":"hubspot_owner_assigneddate","value"
:"1456622756908","timestamp":1456622756908,"source":"API","sourceVid":[]}]},"num_associated_contacts":{"value":"0","timestamp":0,"source":"CALCULATED","sour
ceId":null,"versions":[{"name":"num_associated_contacts","value":"0","source":"CALCULATED","sourceVid":[]}]},"dealstage":{"value":"qualifiedtobuy","timestam
p":1456622756908,"source":"API","sourceId":null,"versions":[{"name":"dealstage","value":"qualifiedtobuy","timestamp":1456622756908,"source":"API","sourceVid
":[]}]},"hs_createdate":{"value":"1456622756943","timestamp":1456622756943,"source":null,"sourceId":null,"versions":[{"name":"hs_createdate","value":"145662
2756943","timestamp":1456622756943,"sourceVid":[]}]},"createdate":{"value":"1456622756943","timestamp":1456622756943,"source":null,"sourceId":null,"versions
":[{"name":"createdate","value":"1456622756943","timestamp":1456622756943,"sourceVid":[]}]},"dealtype":{"value":"existingbusiness","timestamp":1456622756908
,"source":"API","sourceId":null,"versions":[{"name":"dealtype","value":"existingbusiness","timestamp":1456622756908,"source":"API","sourceVid":[]}]}},"impor
ts":[]}

对于简单的字段提取，您可以执行以下操作：

curl -s https://api.hubapi.com/deals/v1/deal/all?hapikey=demo | \
    jq --compact-output --raw-output \
    '.deals[] | select(.properties.dealstage.value=="qualifiedtobuy") | [.properties.amount.value, .properties.closedate.value, .properties.createdate.value, .properties.dealname.value] | @csv'

这将给出匹配的行，每行包含交易金额、结束日期、创建日期和交易名称，当然用逗号分隔。 --raw-output 标志消除了引号的转义，使输出更清晰。对于您的测试网址，它给出：

"150000","1461042000000","1456622756943","Google Website"

更多匹配，更多行。重定向输出并完成您的一天。更少的开发时间，更美好的世界。

Answer 3

import json
dict_object = json.loads(jsonDeals)

会给你你的字典，你不想只是将字符串转换为字典，因为一些 json 组件与 python 中的组件不同，例如小写 true 与 python 的 True

要与项目交互，您只需根据键索引到字典中

for deal in dict_object['deals']:
     print deal['dealId'], 'for instance'

您能否确认jsonDeals 是一个json 字符串...如果它是您在问题顶部发布的内容，它应该可以工作，我已对其进行了测试：

>>> a = """
... {  
...    "deals":[  
...       {  
...          "portalId":62515,
...          "dealId":17886969,
...          "isDeleted":false,
...          "associations":{
...             "associatedCompanyIds":[  
...                113448746
...             ]
...          },
...          "properties":{
...             "dealname":{  
...                "value":"Google Website"
...             },
...             "amount":{  
...                "value":"150000"
...             },
...             "hubspot_owner_id":{  
...                "value":"72"
...             },
...             "dealstage":{  
...                "value":"qualifiedtobuy"
...             },
...             "dealtype":{  
...                "value":"existingbusiness"
...             }
...          },
...          "imports":[  
... 
...          ]
...       }
...    ],
...    "hasMore":false,
...    "offset":28692358
... }"""
>>> import json
>>> d = json.loads(a)
>>>