【发布时间】：2022-01-20 15:09:07
【问题描述】：
这是输出格式，基于“CVE_data_meta”，我需要对匹配的 ID 进行重复数据删除。
#pull references
for ref in item["cve"]["references"]["reference_data"]:
references = ref["url"]
cleanData.append({"CVE_data_meta": cve_data_meta_id,
"description": description,
"baseScore": baseScore,
"vectorSring": vectorString,
"cweID": cweValue,
"cweID URL": ("https://cwe.mitre.org/data/definitions/"
+ str(cweValue) + ".html"),
"references": references,
"publishedDate": pub_date,
"lastModifiedDate": last_mod_date
})
这是我从 API 的清理响应中提取数据并输出到 JSON 文件的迭代：
# # ==========================================================================================
# # narrow response with additional 'keywords'
# # ==========================================================================================
myResults = open("2-cleanData.json", "r")
scope = json.load(myResults)
output_json=[]
results = []
for k in keywords:
counter = 0
items = [x for x in scope if k in x['description']]
for item in items:
output_json.append(item)
counter += 1
results.append(counter)
with open("3-Final CVEs.json", "w+") as outFile2:
outFile2.write(json.dumps(output_json, indent=2,))
keywords 变量可由用户更改；但希望任何能够添加关键字并且不会在输出文件中获得重复条目。
完整代码here.
示例输出：（3 个 CVE 条目）
{
"CVE_data_meta": "CVE-2021-0924",
"description": "In xhci_vendor_get_ops of xhci.c, there is a possible out of bounds read due to a missing bounds check. This could lead to local escalation of privilege with no additional execution privileges needed. User interaction is not needed for exploitation.Product: AndroidVersions: Android kernelAndroid ID: A-194461020References: Upstream kernel",
"baseScore": 7.8,
"vectorSring": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H",
"cweID": "CWE-125",
"cweID URL": "https://cwe.mitre.org/data/definitions/CWE-125.html",
"references": "https://source.android.com/security/bulletin/2021-11-01",
"publishedDate": "2021-12-15T19:15Z",
"lastModifiedDate": "2021-12-17T18:12Z"
},
{
"CVE_data_meta": "CVE-2021-0981",
"description": "In enqueueNotificationInternal of NotificationManagerService.java, there is a possible way to run a foreground service without showing a notification due to improper input validation. This could lead to local escalation of privilege with no additional execution privileges needed. User interaction is not needed for exploitation.Product: AndroidVersions: Android-12Android ID: A-191981182",
"baseScore": 7.8,
"vectorSring": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H",
"cweID": "CWE-269",
"cweID URL": "https://cwe.mitre.org/data/definitions/CWE-269.html",
"references": "https://source.android.com/security/bulletin/pixel/2021-12-01",
"publishedDate": "2021-12-15T19:15Z",
"lastModifiedDate": "2021-12-17T18:09Z"
...several entries later...
"CVE_data_meta": "CVE-2021-0924",
"description": "In xhci_vendor_get_ops of xhci.c, there is a possible out of bounds read due to a missing bounds check. This could lead to local escalation of privilege with no additional execution privileges needed. User interaction is not needed for exploitation.Product: AndroidVersions: Android kernelAndroid ID: A-194461020References: Upstream kernel",
"baseScore": 7.8,
"vectorSring": "CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H",
"cweID": "CWE-125",
"cweID URL": "https://cwe.mitre.org/data/definitions/CWE-125.html",
"references": "https://source.android.com/security/bulletin/2021-11-01",
"publishedDate": "2021-12-15T19:15Z",
"lastModifiedDate": "2021-12-17T18:12Z"
},
现在，只需要删除重复项...
【问题讨论】：
-
只需要根据
'CVE_data_meta'条目的值去重吗？ -
是的！并且具有重复 CVE 的相应值也不会被追加或仅在追加后重复数据删除。因此，重复数据删除可以在迭代期间完成，或者创建另一个循环附加到对 outFile 进行重复数据删除。
-
好的，这让事情变得相对容易。请参阅我发布的答案。
标签： python json object duplicates