jq 或 python 脚本删除 json 字段中日期之后的文本

Question

我有一个包含数百个条目的 json 文件，例如：

{
    "url":"http://example.com/10618/",
    "metatag.eprints.publication":"Journal of Corporate Real Estate",
    "metatag.eprints.title":"Corporate Real Estate Strategy",
    "metatag.eprints.citation":"Adair, P, McGrogan, WS, and Webb, JR (2006) Corporate Real Estate Strategy. Journal of Corporate Real Estate"}
{
    "url":"http://example.com/23552/",
    "metatag.eprints.publication":"European Journal of Cardio-Thoracic Surgery",
    "metatag.eprints.title":"Long-term survival from coronary endarterectomies in coronary artery disease",
    "metatag.eprints.citation":"Aaron, P, Jones, K, Pallin, C, and Nash, R (2012) Long-term survival from coronary endarterectomies in coronary artery disease. European Journal of Cardio-Thoracic Surgery"}

任何人都可以帮助编写一个 jq 或 python 脚本，对于每个块，都会更改“metatag.eprints.citation”，以便删除日期之后的所有文本？

所以上面的块会变成：

{
    "url":"http://example.com/10618/",
    "metatag.eprints.publication":"Journal of Corporate Real Estate",
    "metatag.eprints.title":"Corporate Real Estate Strategy",
    "metatag.eprints.citation":"Adair, P, McGrogan, WS, and Webb, JR (2006)"}
{
    "url":"http://example.com/23552/",
    "metatag.eprints.publication":"European Journal of Cardio-Thoracic Surgery",
    "metatag.eprints.title":"Long-term survival from coronary endarterectomies in coronary artery disease",
    "metatag.eprints.citation":"Aaron, P, Jones, K, Pallin, C, and Nash, R (2012)"}

Answer 1

jq '.["metatag.eprints.citation"] |= match(".*?\\\)").string // .'

需要 jq 1.5。这样做的目的是将metatag.eprints.citation 的值设置为与正则表达式.*?\) 匹配的结果，这将匹配第一个右括号之前的所有内容。如果由于某种原因没有右括号，我们使用替代运算符// 将值设置回原来的值。

Answer 2

一旦你的格式就像你的问题，你可以使用itertools.groupby按左括号分组，用str.join加入行并使用json.loads来获取一个dict，那么这只是一个访问的问题键并将更新的数据写入临时文件。最后使用shutil.move替换原文件，如果你想要一个全新的文件，只需将NamedTemporaryFile更改为使用open：

from tempfile import NamedTemporaryFile
from shutil import move
from itertools import groupby

import json

with open("in.txt") as f, NamedTemporaryFile("w", dir=".",delete=False) as out:
    for k, v in groupby(f, key=lambda x: x.lstrip().startswith("{")):
        if not k:
            d = json.loads("{" + "".join(v))
            v = d["metatag.eprints.citation"]
            d["metatag.eprints.citation"] = v[:v.find(")")+1]
            json.dump(d, out)
            out.write("\n")
move(out.name,"in.txt")

in.txt 之前：

{
    "url":"http://example.com/10618/",
    "metatag.eprints.publication":"Journal of Corporate Real Estate",
    "metatag.eprints.title":"Corporate Real Estate Strategy",
    "metatag.eprints.citation":"Adair, P, McGrogan, WS, and Webb, JR (2006) Corporate Real Estate Strategy. Journal of Corporate Real Estate"}
{
    "url":"http://example.com/23552/",
    "metatag.eprints.publication":"European Journal of Cardio-Thoracic Surgery",
    "metatag.eprints.title":"Long-term survival from coronary endarterectomies in coronary artery disease",
    "metatag.eprints.citation":"Aaron, P, Jones, K, Pallin, C, and Nash, R (2012) Long-term survival from coronary endarterectomies in coronary artery disease. European Journal of Cardio-Thoracic Surgery"}

in.txt 之后：

{"url": "http://example.com/10618/", "metatag.eprints.publication": "Journal of Corporate Real Estate", "metatag.eprints.citation": "Adair, P, McGrogan, WS, and Webb, JR (2006)", "metatag.eprints.title": "Corporate Real Estate Strategy"}
{"url": "http://example.com/23552/", "metatag.eprints.publication": "European Journal of Cardio-Thoracic Surgery", "metatag.eprints.citation": "Aaron, P, Jones, K, Pallin, C, and Nash, R (2012)", "metatag.eprints.title": "Long-term survival from coronary endarterectomies in coronary artery disease"}

如果您必须在以后编辑它，您可以简单地遍历文件和json.loads 每行以获取字典，再次使用密钥更新并写入文件。每行一个将使您的生活更轻松。

如果您可以在日期之前有一个左括号，则可以使用正则表达式搜索特定的子字符串，括号之间为 4 位：

r = re.compile("\(\d{4}\)")
for k, v in groupby(f, key=lambda x: x.lstrip().startswith("{")):
    if not k:
        d = json.loads("{" + "".join(v))
        v = d["metatag.eprints.citation"]
        d["metatag.eprints.citation"] = v[:next(r.finditer(v)).end()]
        json.dump(d, out)
        out.write("\n")

如果您得到一个空文件，那么您的数据实际上必须是每行一个 dict，因此只需遍历文件对象并应用相同的逻辑即可：

with open("in.txt") as f, NamedTemporaryFile("w", dir=".",delete=False) as out:
    for line in f:
            d = json.loads(line)
            v = d["metatag.eprints.citation"]
            d["metatag.eprints.citation"] = v[:v.find(")")+1]
            json.dump(d, out)
            out.write("\n")
move(out.name,"in.txt")