将 json 数组折叠成具有组合键值的单个数组答案

【问题标题】：Collapse json arrays into single array with combined key values将 json 数组折叠成具有组合键值的单个数组
【发布时间】：2020-02-25 07:55:21
【问题描述】：

我正在尝试将多个 JSON 数组组合成一个折叠的类似键值的数组。这是我得到的代码。

[
  {
    "projectId": 35525710,
    "jobs": [
        {
          "targetLanguage": "zh_CN",
          "jobId": 35826845,
          "earlyAccessStatus": "NOT_STARTED"
        },
        {
          "targetLanguage": "pt_BR",
          "jobId": 35826846,
          "earlyAccessStatus": "IN_PROGRESS"
        },
        {
          "targetLanguage": "zh_CN",
          "jobId": 35826836,
          "earlyAccessStatus": "IN_PROGRESS"
        },
        {
          "targetLanguage": "pt_BR",
          "jobId": 35826837,
          "earlyAccessStatus": "IN_PROGRESS"
        }
      ]
   }
]

所需的输出将是：（在下方编辑以采用不同的路线）

[
  {
    "projectId": 35525710,
    "jobs": [
        {
           "targetLanguage": "zh_CN",
           "jobIds": [35826845,35826836],
           "earlyAccessStatus": ["NOT_STARTED","IN_PROGRESS"]
        },
        {
           "targetLanguage": "pt_BR",
           "jobIds": [35826846,35826837],
           "earlyAccessStatus": ["IN_PROGRESS"]
        }
     ] 
   }
]

老实说，我不确定重复的 .earlyAccessStatus 键值会发生什么，无论是 ["IN_PROGRESS","IN_PROGRESS"] 还是 ["IN_PROGRESS"]。后者适合我的最终需求，即为任何只有.earlyAccessStatus 是["IN_PROGRESS"] 的语言获取.targetLanguage 值。

将此过滤器与jq -r 选项一起使用，我可以为任何只有.earlyAccessStatus 是["IN_PROGRESS"] 的语言提取.targetLanguage 值。任何帮助将不胜感激！

.[].jobs[] | select(.[] | .earlyAccessStatus == ["IN_PROGRESS"] ) | .[] | .targetLanguage

更新：这是在我操作任何东西之前原始 JSON 的样子。我不依附于任何特定的方法。我要做的是获取有关具有各种作业的项目的数据，并隔离所有相关作业已达到特定步骤（目标文件导出1）的语言，如“IN_PROGRESS”所示。如果该语言在不同的步骤中有一项工作，例如下面的 zh_CN，那么该语言还没有准备好，不应该通过过滤器。

为了得到上面的输出，我已经做到了（为了消除已经建议的“作业”中的冗余数组而进行了一次更新，对于你在 jq 中看到的任何科学怪人尝试行分离的任何尝试，我提前道歉命令）：

. | [{"projectId": .projectId, 
    "jobs": [( .jobs[] | 
        { "targetLanguage": .targetLanguage, 
          "jobId": .jobId,
          "earlyAccessStatus": (.steps[] | select(.workflowStepName == "Target file export1") | .status) } )] }]

原始 JSON：

{
  "projectId": 35902499,
  "completionStatus": "IN_PROGRESS",
  "activity": "ACTIVE",
  "sourceLanguage": "en_US",
  "jobs": [
    {
      "jobId": 35902526,
      "completionStatus": "IN_PROGRESS",
      "targetLanguage": "pt_BR",
      "steps": [
        {
          "workflowStepName": "Project Intake and Quote Generation1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Translate1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Correct1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Segment greenification1",
          "status": "FINISHED",
          "autoStatus": "SUCCESS"
        },
        {
          "workflowStepName": "Target file export1",
          "status": "IN_PROGRESS"
        }
      ]
    },
    {
      "jobId": 35902516,
      "completionStatus": "IN_PROGRESS",
      "targetLanguage": "zh_CN",
      "steps": [
        {
          "workflowStepName": "Project Intake and Quote Generation1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Translate1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Correct1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Segment greenification1",
          "status": "FINISHED",
          "autoStatus": "SUCCESS"
        },
        {
          "workflowStepName": "Target file export1",
          "status": "IN_PROGRESS"
        }
      ]
    },
    {
      "jobId": 36433561,
      "completionStatus": "IN_PROGRESS",
      "targetLanguage": "pt_BR",
      "steps": [
        {
          "workflowStepName": "Project Intake and Quote Generation1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Translate1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Correct1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Segment greenification1",
          "status": "FINISHED",
          "autoStatus": "SUCCESS"
        },
        {
          "workflowStepName": "Target file export1",
          "status": "IN_PROGRESS"
        }
      ]
    },
    {
      "jobId": 36433560,
      "completionStatus": "IN_PROGRESS",
      "targetLanguage": "zh_CN",
      "steps": [
        {
          "workflowStepName": "Project Intake and Quote Generation1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Translate1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Correct1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Segment greenification1",
          "status": "FINISHED",
          "autoStatus": "SUCCESS"
        },
        {
          "workflowStepName": "Target file export1",
          "status": "IN_PROGRESS"
        }
      ]
    },
    {
      "jobId": 36433552,
      "completionStatus": "IN_PROGRESS",
      "targetLanguage": "pt_BR",
      "steps": [
        {
          "workflowStepName": "Project Intake and Quote Generation1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Translate1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Correct1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Segment greenification1",
          "status": "FINISHED",
          "autoStatus": "SUCCESS"
        },
        {
          "workflowStepName": "Target file export1",
          "status": "IN_PROGRESS"
        }
      ]
    },
    {
      "jobId": 36433551,
      "completionStatus": "IN_PROGRESS",
      "targetLanguage": "zh_CN",
      "steps": [
        {
          "workflowStepName": "Project Intake and Quote Generation1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Translate1",
          "status": "FINISHED"
        },
        {
          "workflowStepName": "Correct1",
          "status": "IN_PROGRESS"
        },
        {
          "workflowStepName": "Segment greenification1",
          "status": "NOT_STARTED"
        },
        {
          "workflowStepName": "Target file export1",
          "status": "NOT_STARTED"
        }
      ]
    }
  ]
}

新方法

传奇还在继续。走另一条路让我如此接近。

.[].jobs | 
    map({targetLanguage: .targetLanguage, 
        earlyAccess: {
            jobId: .jobId, 
            earlyAccessStatus: .earlyAccessStatus}}) 
        | group_by(.targetLanguage) 
        | map({targetLanguage: .[0].targetLanguage, 
            jobId: map(.jobId) | unique, 
            earlyAccessStatus: map(.earlyAccessStatus) | unique})

这基本上给了我想要的输出，除了我需要的丢失数据，我想这是一个挫折：

[
  {
    "targetLanguage": "pt_BR",
    "jobId": [
      null
    ],
    "earlyAccessStatus": [
      null
    ]
  },
  {
    "targetLanguage": "zh_CN",
    "jobId": [
      null
    ],
    "earlyAccessStatus": [
      null
    ]
  }
]

理想情况下，这将输出包含在 jobId 和 earlyAccessStatus 键中的唯一值，如下所示：

[
  {
    "targetLanguage": "pt_BR",
    "jobId": [
      35826846, 35826837
    ],
    "earlyAccessStatus": [
      "IN_PROGRESS"
    ]
  },
  {
    "targetLanguage": "zh_CN",
    "jobId": [
      35826845, 35826836
    ],
    "earlyAccessStatus": [
      "IN_PROGRESS", "NOT_STARTED"
    ]
  }
]

这样我就可以根据earlyAccessStatus == ["IN_PROGRESS"]轻松过滤targetLanguage。

【问题讨论】：

"jobs" 下的双数组在结构上不是多余的吗？您已经对对象进行了分组，为什么需要将每个对象保存在单独的数组中？

标签： arrays json bash jq

【解决方案1】：

您可能需要重新考虑所有这些数组中的数组，但除了组和键的排序之外，以下实现了您想要的：

# meld all the objects in the given stream of objects
def meld(objects): 
  reduce objects as $o ({}; 
    reduce ($o|keys[]) as $k (.; .[$k] += [$o[$k]]));


.[0].jobs |=
  (group_by(.[0].targetLanguage)
   | map( [meld(.[] | .[0] ) | .targetLanguage |= .[0] ] ))

这会产生如下所示的输出。删除重复项的最简单方法是添加 unique，例如作为meld 函数的最后一步。

同样，如果您希望以不同方式对组进行排序，您可以将顺序指定为后处理步骤，或编写您自己的group_by。

输出

[
  {
    "projectId": 35525710,
    "jobs": [
      [
        {
          "earlyAccessStatus": [
            "IN_PROGRESS",
            "IN_PROGRESS"
          ],
          "jobId": [
            35826846,
            35826837
          ],
          "targetLanguage": "pt_BR"
        }
      ],
      [
        {
          "earlyAccessStatus": [
            "NOT_STARTED",
            "IN_PROGRESS"
          ],
          "jobId": [
            35826845,
            35826836
          ],
          "targetLanguage": "zh_CN"
        }
      ]
    ]
  }
]

【讨论】：

【解决方案2】：

所有这些 1 元素长数组都很痛苦，我确信有更好的方法来做到这一点，但以下方法可行：

jq '[.[]
     | .jobs |=
       [flatten
        | group_by(.targetLanguage)
        | .[]
        | [reduce .[] as $job ({jobId:[], earlyAccessStatus:[]};
             {targetLanguage: $job.targetLanguage,
              jobId: (.jobId + [$job.jobId]),
              earlyAccessStatus: (.earlyAccessStatus + [$job.earlyAccessStatus])})]
        | walk(if type == "object" and has("earlyAccessStatus")
               then .earlyAccessStatus |= unique
               else . end) ] ]' input.json

[
  {
    "projectId": 35525710,
    "jobs": [
      [
        {
          "targetLanguage": "pt_BR",
          "jobId": [
            35826846,
            35826837
          ],
          "earlyAccessStatus": [
            "IN_PROGRESS"
          ]
        }
      ],
      [
        {
          "targetLanguage": "zh_CN",
          "jobId": [
            35826845,
            35826836
          ],
          "earlyAccessStatus": [
            "IN_PROGRESS",
            "NOT_STARTED"
          ]
        }
      ]
    ]
  }
]

【讨论】：

【解决方案3】：

想出了一些可行的方法！走的是group_by和unique的路线。

. | [{projectId: .projectId, 
      jobs: [( .jobs[] | { 
            targetLanguage: .targetLanguage, 
            jobId: .jobId, 
            earlyAccessStatus: (.steps[] | select(.workflowStepName == "Target file export1") | .status), 
            finalStatus: (.steps[] | select(.workflowStepName == "Target file export2") | .status) } )] }] | 
      .[].jobs | group_by(.targetLanguage) | map({
            targetLanguage: .[0].targetLanguage, 
            jobId: map(.jobId),
            earlyAccessStatus: map(.earlyAccessStatus) | unique, 
            finalStatus: map(.finalStatus) | unique})

此方法分两步工作，首先提取相关数据并按如下方式组织：

[
  {
    "projectId": 35525710,
    "jobs": [
        {
          "targetLanguage": "zh_CN",
          "jobId": 35826845,
          "earlyAccessStatus": "NOT_STARTED"
        },
        {
          "targetLanguage": "pt_BR",
          "jobId": 35826846,
          "earlyAccessStatus": "IN_PROGRESS"
        },
        {
          "targetLanguage": "zh_CN",
          "jobId": 35826836,
          "earlyAccessStatus": "IN_PROGRESS"
        },
        {
          "targetLanguage": "pt_BR",
          "jobId": 35826837,
          "earlyAccessStatus": "IN_PROGRESS"
        }
      ]
   }
]

以这种方式简化内容后，将按.targetLanguage 分组，然后将.jobId、.earlyAccessStatus 和.finalStatus 键映射为每个.targetLanguage 键值下的唯一值一对。我确信有一种更简单的方法来格式化它，但它完全完成了我打算完成的工作，将数据保留为这种格式：

[
  {
    "targetLanguage": "pt_BR",
    "jobId": [
      35902526,
      36433561,
      36433552
    ],
    "earlyAccessStatus": [
      "IN_PROGRESS"
    ],
    "finalStatus": [
      "IN_PROGRESS"
    ]
  },
  {
    "targetLanguage": "zh_CN",
    "jobId": [
      35902516,
      36433560,
      36433551
    ],
    "earlyAccessStatus": [
      "IN_PROGRESS",
      "NOT_STARTED"
    ],
    "finalStatus": [
      "IN_PROGRESS"
    ]
  }
]

【讨论】：