使用 jq 将 JSON 导出为 CSV

Question

我有以下 JSON 文件：

{"data":[{"id":"DDM003","base":[{"date":"2020-06-04T00:30:00Z","value":335.2,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":330.1,"state":"A","validated":2}]},{"id":"DTR001","base":[{"date":"2020-06-04T00:30:00Z","value":0.2,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":0.1,"state":"A","validated":2}]},{"id":"FFM003","base":[{"date":"2020-06-04T00:30:00Z","value":2.62,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":3.15,"state":"A","validated":2}]},{"id":"RAIN12","base":[{"date":"2020-06-04T00:30:00Z","value":15.0,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":16.0,"state":"A","validated":2}]},{"id":"RHM003","base":[{"date":"2020-06-04T00:30:00Z","value":85.41,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":85.35,"state":"A","validated":2}]},{"id":"WVM003","base":[{"date":"2020-06-04T00:30:00Z","value":2.56,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":3.08,"state":"A","validated":2}]},{"id":"TLR001","base":[{"date":"2020-06-04T00:30:00Z","value":14.28,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":14.36,"state":"A","validated":2}]},{"id":"THR001","base":[{"date":"2020-06-04T00:30:00Z","value":14.07,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":14.23,"state":"A","validated":2}]},{"id":"PPR001","base":[{"date":"2020-06-04T00:30:00Z","value":999.2,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":998.9,"state":"A","validated":2}]},{"id":"RHR001","base":[{"date":"2020-06-04T00:30:00Z","value":80.5,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":80.0,"state":"A","validated":2}]},{"id":"WDR001","base":[{"date":"2020-06-04T00:30:00Z","value":317.71,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":320.31,"state":"A","validated":2}]},{"id":"WVR001","base":[{"date":"2020-06-04T00:30:00Z","value":2.75,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":2.33,"state":"A","validated":2}]},{"id":"WSR001","base":[{"date":"2020-06-04T00:30:00Z","value":2.91,"state":"A","validated":2},{"date":"2020-06-04T01:00:00Z","value":2.44,"state":"A","validated":2}]}]}

我想使用jq 将其（以及任何具有较大日期范围的此类文件）导出为以下 CSV 格式（注意字段的重新排序）：

date;WDR001;WVR001;WSR001;TLR001;THR001;DTR001;PPR001;RHR001;DDM003;WVM003;FFM003;RHM003;RAIN12
2020-06-04 00:30:00;317.71;2.75;2.91;14.28;14.36;0.2;999.2;80.5;335.2;2.56;2.62;85.41;15
2020-06-04 01:00:00;320.31;2.33;2.44;14.07;14.23;0.1;998.9;80;330.1;3.08;3.15;85.35;16

条件是，与不同于 {A,R,O,W,K} 的“状态”属性（始终存在）或不同于 2 的“已验证”属性（可能缺失）相关联的任何值是设置为默认值-9999。

有人可以帮我解决实现此目的所需的jq 过滤器吗？

非常感谢。

Answer 1

使用 -r 命令行选项，以下过滤器会生成如下所示的 CSV：

def adjust:
  if (.state | test("^[AROWK]$") | not) or .validated != 2 
  then .value = -9999 
  else . 
  end;


["WDR001", "WVR001", "WSR001", "TLR001", "THR001", "DTR001", "PPR001", "RHR001", "DDM003", "WVM003", "FFM003", "RHM003", "RAIN12"] as $keys
| [.data[] | .id as $id | .base[] | . + {id: $id}]
| group_by(.date)[]
| map(adjust)
| .[0].date as $date
| ( INDEX(.[]; .id ) | map_values({value}) )
| [ $date, ($keys[] as $k | .[$k].value)]
| @csv

换句话说：将 .id 存储在 .base 对象中；按 .date 对这些对象进行分组；调整.value；创建从 .id 到 .value 的映射；最后发出所需的输出。

输出：

"2020-06-04T00:30:00Z",317.71,2.75,2.91,14.28,14.07,0.2,999.2,80.5,335.2,2.56,2.62,85.41,15
"2020-06-04T01:00:00Z",320.31,2.33,2.44,14.36,14.23,0.1,998.9,80,330.1,3.08,3.15,85.35,16

添加标题很容易。

如果您想用分号分隔的值不带引号，您可以使用join(;) 而不是@csv。

Terser 仍然

["WDR001", "WVR001", "WSR001", "TLR001", "THR001", "DTR001", "PPR001", "RHR001", "DDM003", "WVM003", "FFM003", "RHM003", "RAIN12"] as $keys
| [.data[] | .base[] + {id}]
| group_by(.date)[]
| map(adjust)
| .[0].date as $date
| ( INDEX(.[]; .id ) | map_values({value}) )
| [ $date, (.[$keys[]]|.value)]
| @csv