具有跨越多行的值字符串的 jq 构造答案

【问题标题】：jq construct with value strings spanning multiple lines具有跨越多行的值字符串的 jq 构造
【发布时间】：2017-10-24 05:49:00
【问题描述】：

我正在尝试使用jq 构建一个 JSON 结构，理想情况下应该如下所示：-

{
  "api_key": "XXXXXXXXXX-7AC9-D655F83B4825",
  "app_guid": "XXXXXXXXXXXXXX",
  "time_start": 1508677200,
  "time_end": 1508763600,
  "traffic": [
    "event"
  ],
  "traffic_including": [
    "unattributed_traffic"
  ],
  "time_zone": "Australia/NSW",
  "delivery_format": "csv",
  "columns_order": [
    "attribution_attribution_action",
    "attribution_campaign",
    "attribution_campaign_id",
    "attribution_creative",
    "attribution_date_adjusted",
    "attribution_date_utc",
    "attribution_matched_by",
    "attribution_matched_to",
    "attribution_network",
    "attribution_network_id",
    "attribution_seconds_since",
    "attribution_site_id",
    "attribution_site_id",
    "attribution_tier",
    "attribution_timestamp",
    "attribution_timestamp_adjusted",
    "attribution_tracker",
    "attribution_tracker_id",
    "attribution_tracker_name",
    "count",
    "custom_dimensions",
    "device_id_adid",
    "device_id_android_id",
    "device_id_custom",
    "device_id_idfa",
    "device_id_idfv",
    "device_id_kochava",
    "device_os",
    "device_type",
    "device_version",
    "dimension_count",
    "dimension_data",
    "dimension_sum",
    "event_name",
    "event_time_registered",
    "geo_city",
    "geo_country",
    "geo_lat",
    "geo_lon",
    "geo_region",
    "identity_link",
    "install_date_adjusted",
    "install_date_utc",
    "install_device_version",
    "install_devices_adid",
    "install_devices_android_id",
    "install_devices_custom",
    "install_devices_email_0",
    "install_devices_email_1",
    "install_devices_idfa",
    "install_devices_ids",
    "install_devices_ip",
    "install_devices_waid",
    "install_matched_by",
    "install_matched_on",
    "install_receipt_status",
    "install_san_original",
    "install_status",
    "request_ip",
    "request_ua",
    "timestamp_adjusted",
    "timestamp_utc"
  ]
}

到目前为止我没有成功的尝试如下：-

json_construct=$(cat <<EOF
{
"api_key": "6AEC90B5-4169-59AF-7AC9-D655F83B4825",
"app_guid": "komacca-s-rewards-app-au-ios-production-cv8tx71",
"time_start": 1508677200,
"time_end": 1508763600,
"traffic": ["event"],
"traffic_including": ["unattributed_traffic"],
"time_zone": "Australia/NSW",
"delivery_format": "csv"
"columns_order": ["attribution_attribution_action","attribution_campaign","attribution_campaign_id","attribution_creative","attribution_date_adjusted","attribution_date_utc","attribution_matched_by","attribution_matched_to","attributio
network","attribution_network_id","attribution_seconds_since","attribution_site_id","attribution_tier","attribution_timestamp","attribution_timestamp_adjusted","attribution_tracker","attribution_tracker_id","attribution_tracker_name","
unt","custom_dimensions","device_id_adid","device_id_android_id","device_id_custom","device_id_idfa","device_id_idfv","device_id_kochava","device_os","device_type","device_version","dimension_count","dimension_data","dimension_sum","ev
t_name","event_time_registered","geo_city","geo_country","geo_lat","geo_lon","geo_region","identity_link","install_date_adjusted","install_date_utc","install_device_version","install_devices_adid","install_devices_android_id","install_
vices_custom","install_devices_email_0","install_devices_email_1","install_devices_idfa","install_devices_ids","install_devices_ip","install_devices_waid","install_matched_by","install_matched_on","install_receipt_status","install_san_
iginal","install_status","request_ip","request_ua","timestamp_adjusted","timestamp_utc"]
}
EOF)

接着是：-

echo "$json_construct" | jq '.'

我收到以下错误：-

解析错误：第 10 行第 15 列的值之间的预期分隔符

我猜这是因为跨越多行的字符串文字 jq 无法解析它。

【问题讨论】：

您不需要使用 $(cat ... EOF)。只需使用单引号' 直接设置json_construct='...'。这是一个Try it online! 示例。
我正在使用$(cat... EOF)，因为我这里显示的值实际上是变量。我知道你可以传递参数。
你能提供变量吗？合并这些应该很容易。例如json_construct='{"api_key":'"$key"'"}' - 注意谨慎使用'。
你有jq;用它来构造你的 JSON，而不是 cat。

标签： json jq string-literals

【解决方案1】：

使用jq 本身：

my_formatted_json=$(jq -n '{
  "api_key": "XXXXXXXXXX-7AC9-D655F83B4825",
  "app_guid": "XXXXXXXXXXXXXX",
  "time_start": 1508677200,
  "time_end": 1508763600,
  "traffic": ["event"],
  "traffic_including": ["unattributed_traffic"],
  "time_zone": "Australia/NSW",
  "delivery_format": "csv",
  "columns_order": [
    "attribution_attribution_action",
    "attribution_campaign",
    ...,
    "timestamp_utc"
  ]
}')

【讨论】：

【解决方案2】：

如错误消息所示，您输入的“JSON”不是有效的 JSON。

第一个错误是键/值对后缺少逗号：“delivery_format”：“csv”，但还有其他错误——值得注意的是，JSON 字符串不能跨行拆分。一旦您解决了键/值对问题和错误拆分的 JSON 字符串，jq . 将与您的文本一起使用。（请注意，一旦您的输入得到纠正，最长的 JSON 字符串就会很短——大约 50 个字符——而 jq 可以非常快速地处理长度为 10^8 的字符串......）

一般来说，jq 对于类似 JSON 的输入是相当宽松的，但如果您有疑问，使用验证器（例如 jsonlint.com 上的在线验证器）会很有意义

顺便说一句，jq 常见问题解答确实提出了处理非严格 JSON 输入的各种方法——请参阅https://github.com/stedolan/jq/wiki/FAQ#processing-not-quite-valid-json

【讨论】：

【解决方案3】：

按照chepner 的建议，由于 jq 可以读取原始文本数据，因此您可以使用 jq 过滤器从脚本变量生成合法的 json 对象。例如：

#!/bin/bash

# whatever logic you have to obtain bash variables goes here
key=XXXXXXXXXX-7AC9-D655F83B4825
guid=XXXXXXXXXXXXXX

# now use jq filter to read raw text and construct legal json object
json_construct=$(jq -MRn '[inputs]|map(split(" ")|{(.[0]):.[1]})|add' <<EOF
api_key $key
app_guid $guid
EOF)

echo $json_construct

示例运行（假设可执行脚本位于script.sh）

$ ./script.sh
{ "api_key": "XXXXXXXXXX-7AC9-D655F83B4825", "app_guid": "XXXXXXXXXXXXXX" }

Try it online!

【讨论】：