【问题标题】:Converting Google Cloud Video intelligence json to newline delimited json to upload to Big Query将 Google Cloud Video Intelligence json 转换为换行符分隔的 json 以上传到 Big Query
【发布时间】:2020-08-15 08:04:29
【问题描述】:

我创建了一个函数,它使用 Google Video Intelligence API 输出下面的 json。我正在尝试将 json 转换为换行符分隔格式,以便可以将 json 文件上传到 Big Query。

我尝试了以下 JQ 代码,但它没有将其转换为 BiG Query 接受的格式。

非常感谢任何帮助。

$ cat a.json | jq -c '.[]'

来自视频智能的完整 json 输出

{
  "annotation_results": [ {
    "input_uri": "/mastersproject-252719.appspot.com/AEnB2UoTKtyPJd1X88yUBKC60dn3kp6qrnnE91Q5PylS9RqotQWrt2TzFPE7kG1UHtxoBYyWX5mo7zoXU1jzwnNe_WRD-t0QeFSoHU4g-WJZTfeQV4cpFNY.4-FicKZIT3vtFBd2",
    "segment": {
      "start_time_offset": {

      },
      "end_time_offset": {
        "seconds": 126,
        "nanos": 160000000
      }
    },
    "shot_annotations": [ {
      "start_time_offset": {

      },
      "end_time_offset": {
        "seconds": 3,
        "nanos": 960000000
      }
    }, {
      "start_time_offset": {
        "seconds": 4
      },
      "end_time_offset": {
        "seconds": 5,
        "nanos": 840000000
      }
    }, {
      "start_time_offset": {
        "seconds": 5,
        "nanos": 880000000
      },
      "end_time_offset": {
        "seconds": 11,
        "nanos": 40000000
      }
    }, {
      "start_time_offset": {
        "seconds": 11,
        "nanos": 80000000
      },
      "end_time_offset": {
        "seconds": 14,
        "nanos": 640000000
      }
    }, {
      "start_time_offset": {
        "seconds": 14,
        "nanos": 680000000
      },
      "end_time_offset": {
        "seconds": 15,
        "nanos": 400000000
      }
    }, {
      "start_time_offset": {
        "seconds": 15,
        "nanos": 440000000
      },
      "end_time_offset": {
        "seconds": 19,
        "nanos": 840000000
      }
    }, {
      "start_time_offset": {
        "seconds": 19,
        "nanos": 880000000
      },
      "end_time_offset": {
        "seconds": 21,
        "nanos": 880000000
      }
    }, {
      "start_time_offset": {
        "seconds": 21,
        "nanos": 920000000
      },
      "end_time_offset": {
        "seconds": 25,
        "nanos": 120000000
      }
    }, {
      "start_time_offset": {
        "seconds": 25,
        "nanos": 160000000
      },
      "end_time_offset": {
        "seconds": 30,
        "nanos": 880000000
      }
    }, {
      "start_time_offset": {
        "seconds": 30,
        "nanos": 920000000
      },
      "end_time_offset": {
        "seconds": 32,
        "nanos": 960000000
      }
    }, {
      "start_time_offset": {
        "seconds": 33
      },
      "end_time_offset": {
        "seconds": 38,
        "nanos": 80000000
      }
    }, {
      "start_time_offset": {
        "seconds": 38,
        "nanos": 120000000
      },
      "end_time_offset": {
        "seconds": 44,
        "nanos": 80000000
      }
    }, {
      "start_time_offset": {
        "seconds": 44,
        "nanos": 120000000
      },
      "end_time_offset": {
        "seconds": 47,
        "nanos": 680000000
      }
    }, {
      "start_time_offset": {
        "seconds": 47,
        "nanos": 720000000
      },
      "end_time_offset": {
        "seconds": 51,
        "nanos": 120000000
      }
    }, {
      "start_time_offset": {
        "seconds": 51,
        "nanos": 160000000
      },
      "end_time_offset": {
        "seconds": 56,
        "nanos": 640000000
      }
    }, {
      "start_time_offset": {
        "seconds": 56,
        "nanos": 680000000
      },
      "end_time_offset": {
        "seconds": 59,
        "nanos": 40000000
      }
    }, {
      "start_time_offset": {
        "seconds": 59,
        "nanos": 80000000
      },
      "end_time_offset": {
        "seconds": 62,
        "nanos": 880000000
      }
    }, {
      "start_time_offset": {
        "seconds": 62,
        "nanos": 920000000
      },
      "end_time_offset": {
        "seconds": 70,
        "nanos": 640000000
      }
    }, {
      "start_time_offset": {
        "seconds": 70,
        "nanos": 680000000
      },
      "end_time_offset": {
        "seconds": 74,
        "nanos": 240000000
      }
    }, {
      "start_time_offset": {
        "seconds": 74,
        "nanos": 280000000
      },
      "end_time_offset": {
        "seconds": 76,
        "nanos": 40000000
      }
    }, {
      "start_time_offset": {
        "seconds": 76,
        "nanos": 80000000
      },
      "end_time_offset": {
        "seconds": 82,
        "nanos": 240000000
      }
    }, {
      "start_time_offset": {
        "seconds": 82,
        "nanos": 280000000
      },
      "end_time_offset": {
        "seconds": 86
      }
    }, {
      "start_time_offset": {
        "seconds": 86,
        "nanos": 40000000
      },
      "end_time_offset": {
        "seconds": 89,
        "nanos": 720000000
      }
    }, {
      "start_time_offset": {
        "seconds": 89,
        "nanos": 760000000
      },
      "end_time_offset": {
        "seconds": 99,
        "nanos": 680000000
      }
    }, {
      "start_time_offset": {
        "seconds": 99,
        "nanos": 720000000
      },
      "end_time_offset": {
        "seconds": 104,
        "nanos": 360000000
      }
    }, {
      "start_time_offset": {
        "seconds": 104,
        "nanos": 400000000
      },
      "end_time_offset": {
        "seconds": 107,
        "nanos": 440000000
      }
    }, {
      "start_time_offset": {
        "seconds": 107,
        "nanos": 480000000
      },
      "end_time_offset": {
        "seconds": 108,
        "nanos": 640000000
      }
    }, {
      "start_time_offset": {
        "seconds": 108,
        "nanos": 680000000
      },
      "end_time_offset": {
        "seconds": 112,
        "nanos": 160000000
      }
    }, {
      "start_time_offset": {
        "seconds": 112,
        "nanos": 200000000
      },
      "end_time_offset": {
        "seconds": 114,
        "nanos": 640000000
      }
    }, {
      "start_time_offset": {
        "seconds": 114,
        "nanos": 680000000
      },
      "end_time_offset": {
        "seconds": 117,
        "nanos": 520000000
      }
    }, {
      "start_time_offset": {
        "seconds": 117,
        "nanos": 560000000
      },
      "end_time_offset": {
        "seconds": 118,
        "nanos": 760000000
      }
    }, {
      "start_time_offset": {
        "seconds": 118,
        "nanos": 800000000
      },
      "end_time_offset": {
        "seconds": 121,
        "nanos": 80000000
      }
    }, {
      "start_time_offset": {
        "seconds": 121,
        "nanos": 120000000
      },
      "end_time_offset": {
        "seconds": 122,
        "nanos": 320000000
      }
    }, {
      "start_time_offset": {
        "seconds": 122,
        "nanos": 360000000
      },
      "end_time_offset": {
        "seconds": 124,
        "nanos": 200000000
      }
    }, {
      "start_time_offset": {
        "seconds": 124,
        "nanos": 240000000
      },
      "end_time_offset": {
        "seconds": 126,
        "nanos": 160000000
      }
    } ]
  } ]
}

【问题讨论】:

    标签: json google-cloud-platform jq video-intelligence-api


    【解决方案1】:

    我看到您在this 堆栈溢出帖子中尝试了建议的 JQ 命令。您需要使用适合您情况的不同 JQ 命令。

    我亲自尝试过:

    cat <filename>.json | jq -c '.'
    

    它有效。意思是,最终结果是一个有效的换行分隔的 json。

    但是,根据您的最终表架构,它可能需要更多调整。

    如果它不起作用,请在说明中包含您的最终表架构,以便我们更好地了解目标是什么。

    【讨论】:

      猜你喜欢
      • 2021-09-04
      • 1970-01-01
      • 2018-12-20
      • 1970-01-01
      • 2021-03-22
      • 2015-12-21
      • 2019-08-18
      • 2023-03-18
      • 1970-01-01
      相关资源
      最近更新 更多