如何动态使用java在spark数据框中展平复杂的嵌套json

Question

我的输入 json 数据框文件如下所示：

company: array (nullable = true)
|    |-- element: struct (containsNull = true)
|    |    |-- address: struct (nullable = true)
|    |    |    |-- city: string (nullable = true)
|    |    |    |-- county: string (nullable = true)
|    |    |    |-- latitude: string (nullable = true)
|    |    |    |-- line1: string (nullable = true)
|    |    |    |-- line2: string (nullable = true)
|    |    |    |-- longitude: string (nullable = true)
|    |    |    |-- postalCode: long (nullable = true)
|    |    |    |-- state: struct (nullable = true)
|    |    |    |    |-- code: int (nullable = true)
|    |    |    |    |-- name: string (nullable = true)
|    |    |    |-- stateOtherDescription: string (nullable = true)
|    |    |-- addressSourceOther: string (nullable = true)
|    |    |-- addressSourceType: struct (nullable = true)
|    |    |     |-- code: int (nullable = true)
|    |    |     |-- name: string (nullable = true)
|    |    |-- reasons: array (nullable = true)
|    |    |     |-- element: struct (containsNull = true)
|    |    |     |-- improve: string (nullable = true)
|    |    |     |-- far: string (nullable = true)
|    |    |     |-- home: string (nullable = true)

我想使用 spark java 动态地展平它。有人可以帮我解决这个问题吗

Answer 1

查看http://stefanfrings.de/bfUtilities/bfUtilities.zip 中的 de.stefanfrings/parsing/JsonSlurper 类。它读取 JSON 文档并根据内容创建平面 HashMap。