无法获取嵌套的 json 值作为列

Question

我正在尝试为 json 创建模式，并将其视为数据框中的列

输入json

{"place":{"place_name":"NYC","lon":0,"lat":0,"place_id":1009}, "region":{"region_issues":[{"key":"health","issue_name":"Cancer"},{"key":"sports","issue_name":"swimming"}}}

代码

  val schemaRsvp =  new StructType()
      .add("place",  StructType(Array(
      StructField("place_name", DataTypes.StringType),
      StructField("lon", DataTypes.IntegerType),
      StructField("lat", DataTypes.IntegerType),
      StructField("place_id", DataTypes.IntegerType))))

 val ip =  spark.read.schema(schemaRsvp).json("D:\\Data\\rsvp\\inputrsvp.json")
 ip.show()

它在单列place 中显示所有字段，希望按列显示值

place_name,lon,lat,place_id
NYC,0,0,1009

任何建议，如何解决这个问题？

Answer 1

您可以使用“.*”将结构类型转换为列

ip.select("place.*").show()

+----------+---+---+--------+
|place_name|lon|lat|place_id|
+----------+---+---+--------+
|       NYC|  0|  0|    1009|
+----------+---+---+--------+

更新：

使用新的列数组，您可以展开日期，然后执行相同的“.*”将结构类型转换为列：

ip.select(col("place"), explode(col("region.region_issues")).as("region_issues"))
  .select("place.*", "region_issues.*").show(false)

+---+---+--------+----------+----------+------+
|lat|lon|place_id|place_name|issue_name|key   |
+---+---+--------+----------+----------+------+
|0  |0  |1009    |NYC       |Cancer    |health|
|0  |0  |1009    |NYC       |swimming  |sports|
+---+---+--------+----------+----------+------+