【发布时间】:2021-09-22 08:26:39
【问题描述】:
我有下面的 scala Row,作为 Java 代码中 JavaRdd.map() 调用的一部分。
["A",4,[WrappedArray(["B"],["C"]),WrappedArray(["D"])]]
架构:
{"type":"struct","fields":[{"name":"id","nullable":false,"type":"string","metadata":{}},{"name":"mid","nullable":false,"type":"long","metadata":{}},{"name":"cr","nullable":false,"type":{"type":"struct","fields":[{"name":"psd","type":{"type":"array","elementType":{"type":"struct","fields":[{"name":"cid","type":"string","nullable":true,"metadata":{}}]},"containsNull":true},"nullable":true,"metadata":{}},{"name":"pqd","type":{"type":"array","elementType":{"type":"struct","fields":[{"name":"cid","type":"string","nullable":true,"metadata":{}}]},"containsNull":true},"nullable":true,"metadata":{}}]},"metadata":{}}]}
我想用上面一行的内容创建一个新行,只需将字符串“X”添加到 WrappedArray 的内部元素。
["A",4,[WrappedArray(["X", "B"],["X", "C"]),WrappedArray(["X", D"])]]
架构:
{"type":"struct","fields":[{"name":"id","nullable":false,"type":"string","metadata":{}},{"name":"mid","nullable":false,"type":"long","metadata":{}},{"name":"cr","nullable":false,"type":{"type":"struct","fields":[{"name":"psd","type":{"type":"array","elementType":{"type":"struct","fields":[{"name":"cid","type":"string","nullable":true,"metadata":{}},{"name":"renamed","type":"string","nullable":true,"metadata":{}}]},"containsNull":true},"nullable":true,"metadata":{}},{"name":"pqd","type":{"type":"array","elementType":{"type":"struct","fields":[{"name":"cid","type":"string","nullable":true,"metadata":{}},{"name":"renamed","type":"string","nullable":true,"metadata":{}}]},"containsNull":true},"nullable":true,"metadata":{}}]},"metadata":{}}]}
如何通过向 WrappedArray 内的现有结构添加额外数据来创建新行。
【问题讨论】:
标签: java scala apache-spark apache-spark-sql