【发布时间】:2017-12-20 19:05:52
【问题描述】:
我是 spark 新手,我试图为 json 数据定义架构,但在 (spark-shell,
<console>:28: error: overloaded method value apply with alternatives:
(fields: Array[org.apache.spark.sql.types.StructField])org.apache.spark.sql.types.StructType <and>
(fields: java.util.List[org.apache.spark.sql.types.StructField])org.apache.spark.sql.types.StructType <and>
(fields: Seq[org.apache.spark.sql.types.StructField])org.apache.spark.sql.types.StructType
cannot be applied to (org.apache.spark.sql.types.StructField, org.apache.spark.sql.types.StructField)
val schema = StructType(Array(StructField("type", StructType(StructField("name", StringType, true), StructField("version", StringType, true)), true) :: StructField("value", StructType(StructField("answerBlacklistedEntities", StringType, true) :: StructField("answerBlacklistedPhrase", StringType, true) :: StructField("answerEntities", StringType, true) :: StructField("answerText", StringType, true) :: StructField("blacklistReason", StringType, true) :: StructField("blacklistedDomains", StringType, true) :: StructField("blacklistedEntities", ArrayType(StringType, true), true) :: StructField("customerId", StringType, true) :: StructField("impolitePhrase", StringType, true) :: StructField("isResponseBlacklisted", BooleanType, true) :: StructField("queryString", StringType, true) :: StructField("utteranceDomains", StringType, true) :: StructField("utteranceEntities", ArrayType(StringType, true), true) :: StructField("utteranceId", StructType(StructField("identifier", StringType, true)), true)) :: Nil)))
谁能指导我这里发生了什么? :) 非常感谢您的帮助!
【问题讨论】:
标签: scala apache-spark