将 Databricks 数据帧写入 BLOB 存储

Question

我正在使用 Azure 数据块并已安装 BLOB 存储。 我尝试使用以下方法将我的数据框保存到那里：

df.write.mode("overwrite").format("com.databricks.spark.csv").option("header","true").csv("/mnt/gl")

我收到以下错误：

 shaded.databricks.org.apache.hadoop.fs.azure.AzureException: java.util.NoSuchElementException: An error occurred while enumerating the result, check the original exception for details.

---------------------------------------------------------------------------
Py4JJavaError                             Traceback (most recent call last)
<command-1181559690773266> in <module>()
----> 1 df.write.mode("overwrite").format("com.databricks.spark.csv").option("header","true").csv("/mnt/gl")

/databricks/spark/python/pyspark/sql/readwriter.py in csv(self, path, mode, compression, sep, quote, escape, header, nullValue, escapeQuotes, quoteAll, dateFormat, timestampFormat, ignoreLeadingWhiteSpace, ignoreTrailingWhiteSpace, charToEscapeQuoteEscaping, encoding, emptyValue)
    927                        charToEscapeQuoteEscaping=charToEscapeQuoteEscaping,
    928                        encoding=encoding, emptyValue=emptyValue)
--> 929         self._jwrite.csv(path)
    930 
    931     @since(1.5)

/databricks/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in __call__(self, *args)
   1255         answer = self.gateway_client.send_command(command)
   1256         return_value = get_return_value(
-> 1257             answer, self.gateway_client, self.target_id, self.name)
   1258 
   1259         for temp_arg in temp_args:

/databricks/spark/python/pyspark/sql/utils.py in deco(*a, **kw)

更新： 重新创建 SAS 密钥后，错误现在显示“作业中止”

Answer 1

尝试将您的“csv”更改为“保存”：

df \
.write \
.mode("overwrite") \
.format("com.databricks.spark.csv") \
.option("header","true") \
.save("/mnt/gl")