AWS Glue 上下文不支持 sql 查询答案

【问题标题】：AWS Glue Context does not support sql queryAWS Glue 上下文不支持 sql 查询
【发布时间】：2020-11-23 03:25:40
【问题描述】：

用例：使用 AWS 胶水中的 where 子句传递查询。

source_df = glueContext.read.format("jdbc")
  .option("url","jdbc:oracle:thin://@xxxxx:1521/ORCL")
  .option("user","user")
  .option("password","password")
  .option("dbtable","(Select * from test) as test")
  .option("driver","oracle.jdbc.driver.OracleDriver")
  .load()

我收到以下错误：

Traceback (most recent call last): File "/tmp/comments_jdbc", line 17, in <module> source_df = spark.read.format("jdbc").option("url","jdbc:oracle:thin://xxxxx:1521/ORCL").option("user","user").option("password","password").option("dbtable","(Select * from test) as test").option("driver","oracle.jdbc.driver.OracleDriver").load()

File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 172, in load return self._df(self._jreader.load())

File "/opt/amazon/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__ answer, self.gateway_client, self.target_id, self.name)

File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco return f(*a, **kw)

File "/opt/amazon/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 328, in get_return_value format(target_id, ".", name), value) 

py4j.protocol.Py4JJavaError: An error occurred while calling o77.load. : java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended at 

oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447) at 

oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396) at 

oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951) at 

oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513) at 

oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227) at 

oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531) at 

oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208) at 

oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886) at 

oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175) at

【问题讨论】：

标签： python apache-spark aws-glue

【解决方案1】：

我在您的option 中没有看到dbtable 的任何where 子句，但我尝试直接在Oracle 中运行查询，它给我带来了与您收到的相同的错误。看起来 Oracle 不支持表别名，只允许列。

如果您仍想使用别名，请尝试从 Glue 作业中运行以下查询，或者只选择不带括号的查询。

select test.* from (select * from test) test

或者简单地运行查询，其中包含一些 where 子句：

select * from test where <some_col=value>

【讨论】：