【问题标题】:error in accessing tables from hive using apache drill使用 Apache Drill 从 Hive 访问表时出错
【发布时间】:2018-06-28 07:14:55
【问题描述】:

我正在尝试使用 Drill 从配置单元中的表 abc 中读取数据。为此,我使用下面提到的配置创建了配置单元存储插件

{
"type": "hive",
"enabled": true,
"configProps": {
"hive.metastore.uris": "thrift://<ip>:<port>",
"fs.default.name": "hdfs://<ip>:<port>/",
"hive.metastore.sasl.enabled": "false",
"hive.server2.enable.doAs": "true",
"hive.metastore.execute.setugi": "true"
 }
 }

有了这个,我可以看到 hive 中的数据库,但是当我尝试访问特定数据库中的任何表时

select * from hive.db.abc;

它会抛出以下错误

org.apache.drill.common.exceptions.UserRemoteException:验证 错误:从第 1 行第 15 列到第 1 行第 18 列:对象“abc”不是 在“hive.db”SQL 查询中找到 null [错误 ID: b6c56276-6255-4b5b-a600-746dbc2f3d67 在 centos2.example.com:31010] (org.apache.calcite.runtime.CalciteContextException) 从第 1 行开始, 第 15 列到第 1 行第 18 列:在其中找不到对象“abc” 'hive.db' sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2 sun.reflect.NativeConstructorAccessorImpl.newInstance():62 sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45 java.lang.reflect.Constructor.newInstance():423 org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463 org.apache.calcite.sql.SqlUtil.newContextException():800 org.apache.calcite.sql.SqlUtil.newContextException():788 org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4703 org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127 org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():947 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():928 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2972 org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():267 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2957 org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():267 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3216 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():947 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():928 org.apache.calcite.sql.SqlSelect.validate():226 org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():903 org.apache.calcite.sql.validate.SqlValidatorImpl.validate():613 org.apache.drill.exec.planner.sql.SqlConverter.validate():190 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():630 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():202 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():174 org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():146 org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():84 org.apache.drill.exec.work.foreman.Foreman.runSQL():567 org.apache.drill.exec.work.foreman.Foreman.run():264 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748 由 (org.apache.calcite.sql.validate.SqlValidatorException)对象'abc' 在“hive.db”中找不到 sun.reflect.NativeConstructorAccessorImpl.newInstance0():-2 sun.reflect.NativeConstructorAccessorImpl.newInstance():62 sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45 java.lang.reflect.Constructor.newInstance():423 org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463 org.apache.calcite.runtime.Resources$ExInst.ex():572 org.apache.calcite.sql.SqlUtil.newContextException():800 org.apache.calcite.sql.SqlUtil.newContextException():788 org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4703 org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl():127 org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl():177 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():947 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():928 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2972 org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():267 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2957 org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom():267 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3216 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():947 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():928 org.apache.calcite.sql.SqlSelect.validate():226 org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():903 org.apache.calcite.sql.validate.SqlValidatorImpl.validate():613 org.apache.drill.exec.planner.sql.SqlConverter.validate():190 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode():630 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert():202 org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():174 org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan():146 org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():84 org.apache.drill.exec.work.foreman.Foreman.runSQL():567 org.apache.drill.exec.work.foreman.Foreman.run():264 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748

【问题讨论】:

  • USE HIVE;SHOW TABLES; 的结果是什么?
  • 您使用哪些版本的 Drill 和 Hive?
  • 你好@vitalii 我正在使用 Hive 1.2.2 Apache Drill 1.13.0 关于使用使用 hive;在 webcosnole 中,它显示选择的配置单元,并在使用显示表时显示错误。

标签: plugins hive apache-drill query-engine


【解决方案1】:

您应该升级到较新的 Hive 版本。对于 Drill 1.13,它是 Hive 2.3.2 版本吗?从 Drill-1.13 开始,Drill 利用 2.3.2 版本的 Hive 客户端 [1]。

即将支持 Hive 3.0 版本 [2]。

另外,请按照以下指南为您的环境提供必要的 Hive 插件配置 [3]。您可以省略“hive.metastore.sasl.enabled”、“hive.server2.enable.doAs”和“hive.metastore.execute.setugi”属性,因为您已指定默认值 [4]。关于“hive.metastore.uris”和“fs.default.name”,您应该为它们指定与 hive-site.xml 中相同的值。

[1]https://drill.apache.org/docs/hive-storage-plugin
[2]https://issues.apache.org/jira/browse/DRILL-6604
[3]https://drill.apache.org/docs/hive-storage-plugin/#hive-remote-metastore-configuration
[4]https://github.com/apache/hive/blob/rel/release-2.3.2/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java#L824

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多