尝试在配置单元中创建外部表时出错答案

【问题标题】：Error while trying to create external table in hive尝试在配置单元中创建外部表时出错
【发布时间】：2020-11-26 18:53:36
【问题描述】：

我正在尝试使用带有 hadoop 的 hive 创建一个外部表，但不知何故它失败了。这些是我尝试运行查询时遇到的错误。

02:23:29.516 [HiveServer2-Background-Pool: Thread-39] ERROR hive.ql.exec.DDLTask - org.apache.hadoop.hive.ql.metadata.HiveException: Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
        at org.apache.hadoop.hive.ql.exec.DDLTask.validateSerDe(DDLTask.java:3858)
        at org.apache.hadoop.hive.ql.plan.CreateTableDesc.toTable(CreateTableDesc.java:700)
        at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3960)
        at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:333)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077)
        at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235)
        at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90)
        at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
        at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: Class org.openx.data.jsonserde.JsonSerDe not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2329)
        at org.apache.hadoop.hive.ql.exec.DDLTask.validateSerDe(DDLTask.java:3852)
        ... 22 more

我该如何解决？

【问题讨论】：

您能发布您的CREATE TABLE 脚本吗？

标签： json hadoop hive hive-serde hiveddl

【解决方案1】：

异常说

java.lang.ClassNotFoundException: 类 org.openx.data.jsonserde.JsonSerDe 未找到

安装 JSONSerDe（从 http://www.congiu.net/hive-json-serde/ 下载 JARS 并将它们放入 hive/lib 中），在此处阅读说明：Hive-JSON-Serde

您也可以尝试在 hive 会话中添加 jars，而不是将 jars 放入 hive/lib：

ADD JAR ADD JAR /usr/lib/hive/lib/json-serde-1.3.8-jar-with-dependencies.jar;
ADD JAR ADD JAR /usr/lib/hive/lib/json-udf-1.3.8-jar-with-dependencies.jar;

或者，您可以尝试原生 Hive JSONSerDe：org.apache.hive.hcatalog.data.JsonSerDe - 只需更改表 DDL SerDe 中的类名。它应该已经安装了。在此处阅读有关差异的更多详细信息：https://docs.aws.amazon.com/athena/latest/ug/json-serde.html

【讨论】：