【问题标题】:Hive ACID Transaction properties using hive jdbc connection使用 hive jdbc 连接的 Hive ACID Transaction 属性
【发布时间】:2015-04-01 19:04:52
【问题描述】:

我试图在 Hive 0.14 中执行 hive ACID 事务属性,例如通过 Java 插入、删除和更新。我能够设置所需的 ACID 事务属性。还能够创建具有事务属性的表。但它失败了。下面是示例代码:

public class HiveJdbcClient {
    private static String driverName = "org.apache.hive.jdbc.HiveDriver";

    public static void main(String[] args) throws SQLException {
        try {
            Class.forName(driverName);
        } catch (ClassNotFoundException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            System.exit(1);
        }
        Connection con = DriverManager.getConnection("jdbc:hive2://ipaddress:port/default", "user", "password");
        Statement stmt = con.createStatement();

        stmt.execute("set set hive.support.concurrency=true");
        stmt.execute("set hive.enforce.bucketing=true");
        stmt.execute("set hive.exec.dynamic.partition.mode=nonstrict");
        stmt.execute("set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager");
        stmt.execute("set hive.compactor.initiator.on=true");
        stmt.execute("set hive.compactor.worker.threads=2");



        String tableName = "hive_acid_test";

        stmt.execute("drop table if exists " + tableName);


        stmt.execute("create table " + tableName + " (id int, name string,department string) clustered by (department) into 3 buckets stored as orc TBLPROPERTIES   ('transactional'='true')");


        stmt.execute("insert into table hive_acid_test values(1,'jon','sales')");
    }
}

尝试插入时出现以下异常:

线程“main”java.sql.SQLException 中的异常:处理语句时出错:FAILED:执行错误,从 org.apache.hadoop.hive.ql.exec.mr.MapRedTask 返回代码 1 在 org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:296) 在 promo.hive.sample.HiveJdbcClient.main(HiveJdbcClient.java:49)

相同的插入命令正在命令行中工作。请帮我找出问题。

【问题讨论】:

标签: java hadoop jdbc hive hortonworks-data-platform


【解决方案1】:

我能够重现问题并检查了配置单元服务器日志 - hiveserver2.log。 这现在是一个已知问题 - https://issues.apache.org/jira/browse/HIVE-8326 它已在 Hive 1.1 版中修复

问题与您的 ACID 配置中使用的 DbTxManager 有关。

错误日志:

java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.heartbeat(DbTxnManager.java:254)
        at org.apache.hadoop.hive.ql.exec.Heartbeater.heartbeat(Heartbeater.java:81)
        at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:242)
        at org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:547)
        at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:435)
        at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1364)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1177)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:999)

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2014-09-29
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2023-03-09
    • 1970-01-01
    相关资源
    最近更新 更多