1.动态分区操作
–动态分区插数
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.exec.max.dynamic.partitions=100000;
set hive.exec.max.dynamic.partitions.pernode=100000;
INSERT overwrite table 目标表名 partition(分区字段)
select 表字段,分区字段
from 源表;
2.但是如果遇到想批量插入某些分区的数据的情况,可不可以限制分区范围动态分区插入数据呢?下面来测试一下。
1)建两张测试表。
create table db_test.test_dtfq (a string comment ‘’,b bigint comment ‘’)partitioned by (ds string) STORED AS orc;
create table db_test.test_dtfq_bak (a string comment ‘’,b bigint comment ‘’)partitioned by (ds string) STORED AS orc;
2)造几条测试数据。
insert into db_test.test_dtfq partition(ds=‘2020-07-21’) values(‘1’,1);
insert into db_test.test_dtfq_bak partition(ds=‘2020-06-01’) values(‘4’,4);
insert into db_test.test_dtfq_bak partition(ds=‘2020-06-21’) values(‘2’,2);
insert into db_test.test_dtfq_bak partition(ds=‘2020-06-31’) values(‘3’,3);
3)动态分区插入数据。
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.exec.max.dynamic.partitions=100000;
set hive.exec.max.dynamic.partitions.pernode=100000;
INSERT overwrite table test_dtfq partition(ds)
select
a
,b
, ds
from db_test.test_dtfq_bak
where ds<=‘2020-06-21’
;
4)执行查看结果。
select * from db_test.test_dtfq;
结果显而易见,可以限制某些分区动态插入数据。噢耶!!!