【问题标题】:Why is Apache Orc RecordReader.searchArgument() without success?为什么 Apache Orc RecordReader.searchArgument() 没有成功?
【发布时间】:2019-06-12 09:12:13
【问题描述】:

读取时设置的谓词被下推了,但是从打印结果看好像不行,全部打印出来了,这不是我想要的

我参考了以下解决方案,但没有解决方案 enter link description here为什么?

谢谢!

public class parseOrcFile {

    public static void main(String[] args) {
        Configuration conf = new Configuration();
        try {
            String file_path = "/apps/hive/warehouse/orc_stu/test.orc";
            ReaderOptions readerOptions = OrcFile.readerOptions(conf);
            Path path = new Path(file_path);
            Reader reader = OrcFile.createReader(path, readerOptions);
            List<StripeInformation> sis = reader.getStripes();
            TypeDescription schema = reader.getSchema();
            SearchArgument sarg = SearchArgumentFactory.newBuilder()
                        .startNot()
                        .lessThan("id", PredicateLeaf.Type.LONG, 100L)
                        .end()
                        .startAnd()
                        .lessThan("id", PredicateLeaf.Type.LONG, 200L)
                        .end()
                        .build();

            Reader.Options opt = reader.options()
                        .schema(schema)
                        .include(new boolean[]{true, true, true, true, true})
                        .searchArgument(sarg, new String[]{null, "id", "name", "age", "sex"});

            RecordReader read_row_opt = reader.rows(opt);
            VectorizedRowBatch rowBatch = schema.createRowBatch();
            while (read_row_opt.nextBatch(rowBatch)) {
                System.out.println(rowBatch.toString());
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

【问题讨论】:

    标签: java


    【解决方案1】:

    SearchArgument filters 仅限文件、条带、行组。它不会过滤行组中的行。

    【讨论】:

      猜你喜欢
      • 2017-11-25
      • 1970-01-01
      • 1970-01-01
      • 2018-04-20
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2021-08-05
      • 1970-01-01
      相关资源
      最近更新 更多