【问题标题】:How to skip header row while loading csv data in google BigQuery using client library (java)如何在使用客户端库(java)在谷歌 BigQuery 中加载 csv 数据时跳过标题行
【发布时间】:2020-06-29 08:04:45
【问题描述】:

在将 csv 数据加载到 bigquery 表时,Web UI/python 客户端库等中有跳过标题行选项。 How to skip rows of csv file in BIGQUERY load API 但我在 BigQuery 的 Java 客户端库中找不到类似的选项。 目前我的代码如下

public long writeFileToTable(String datasetName, String tableName, InputStream inStream, String location)
          throws IOException, InterruptedException,TimeoutException {
        // [START bigquery_load_from_file]
        BigQuery bigquery =BigQueryOptions.getDefaultInstance().getService();
        TableId tableId = TableId.of(datasetName, tableName);
        WriteChannelConfiguration writeChannelConfiguration =
            WriteChannelConfiguration.newBuilder(tableId)               
            .setFormatOptions(FormatOptions.csv())

            .build();
        // The location must be specified; other fields can be auto-detected.
        JobId jobId = JobId.newBuilder().setLocation(location).build();
        TableDataWriteChannel writer = bigquery.writer(jobId, writeChannelConfiguration);
        // Write data to writer
        try (OutputStream stream = Channels.newOutputStream(writer)) {

          IOUtils.copy(inStream, stream);
        }
        // Get load job
        Job job = writer.getJob();

        job = job.waitFor();

        LoadStatistics stats = job.getStatistics();
        return stats.getOutputRows();
        // [END bigquery_load_from_file]
      }

但这也是在表中写入标题记录。我假设 WriteChannelConfiguration 中应该有一些方法可以做到这一点。但是没找到

【问题讨论】:

    标签: java google-cloud-platform google-bigquery


    【解决方案1】:

    抱歉,我找到了如下选项

    WriteChannelConfiguration writeChannelConfiguration =
                WriteChannelConfiguration.newBuilder(tableId)               
                .setFormatOptions(CsvOptions.newBuilder().setSkipLeadingRows(1).build())                
                .build();
    

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 2018-02-25
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多