【发布时间】:2020-06-29 08:04:45
【问题描述】:
在将 csv 数据加载到 bigquery 表时,Web UI/python 客户端库等中有跳过标题行选项。 How to skip rows of csv file in BIGQUERY load API 但我在 BigQuery 的 Java 客户端库中找不到类似的选项。 目前我的代码如下
public long writeFileToTable(String datasetName, String tableName, InputStream inStream, String location)
throws IOException, InterruptedException,TimeoutException {
// [START bigquery_load_from_file]
BigQuery bigquery =BigQueryOptions.getDefaultInstance().getService();
TableId tableId = TableId.of(datasetName, tableName);
WriteChannelConfiguration writeChannelConfiguration =
WriteChannelConfiguration.newBuilder(tableId)
.setFormatOptions(FormatOptions.csv())
.build();
// The location must be specified; other fields can be auto-detected.
JobId jobId = JobId.newBuilder().setLocation(location).build();
TableDataWriteChannel writer = bigquery.writer(jobId, writeChannelConfiguration);
// Write data to writer
try (OutputStream stream = Channels.newOutputStream(writer)) {
IOUtils.copy(inStream, stream);
}
// Get load job
Job job = writer.getJob();
job = job.waitFor();
LoadStatistics stats = job.getStatistics();
return stats.getOutputRows();
// [END bigquery_load_from_file]
}
但这也是在表中写入标题记录。我假设 WriteChannelConfiguration 中应该有一些方法可以做到这一点。但是没找到
【问题讨论】:
标签: java google-cloud-platform google-bigquery