【问题标题】:Input record does not contain <field> field in data fusion输入记录在数据融合中不包含 <field> 字段
【发布时间】:2021-08-13 04:25:26
【问题描述】:

我正在计算引擎中从 sql server 实例创建一个管道,我想将此数据迁移到 bigquery,但在预览实例中一切正常,我可以毫无问题地观察行,但是当我在 datafusion 中运行部署时实例给我下一个错误

0:c.g.c.h.i.b.BigQueryFactory@76] - Creating BigQuery from given credential.
2021-05-24 21:22:28,967 - INFO  [Executor task launch worker for task 0:c.g.c.h.i.b.BigQueryFactory@76] - Creating BigQuery from default credential.
2021-05-24 21:22:28,978 - INFO  [Executor task launch worker for task 0:c.g.c.h.i.b.BigQueryFactory@76] - Creating BigQuery from given credential.
2021-05-24 21:22:28,991 - INFO  [Executor task launch worker for task 0:c.g.c.h.i.b.o.ForwardingBigQueryFileOutputFormat@76] - Delegating functionality to 'AvroOutputFormat'.
2021-05-24 21:22:31,811 - ERROR [Executor task launch worker for task 0:o.a.s.u.Utils@91] - Aborting task  
**java.lang.IllegalArgumentException:** **Input record does not contain the DatabaseLogId field.** ```

what is your suggestion about this

【问题讨论】:

  • 能否附上完整的堆栈跟踪和管道配置?

标签: google-cloud-platform cloud google-cloud-dataproc google-cloud-data-fusion


【解决方案1】:

单纯使用SQLServer Source --> BQ Table Sink时,输入输出的列名必须一致。

例如:

查询来源:选择dt作为DatabaseLogId

接收器中的列:data_base_log_id(将抛出您的异常)

接收器中的列:DatabaseLogId(正常)

【讨论】:

    猜你喜欢
    • 2011-05-31
    • 2019-12-31
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2019-02-13
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多