【问题标题】:'0000-00-00 00:00:00' as an invalid timestamp'0000-00-00 00:00:00' 作为无效时间戳
【发布时间】:2017-11-10 08:20:47
【问题描述】:

当我尝试将数据文件加载到 BigQuery 时,它给了我一个错误,MySQL 的默认时间戳值(至少来自 mysqldump)0000-00-00 00:00:00。有没有办法更改我的 json 文件以允许 datetime 字段中的这些值(见下文)?

错误:

gs://pfam31/pfamseq_part_04: CSV 表遇到太多错误,放弃。行数:11664;错误:1.(错误代码:无效)

gs://pfam31/pfamseq_part_04:无法将“0000-00-00 00:00:00”解析为从位置 8743229 开始创建的字段(位置 13)的日期时间(错误代码:无效)

[
  {
    "mode": "REQUIRED",
    "name": "pfamseq_acc",
    "type": "STRING"
  },
  {
    "mode": "REQUIRED",
    "name": "pfamseq_id",
    "type": "STRING"
  },
  {
    "mode": "REQUIRED",
    "name": "seq_version",
    "type": "INTEGER"
  },
  {
    "mode": "REQUIRED",
    "name": "crc64",
    "type": "STRING"
  },
  {
    "mode": "REQUIRED",
    "name": "md5",
    "type": "STRING"
  },
  {
    "mode": "REQUIRED",
    "name": "description",
    "type": "STRING"
  },
  {
    "mode": "REQUIRED",
    "name": "evidence",
    "type": "INTEGER"
  },
  {
    "mode": "REQUIRED",
    "name": "length",
    "type": "INTEGER"
  },
  {
    "mode": "REQUIRED",
    "name": "species",
    "type": "STRING"
  },
  {
    "mode": "NULLABLE",
    "name": "taxonomy",
    "type": "STRING"
  },
  {
    "mode": "NULLABLE",
    "name": "is_fragment",
    "type": "INTEGER"
  },
  {
    "mode": "REQUIRED",
    "name": "sequence",
    "type": "STRING"
  },
  {
    "mode": "REQUIRED",
    "name": "updated",
    "type": "TIMESTAMP"
  },
  {
    "mode": "NULLABLE",
    "name": "created",
    "type": "DATETIME"
  },
  {
    "mode": "REQUIRED",
    "name": "ncbi_taxid",
    "type": "INTEGER"
  },
  {
    "mode": "NULLABLE",
    "name": "auto_architecture",
    "type": "INTEGER"
  },
  {
    "mode": "NULLABLE",
    "name": "treefam_acc",
    "type": "STRING"
  },
  {
    "mode": "NULLABLE",
    "name": "swissprot",
    "type": "INTEGER"
  }
]

【问题讨论】:

    标签: mysql datetime timestamp google-bigquery


    【解决方案1】:

    您不能将年/月/日设置为零。TIMESTAMP 的范围为 '1970-01-01 00:00:01' UTC 到 '2038-01-19 03:14:07' UTC。 更多请查看here

    【讨论】:

    • 来自那些文档“MySQL 不接受在日或月列中包含零的 TIMESTAMP 值或无效日期的值。此规则的唯一例外是特殊的“零”值'0000-00-00 00:00:00'。”
    • 这可能是我在 MySQL 中的有效日期,但它不在 BigQuery 中。您也可以尝试在 CSV 文件上进行字符串替换,以在加载到 BigQuery 之前将这些零值替换为 NULL。
    • 替代方案:作为字符串导入 BigQuery,然后在 BQ 中运行转换以节省 ETL 时间
    猜你喜欢
    • 2019-03-26
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2011-06-27
    • 1970-01-01
    • 2010-11-24
    相关资源
    最近更新 更多