sqoop增量导入命令出错

Question

我正在处理一个 sqoop 增量导入命令。但我最后收到错误消息，我不明白问题出在哪里。 下面是我的 MySQL 表数据

+----+-----------+
| ID | NAME      |
+----+-----------+
|  1 | Sidhartha |
|  2 | Sunny     |
|  3 | Saketh    |
|  4 | Bobby     |
|  5 | Yash      |
|  6 | Nimmi     |
+----+-----------+

具有 4 条记录的 Hive 表：DAY 是分区列

importedtable.id    importedtable.name  importedtable.day
1   Sidhartha   1
2   Sunny   1
3   Saketh  1
4   Bobby   1

我的 Sqoop 命令：

sqoop import --connect jdbc:mysql://127.0.0.1/mydb --table MYTAB --driver com.mysql.jdbc.Driver --username root --password cloudera --hive-import --hive-table importedtable --incremental append --check-column id --last-value $(hive -e "select max(id) from importedtable") --target-dir '/home/incdata';

错误信息：

17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Error parsing arguments for import:
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: WARN:
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: The
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: method
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: class
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: org.apache.commons.logging.impl.SLF4JLogFactory#release()
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: was
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: invoked.
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: WARN:
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: Please
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: see
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: http://www.slf4j.org/codes.html#release
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: for
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: an
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: explanation.
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: --target-dir
17/03/08 12:15:14 ERROR tool.BaseSqoopTool: Unrecognized argument: /home/incdata

谁能告诉我我执行 sqoop 命令的错误是什么。

Answer 1

问题在于作为 --last-value 参数的值传递的配置单元查询，

--last-value $(hive -e "select max(id) from importedtable")

这会将日志消息连同结果一起发送到--last-value。

在查询中使用-S (--silent) 标志，

--last-value $(hive -S -e "select max(id) from importedtable")