如何在 Azure 数据工厂 V2 中仅使用失败的活动/数据集重新运行管道？答案

【问题标题】：How do i Re-run pipeline with only failed activities/Dataset in Azure Data Factory V2?如何在 Azure 数据工厂 V2 中仅使用失败的活动/数据集重新运行管道？
【发布时间】：2018-03-22 12:37:37
【问题描述】：

我正在运行一个管道，在该管道中循环遍历 INFORMATION.SCHEMA.TABLES 中的所有表并将其复制到 Azure Data Lake 存储中。我的问题是，只有在有任何表的情况下，我才如何为失败的表运行此管道复制失败？

【问题讨论】：

有多种方法可以做到这一点，但您必须提供有关您的案例的更多信息。
嗨，马丁，添加了一张图片以供参考。有没有办法只运行失败的活动？提前致谢
是的，有办法，但我认为这不是一个好习惯。为什么会失败？也许我们可以做点什么让它不运行 10 分钟然后失败。
这是一个合理的问题，包含所有需要的详细信息。问题不在于“为什么”工作失败，而在于“如何”从错误中恢复。 ADFv1 和 ADFv2 不同：请添加版本。

标签： azure-data-factory azure-data-factory-2

【解决方案1】：

我发现的最佳方法是将您的流程编码为：

0. Yes, root cause the failure and identify if it is something wrong with the pipeline or if it is a “feature” of your dependency you have to code around.

1. Be idempotent.  If your process ensures a clean state as the very first step, similar to Command Design pattern’s undo (but more naive), then your process can re-execute. 

    * with #1, you can safely use “retry” in your pipeline activities, along with sufficient time between retries.
    * this is an ADFv1 or v2 compatible approach

2. If ADFv2, then you have more options and can have more complex logic to handle errors:

    * for the activity that is failing, wrap this in an until-success loop, and be sure to include a bound on execution.  
    * you can add more activities in the loop to handle failure and log, notify, or resolve known failure conditions due to externalities out of your control.

3. You can also use asynchronous communication to future process executions that save success to a central store.  Then later executions “if” I already was successful then stop processing before the activity.  

    * this is powerful for more generalized pipelines, since you can choose where to begin

4. Last resort I know (and I would love to learn new ways to handle) is manual re-execution of failed activities.

希望这会有所帮助， J

【讨论】：

第二个选项，将其包装到单元成功循环中对我有用。关于您的第四点，我看不到任何手动运行失败活动的选项。 PIpeline Re-run 选项存在，但不针对失败的活动。如果您也能对此有所了解，将不胜感激。谢谢。