执行spark任务发现 提示信息“ another attempt succeeded”,这是由于设置了spark.speculation为true,该参数的含义如下:

spark another attempt succeeded

参考:

If a task appears to be taking an unusually long time to complete, Spark may launch extra duplicate copies of that task in case they can complete sooner. This is referred to as speculation or speculative execution. If one copy succeeds, the others can be killed.

See the parameters starting with spark.speculation here: https://spark.apache.org/docs/latest/configuration.html

相关文章:

  • 2022-02-05
  • 2022-12-23
  • 2022-12-23
  • 2021-06-02
  • 2021-08-16
  • 2021-05-19
  • 2022-12-23
  • 2022-12-23
猜你喜欢
  • 2022-12-23
  • 2021-05-31
  • 2021-07-07
  • 2021-10-31
  • 2021-12-28
  • 2022-02-15
相关资源
相似解决方案