YARN的执行流程

职责说明

• Resource Manager : the ultimate authority that arbitrates resources among all the applications in the system. 统筹全局，协调资源分配
• Node Manager : the per-machine framework agent who is responsible for containers, monitoring their resource usage (cpu, memory, disk, network) and reporting the same to the ResourceManager/Scheduler.每台机器对应一个NM，监控 Containers 的资源使用情况并报告给RM，定时向RM发送心跳以检查健康状态
• ApplicationMaster : per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks.每个应用程序对应一个AM，向RM申请用于执行task的资源，和NM一起监控执行task的运行情况
• Container : incorporates elements such as memory, cpu, disk, network etc.

流程概述

1. 客户端 Client 提交任务到 Resource Manager
2. Resource Manager 与 Node Manager 进行通信，要求启动一个Container 去执行 task
3. 启动一个 Container 去运行 Application Master
4. Application Master 注册到 Resource Manager，并向RM申请资源
5. 申请到资源后向 Node Manager 请求启动 Container 去执行 task
6. Node Manager 启动 Container 去执行 task