一、概述

Flink's distributed execution consists of two important processes, master and worker. When a Flink program is executed, various processes take part in the execution, namely Job Manager, Task Manager, and Job Client.

image.png

Flink程序需要提交給Job Client。然后，Job Client將作業(yè)提交給Job Manager。Job Manager負(fù)責(zé)協(xié)調(diào)資源分配和作業(yè)執(zhí)行。它首先要做的是分配所需的資源。資源分配完成后，任務(wù)將提交給相應(yīng)的Task Manager。當(dāng)接收到任務(wù)時， Task Manager啟動一個線程以開始執(zhí)行。執(zhí)行到位時，Task Manager會繼續(xù)向Job Manager報告狀態(tài)更改?？梢杂懈鞣N狀態(tài)，例如開始執(zhí)行，正在進行或已完成。作業(yè)執(zhí)行完成后，結(jié)果將發(fā)送回客戶端。

二、Job Manager

• The master processes, also known as Job Managers, coordinate and manage the execution of the program. Their main responsibilities include scheduling tasks, managing checkpoints, failure recovery, and so on.
• There can be many Masters running in parallel and sharing these responsibilities. This helps in achieving high availability. One of the masters needs to be the leader. If the leader node goes down, the master node (standby) will be elected as leader.
• The Job Manager consists of the following important components:
  • 1、Actor system
  • 2、Scheduler
  • 3、Check pointing
• Flink internally uses the Akka actor system for communication between the Job Managers and the Task Managers.

2.1： Actor system(參與者系統(tǒng))

• An actor system is a container of actors with various roles. It provides services such as scheduling, configuration, logging, and so on. It also contains a thread pool from where all actors are initiated. All actors reside in a hierarchy. Each newly created actor would be assigned to a parent. Actors talk to each other using a messaging system. Each actor has its own mailbox from where it reads all the messages. If the actors are local, the messages are shared through shared memory but if the actors are remote then messages are passed thought RPC calls.

Actor system是具有各種角色的actor的容器。它提供諸如調(diào)度，配置，日志記錄等服務(wù)。它還包含一個啟動所有actor的線程池。所有actors都位于層次結(jié)構(gòu)中。每個新創(chuàng)建的actor都將分配給父級。actor使用消息傳遞系統(tǒng)相互交談。每個actor都有自己的郵箱，從中讀取所有郵件。如果actor是本地的，則消息通過共享內(nèi)存共享，但如果actor是遠(yuǎn)程的，則通過RPC調(diào)用傳遞消息。

• Each parent is responsible for the supervision of its children. If any error happens with the children, the parent gets notified. If an actor can solve its own problem then it can restart its children. If it cannot solve the problem then it can escalate the issue to its own parent:

每位家長負(fù)責(zé)監(jiān)督其子女。如果children發(fā)生任何錯誤，父母會收到通知。如果actor可以解決自己的問題，那么它可以重新啟動它的子節(jié)點。如果它無法解決問題，那么它可以將問題升級到自己的父級：

Actor system in Flink

• In Flink, an actor is a container having state and behavior. An actor's thread sequentially keeps on processing the messages it will receive in its mailbox. The state and the behavior are determined by the message it has received.

在Flink中，actor是具有狀態(tài)和行為的容器。 actor的線程依次持續(xù)處理它將在其郵箱中接收的消息。其狀態(tài)和行為由它收到的消息決定。