部署模式介绍
在一些应用场景中,对于集群资源分配和占用的方式,可能会有特定的需求。Flink为各种场景提供了不同的部署模式,主要有以下三种:会话模式(Session Mode)、单作业模式(Per-Job Mode)、应用模式(Application Mode)。
它们的区别主要在于:集群的生命周期以及资源的分配方式;以及应用的main方法到底在哪里执行——客户端(Client)还是JobManager。
Standalone运行模式
独立模式是独立运行的,不依赖任何外部的资源管理平台;当然独立也是有代价的:如果资源不足,或者出现故障,没有自动扩展或重分配资源的保证,必须手动处理。所以独立模式一般只用在开发测试或作业非常少的场景下。
精简版脚本:
bin/standalone-job.sh start --job-classname com.atguigu.wc.WordCountStreamUnboundedDemo
bin/taskmanager.sh start
bin/taskmanager.sh stop
bin/standalone-job.sh stop
详细展示版:
[atguigu@node001 ~]$ cd /opt/module/flink/flink-1.17.0/bin
[atguigu@node001 bin]$ ./stop-cluster.sh
Stopping taskexecutor daemon (pid: 2723) on host node001.
Stopping taskexecutor daemon (pid: 2294) on host node002.
Stopping taskexecutor daemon (pid: 2292) on host node003.
Stopping standalonesession daemon (pid: 2380) on host node001.
[atguigu@node001 bin]$ jpsall
================ node001 ================
5120 Jps
================ node002 ================
3212 Jps
================ node003 ================
3159 Jps
[atguigu@node001 bin]$ ls
bash-java-utils.jar flink historyserver.sh kubernetes-session.sh sql-client.sh start-cluster.sh stop-zookeeper-quorum.sh zookeeper.sh
config.sh flink-console.sh jobmanager.sh kubernetes-taskmanager.sh sql-gateway.sh start-zookeeper-quorum.sh taskmanager.sh
find-flink-home.sh flink-daemon.sh kubernetes-jobmanager.sh pyflink-shell.sh standalone-job.sh stop-cluster.sh yarn-session.sh
[atguigu@node001 bin]$ cd ../lib/
[atguigu@node001 lib]$ ls
flink-cep-1.17.0.jar flink-dist-1.17.0.jar flink-table-api-java-uber-1.17.0.jar FlinkTutorial-1.17-1.0-SNAPSHOT.jar log4j-core-2.17.1.jar
flink-connector-files-1.17.0.jar flink-json-1.17.0.jar flink-table-planner-loader-1.17.0.jar log4j-1.2-api-2.17.1.jar log4j-slf4j-impl-2.17.1.jar
flink-csv-1.17.0.jar flink-scala_2.12-1.17.0.jar flink-table-runtime-1.17.0.jar log4j-api-2.17.1.jar
[atguigu@node001 lib]$ cd ../
[atguigu@node001 flink-1.17.0]$ bin/standalone-job.sh start --job-classname com.atguigu.wc.WordCountStreamUnboundedDemo
Starting standalonejob daemon on host node001.
[atguigu@node001 flink-1.17.0]$ jpsall
================ node001 ================
5491 StandaloneApplicationClusterEntryPoint
5583 Jps
================ node002 ================
3326 Jps
================ node003 ================
3307 Jps
[atguigu@node001 flink-1.17.0]$ bin/taskmanager.sh
Usage: taskmanager.sh (start|start-foreground|stop|stop-all)
[atguigu@node001 flink-1.17.0]$ bin/taskmanager.sh start
Starting taskexecutor daemon on host node001.
[atguigu@node001 flink-1.17.0]$ jpsall
================ node001 ================
5491 StandaloneApplicationClusterEntryPoint
5995 Jps
5903 TaskManagerRunner
================ node002 ================
3363 Jps
================ node003 ================
3350 Jps
[atguigu@node001 flink-1.17.0]$ bin/taskmanager.sh stop
Stopping taskexecutor daemon (pid: 5903) on host node001.
[atguigu@node001 flink-1.17.0]$ bin/standalone-job.sh stop
No standalonejob daemon (pid: 5491) is running anymore on node001.
[atguigu@node001 flink-1.17.0]$ xcall jps
=============== node001 ===============
6682 Jps
=============== node002 ===============
3429 Jps
=============== node003 ===============
3419 Jps