标签：数仓 http LoggerServer jar 离线 Hadoop Application hadoop1
- 整体架构图
- 服务台账
| |
hadoop1 |
hadoop2 |
hadoop3 |
| |
172.23.112.160 |
172.23.112.161 |
172.23.112.162 |
| HDFS |
NameNode
DataNode
HDFS NameNode HTTP UI：9870
HDFS DataNode HTTP UI：9864 |
DataNode |
SecondaryNameNode
DataNode |
| |
http://hadoop1:9870/dfshealth.html#tab-overview |
|
http://hadoop3:9868/status.html |
| YARN |
NodeManager
8042端口 |
ResourceManager
NodeManager |
NodeManager |
| |
|
http://hadoop2:8088/cluster |
|
| JOBHISTORY |
historyserver |
|
|
| |
http://hadoop1:19888/jobhistory |
|
|
| 日志mock |
user_action.jar |
user_action.jar |
|
| flume-job实时toKfka |
Application |
Application |
|
| flume-job实时toHdfs |
|
|
Application-log |
| 业务mock |
business_db.jar |
|
|
| maxwell实时任务toKafka |
Maxwell |
|
|
| maxwell离线增量+flume-job实时 |
|
|
Application-db |
| dataX离线全量 |
每日定时提交任务 |
|
|
| hive |
metaStoreServer hive2Server |
metaStoreServer hive2Server |
metaStoreServer hive2Server |
| spark/sparkSql |
4040端口 |
|
|
| dolphinscheduler |
MasterServer |
|
|
| |
WorkerServer |
WorkerServer |
WorkerServer |
| |
LoggerServer |
LoggerServer |
LoggerServer |
| |
AlertServer |
|
|
| |
ApiApplicationServer |
|
|
| |
http://hadoop1:12345/dolphinscheduler |
|
|
- 按步骤解析
标签：数仓,
http,
LoggerServer,
jar,
离线,
Hadoop,
Application,
hadoop1
From： https://www.cnblogs.com/xuetieqi/p/17294950.html