首页 > 其他分享 >DataSophon集成DolphinScheduler-3.1.9升级手册

DataSophon集成DolphinScheduler-3.1.9升级手册

时间：2024-12-10 23:13:09浏览次数：8  
标签：DataSophon false name DolphinScheduler server 3.1 dolphinscheduler true

DataSophon集成DolphinScheduler-3.1.9升级手册

下载安装包并进行压缩

DolphinScheduler下载

wget -O  /opt/datasophon/DDP/packages/apache-dolphinscheduler-3.1.9-bin.tar.gz https://archive.apache.org/dist/dolphinscheduler/3.1.9/apache-dolphinscheduler-3.1.9-bin.tar.gz 

cd /opt/datasophon/DDP/packages/
tar -xvf ./apache-dolphinscheduler-3.1.9-bin.tar.gz

修改安装包目录名称

保持和service_ddl.json中 decompressPackageName 一致

mv apache-dolphinscheduler-3.1.9-bin dolphinscheduler-3.1.9

增加jmx文件夹

mkdir -p /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/jmx

cp -r /opt/datasophon/hadoop-3.3.3/jmx/jmx_prometheus_javaagent-0.16.1.jar /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/jmx/

vi /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/jmx/prometheus_config.yml

---
lowercaseOutputLabelNames: true
lowercaseOutputName: true
whitelistObjectNames: ["java.lang:type=OperatingSystem"]
blacklistObjectNames: []
rules:
  - pattern: 'java.lang<type=OperatingSystem><>(committed_virtual_memory|free_physical_memory|free_swap_space|total_physical_memory|total_swap_space)_size:'
    name: os_$1_bytes
    type: GAUGE
    attrNameSnakeCase: true
  - pattern: 'java.lang<type=OperatingSystem><>((?!process_cpu_time)\w+):'
    name: os_$1
    type: GAUGE
    attrNameSnakeCase: true

修改以下脚本的启动命令使jmx生效

vim ./dolphinscheduler-3.1.9/alert-server/bin/start.sh

JAVA_OPTS=${JAVA_OPTS:-"-server -javaagent:$BIN_DIR/../../jmx/jmx_prometheus_javaagent-0.16.1.jar=12359:$BIN_DIR/../../jmx/prometheus_config.yml  -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"}

vim ./dolphinscheduler-3.1.9/api-server/bin/start.sh

JAVA_OPTS=${JAVA_OPTS:-"-server -javaagent:$BIN_DIR/../../jmx/jmx_prometheus_javaagent-0.16.1.jar=12356:$BIN_DIR/../../jmx/prometheus_config.yml  -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"}

vim ./dolphinscheduler-3.1.9/master-server/bin/start.sh

JAVA_OPTS=${JAVA_OPTS:-"-server -javaagent:$BIN_DIR/../../jmx/jmx_prometheus_javaagent-0.16.1.jar=12357:$BIN_DIR/../../jmx/prometheus_config.yml  -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"}

vim ./dolphinscheduler-3.1.9/worker-server/bin/start.sh

JAVA_OPTS=${JAVA_OPTS:-"-server -javaagent:$BIN_DIR/../../jmx/jmx_prometheus_javaagent-0.16.1.jar=12358:$BIN_DIR/../../jmx/prometheus_config.yml  -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"}

jmx的端口号需要和service_ddl.json中的jmx端口号一致

  • api-server：12356
  • master-server：12357
  • worker-server：12358
  • alert-server：12359

修改bin/dolphinscheduler-daemon.sh脚本

修改dolphinscheduler-3.1.9/bin/dolphinscheduler-daemon.sh脚本，在几乎使最下方的地方$state == "STOP"的地方增加一行exit 1

vi /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/bin/dolphinscheduler-daemon.sh

(status)
    get_server_running_status
    if [[ $state == "STOP" ]]; then
      #  font color - red
      state="[ \033[1;31m $state \033[0m ]"
      #增加一行，使得DataSophon执行脚本时可以有返回值判断状态
      exit 1
    else
      # font color - green
      state="[ \033[1;32m $state \033[0m ]"
    fi
    echo -e "$command  $state"
    ;;

  (*)
    echo $usage
    exit 1
    ;;

添加DolphinScheduler需要的驱动包

MySQL8

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/alert-server/libs/mysql-connector-java-8.0.29.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.29/mysql-connector-java-8.0.29.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/api-server/libs/mysql-connector-java-8.0.29.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.29/mysql-connector-java-8.0.29.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/master-server/libs/mysql-connector-java-8.0.29.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.29/mysql-connector-java-8.0.29.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/standalone-server/libs/mysql-connector-java-8.0.29.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.29/mysql-connector-java-8.0.29.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/tools/libs/mysql-connector-java-8.0.29.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.29/mysql-connector-java-8.0.29.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/worker-server/libs/mysql-connector-java-8.0.29.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.29/mysql-connector-java-8.0.29.jar

参考ds的说明https://github.com/apache/dolphinscheduler/blob/3.1.9-release/docs/docs/zh/guide/howto/datasource-setting.md

commons-cli

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/alert-server/libs/commons-cli-1.9.0.jar https://repo1.maven.org/maven2/commons-cli/commons-cli/1.9.0/commons-cli-1.9.0.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/api-server/libs/commons-cli-1.9.0.jar https://repo1.maven.org/maven2/commons-cli/commons-cli/1.9.0/commons-cli-1.9.0.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/master-server/libs/commons-cli-1.9.0.jar https://repo1.maven.org/maven2/commons-cli/commons-cli/1.9.0/commons-cli-1.9.0.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/standalone-server/libs/commons-cli-1.9.0.jar https://repo1.maven.org/maven2/commons-cli/commons-cli/1.9.0/commons-cli-1.9.0.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/tools/libs/commons-cli-1.9.0.jar https://repo1.maven.org/maven2/commons-cli/commons-cli/1.9.0/commons-cli-1.9.0.jar

wget -O /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/worker-server/libs/commons-cli-1.9.0.jar https://repo1.maven.org/maven2/commons-cli/commons-cli/1.9.0/commons-cli-1.9.0.jar

关闭 Python 网关 【可选】

Python 网关服务会默认与 api-server 一起启动，如果不想启动则需要更改 api-server 配置文件 /opt/packages/apache-dolphinscheduler-3.1.9-bin/api-server/conf/application.yaml 中的 python-gateway.enabled : false 来禁用它。

vim /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/api-server/conf/application.yaml

image-20240906201003921

重新构建安装包

tar -zcf dolphinscheduler-3.1.9.tar.gz dolphinscheduler-3.1.9

md5sum dolphinscheduler-3.1.9.tar.gz > dolphinscheduler-3.1.9.tar.gz.md5

修改DS/service_ddl.json，修改以下参数

image-20241206014811756

vi /opt/datasophon/datasophon-manager-1.2.1/conf/meta/DDP-1.2.1/DS/service_ddl.json

{
  "name": "DS",
  "label": "DolphinScheduler",
  "description": "分布式易扩展的可视化工作流任务调度平台",
  "version": "3.1.9",
  "sortNum": 14,
  "dependencies":["ZOOKEEPER"],
  "packageName": "dolphinscheduler-3.1.9.tar.gz",
  "decompressPackageName": "dolphinscheduler-3.1.9",
  "roles": [
    {
      "name": "ApiServer",
      "label": "ApiServer",
      "roleType": "master",
      "cardinality": "1",
      "logFile": "api-server/logs/dolphinscheduler-api.log",
      "jmxPort": 12356,
      "startRunner": {
        "timeout": "60",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "start",
          "api-server"
        ]
      },
      "stopRunner": {
        "timeout": "600",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "stop",
          "api-server"
        ]
      },
      "statusRunner": {
        "timeout": "60",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "status",
          "api-server"
        ]
      },
      "restartRunner": {
        "timeout": "60",
        "program": "control.sh",
        "args": [
          "restart",
          "api-server"
        ]
      },
      "externalLink": {
        "name": "DolphinScheduler Ui",
        "label": "DolphinScheduler Ui",
        "url": "http://${host}:12345/dolphinscheduler/ui"
      }
    },
    {
      "name": "MasterServer",
      "label": "MasterServer",
      "roleType": "master",
      "cardinality": "1+",
      "logFile": "master-server/logs/dolphinscheduler-master.log",
      "jmxPort": 12357,
      "startRunner": {
        "timeout": "60",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "start",
          "master-server"
        ]
      },
      "stopRunner": {
        "timeout": "600",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "stop",
          "master-server"
        ]
      },
      "statusRunner": {
        "timeout": "60",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "status",
          "master-server"
        ]
      },
      "restartRunner": {
        "timeout": "60",
        "program": "control.sh",
        "args": [
          "restart",
          "master-server"
        ]
      }
    },
    {
      "name": "WorkerServer",
      "label": "WorkerServer",
      "roleType": "worker",
      "cardinality": "1+",
      "logFile": "worker-server/logs/dolphinscheduler-worker.log",
      "jmxPort": 12358,
      "startRunner": {
        "timeout": "60",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "start",
          "worker-server"
        ]
      },
      "stopRunner": {
        "timeout": "600",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "stop",
          "worker-server"
        ]
      },
      "statusRunner": {
        "timeout": "60",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "status",
          "worker-server"
        ]
      },
      "restartRunner": {
        "timeout": "60",
        "program": "control.sh",
        "args": [
          "restart",
          "worker-server"
        ]
      }
    },
    {
      "name": "AlertServer",
      "label": "AlertServer",
      "roleType": "master",
      "cardinality": "1",
      "logFile": "alert-server/logs/dolphinscheduler-alert.log",
      "jmxPort": 12359,
      "startRunner": {
        "timeout": "60",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "start",
          "alert-server"
        ]
      },
      "stopRunner": {
        "timeout": "600",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "stop",
          "alert-server"
        ]
      },
      "statusRunner": {
        "timeout": "60",
        "program": "bin/dolphinscheduler-daemon.sh",
        "args": [
          "status",
          "alert-server"
        ]
      },
      "restartRunner": {
        "timeout": "60",
        "program": "control.sh",
        "args": [
          "restart",
          "alert-server"
        ]
      }
    }
  ],
  "configWriter": {
    "generators": [
      {
        "filename": "dolphinscheduler_env.sh",
        "configFormat": "custom",
        "outputDirectory": "bin/env/",
        "templateName": "dolphinscheduler_env.ftl",
        "includeParams": [
          "databaseUrl",
          "username",
          "password",
          "zkUrls"
        ]
      },
      {
        "filename": "common.properties",
        "configFormat": "properties",
        "outputDirectory": "api-server/conf,master-server/conf,worker-server/conf,alert-server/conf",
        "includeParams": [
          "data.basedir.path",
          "resource.storage.type",
          "resource.storage.upload.base.path",
          "resource.aws.access.key.id",
          "resource.aws.secret.access.key",
          "resource.aws.region",
          "resource.aws.s3.bucket.name",
          "resource.aws.s3.endpoint",
          "resource.alibaba.cloud.access.key.id",
          "resource.alibaba.cloud.access.key.secret",
          "resource.alibaba.cloud.region",
          "resource.alibaba.cloud.oss.bucket.name",
          "resource.alibaba.cloud.oss.endpoint",
          "resource.hdfs.root.user",
          "resource.hdfs.fs.defaultFS",
          "hadoop.security.authentication.startup.state",
          "java.security.krb5.conf.path",
          "login.user.keytab.username",
          "login.user.keytab.path",
          "kerberos.expire.time",
          "resource.manager.httpaddress.port",
          "yarn.resourcemanager.ha.rm.ids",
          "yarn.application.status.address",
          "yarn.job.history.status.address",
          "datasource.encryption.enable",
          "datasource.encryption.salt",
          "data-quality.jar.name",
          "support.hive.oneSession",
          "sudo.enable",
          "setTaskDirToTenant.enable",
          "development.state",
          "alert.rpc.port",
          "conda.path",
          "task.resource.limit.state",
          "ml.mlflow.preset_repository",
          "ml.mlflow.preset_repository_version",
          "custom.common.properties"
        ]
      }
    ]
  },
  "parameters": [
    {
      "name": "databaseUrl",
      "label": "DolphinScheduler数据库地址",
      "description": "",
      "configType": "map",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "jdbc:mysql://${apiHost}:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false"
    },
    {
      "name": "username",
      "label": "DolphinScheduler数据库用户名",
      "description": "",
      "configType": "map",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "dolphinscheduler"
    },
    {
      "name": "password",
      "label": "DolphinScheduler数据库密码",
      "description": "",
      "configType": "map",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "dolphinscheduler"
    },
    {
      "name": "zkUrls",
      "label": "ZK地址",
      "description": "",
      "configType": "map",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "${zkUrls}"
    },
    {
      "name": "data.basedir.path",
      "label": "data.basedir.path",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "/tmp/dolphinscheduler"
    },{
      "name": "resource.storage.type",
      "label": "resource.storage.type",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "NONE"
    },{
      "name": "resource.storage.upload.base.path",
      "label": "resource.storage.upload.base.path",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "/dolphinscheduler"
    },{
      "name": "resource.aws.access.key.id",
      "label": "resource.aws.access.key.id",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "minioadmin"
    },{
      "name": "resource.aws.secret.access.key",
      "label": "resource.aws.secret.access.key",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "minioadmin"
    },{
      "name": "resource.hdfs.root.user",
      "label": "resource.hdfs.root.user",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "hdfs"
    },{
      "name": "resource.hdfs.fs.defaultFS",
      "label": "resource.hdfs.fs.defaultFS",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "${fs.defaultFS}"
    },{
      "name": "resource.manager.httpaddress.port",
      "label": "resource.manager.httpaddress.port",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "8088"
    },{
      "name": "yarn.resourcemanager.ha.rm.ids",
      "label": "yarn.resourcemanager.ha.rm.ids",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "${rmHost}"
    },{
      "name": "yarn.application.status.address",
      "label": "yarn.application.status.address",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "http://ds1:%s/ws/v1/cluster/apps/%s"
    },{
      "name": "yarn.job.history.status.address",
      "label": "yarn.job.history.status.address",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "http://ds1:19888/ws/v1/history/mapreduce/jobs/%s"
    },{
      "name": "datasource.encryption.enable",
      "label": "datasource.encryption.enable",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "false"
    },{
      "name": "datasource.encryption.salt",
      "label": "datasource.encryption.salt",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "!@#$%^&*"
    },{
      "name": "data-quality.jar.name",
      "label": "data-quality.jar.name",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "dolphinscheduler-data-quality-dev-SNAPSHOT.jar"
    },{
      "name": "support.hive.oneSession",
      "label": "support.hive.oneSession",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "false"
    },{
      "name": "sudo.enable",
      "label": "sudo.enable",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "true"
    },{
      "name": "setTaskDirToTenant.enable",
      "label": "setTaskDirToTenant.enable",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "false"
    },{
      "name": "development.state",
      "label": "development.state",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "false"
    },{
      "name": "alert.rpc.port",
      "label": "alert.rpc.port",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "50052"
    },{
      "name": "conda.path",
      "label": "conda.path",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "/opt/anaconda3/etc/profile.d/conda.sh"
    },{
      "name": "task.resource.limit.state",
      "label": "task.resource.limit.state",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "false"
    },{
      "name": "ml.mlflow.preset_repository",
      "label": "ml.mlflow.preset_repository",
      "description": "",
      "required": true,
      "type": "input",
      "value": "",
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": "https://github.com/apache/dolphinscheduler-mlflow"
    },{
      "name": "custom.common.properties",
      "label": "自定义配置common.properties",
      "description": "自定义配置",
      "configType": "custom",
      "required": false,
      "type": "multipleWithKey",
      "value": [],
      "configurableInWizard": true,
      "hidden": false,
      "defaultValue": ""
    }
  ]
}

重启

各节点worker重启

sh /opt/datasophon/datasophon-worker/bin/datasophon-worker.sh restart worker

主节点重启api

sh /opt/datasophon/datasophon-manager-1.2.1/bin/datasophon-api.sh restart api

手动创建数据库并且运行sql

CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
use dolphinscheduler;
source /opt/datasophon/DDP/packages/dolphinscheduler-3.1.9/tools/sql/sql/dolphinscheduler_mysql.sql

安装DolphinScheduler服务

image-20241206214057347

添加服务

image-20241206214657210

根据实际情况，选择ApiServer、MasterServer、AlertServer角色部署的节点。这里MasterServer可以选择部署多台

image-20241206214725539

根据实际情况，选择WorkerServer角色部署的节点。这里WorkerServer可以选择部署一台或者多台

image-20241206214332408

根据实际情况修改相关配置。

JPS查看是否安装成功

bigdata1 bigdata2 bigdata3
MasterServer MasterServer
WorkerServer WorkerServer WorkerServer
ApiApplicationServer
AlertServer

标签：DataSophon,false,name,DolphinScheduler,server,3.1,dolphinscheduler,true
From： https://www.cnblogs.com/yixiaocn/p/18598191

相关文章

  • DataSophon1.2.1集成DataX&DataX-Web(多节点)
    DataSophon简单集成DataX&DataX-Web(多节点)DATAX部署环境准备JDK(1.8以上，推荐1.8)Python(2或3都可以，linux自带py2，py3执行脚本会报错，需要修改脚本)ApacheMaven3.x(CompileDataX，如果下载的是官方的压缩包[datax.tar.gz]，不用安装这个，如果是在git拉的项目，打包时需要)安装......
  • DataSophon1.2.1集成DataX&DataX-Web(单节点)
    DataSophon集成DataX&DataX-Web(单节点)DATAX部署环境准备JDK(1.8以上，推荐1.8)Python(2或3都可以，linux自带py2，py3执行脚本会报错，需要修改脚本)ApacheMaven3.x(CompileDataX，如果下载的是官方的压缩包[datax.tar.gz]，不用安装这个，如果是在git拉的项目，打包时需要)安装包编......
  • Apache DolphinScheduler 限制秒级别的定时调度
    背景ApacheDolphinScheduler定时任务配置采用的7位Crontab表达式，分别对应秒、分、时、月天、月、周天、年。在团队日常开发工作中，工作流的定时调度一般不会细化到秒级别。但历史上出现过因配置的疏忽大意而产生故障时间，如应该配置每分钟执行的工作流被配置长了每秒执行，造......
  • USB PD 3.1协议学习
    USBPD3.1那什么是USB_PD_3.1呢？USB3.1是USB的一种传输标准，速度可以达到10Gbit/s，那PD是什么？PD（PowerDelivery）就是电力输送，也就是充放电用的，它可以达到最高100W的功率。1、资料下载地址USB_Power_Deliery_3.1文档下载链接2、Type-C接口信号组信号描述USB3.1RX2+，R......
  • Trimble Business Center 2023.11(TBC2023.11)地理空间办公软件永久许可
    TrimbleBusinessCenter2023.11TrimbleBusinessCenter(TBC)是一个全面综合并可扩展的工具套件，能够用于各种地理空间业务。无论您的业务需要单机许可还是多用户企业许可，TrimbleBusinessCenter都可以为您提供灵活的选项，以满足您的不同需求。TrimbleBusinessCenter为当......
  • 基于 easyExcel 3.1.5依赖的包 实现动态表头 动态表格内容
    1.需求：需要导出的EXCEL示例： 2.依赖：<dependency><groupId>com.alibaba</groupId><artifactId>easyexcel</artifactId><version>3.1.5</version></dependency>3.工具类：package......
  • 【ubuntu】手动更新ubuntu23.10 到24.04 LTS版本
    一、背景   个人有些软件在23.10上安装有问题，所以更新下  二、更新方法有事找官网    https://ubuntu.com/server/docs/how-to-upgrade-your-release 三、更新步骤总体步骤如下：开始之前非常重要的一步，备份资料，以防万一更新失败，资料丢失，备份是个好习惯#......
  • 11月 | Apache DolphinScheduler月度进展总结
    各位热爱ApacheDolphinScheduler的小伙伴们，社区10月份月报更新啦！这里将记录DolphinScheduler社区每月的重要更新，欢迎关注！月度Merge之星感谢以下小伙伴11月份为ApacheDolphinScheduler所做的精彩贡献（排名不分先后）：“@sdhzwc,@GOODBOY008,@kevinjmh,@slimtom95,@deng-jef......
  • Dolphinscheduler DAG核心源码剖析
    背景描述注意:在Dolphinscheduler中，离线任务是有完整的声明周期的，比如说停止、暂停、暂停恢复、重跑等等，都是以DAG(有向无环图的形式进行任务组织)T+1离线任务的。DolphinschedulerDAG实现org.apache.dolphinscheduler.common.graph.DAGDAG三个重要的数据结构://顶点......
  • 用fpc trunk(3.3.1) 编译TMS FNC控件时出现INTERNAL 20231102
    由于fpc trunk一直在增强及调整，用不同时间段的fpc都可能存在兼容问题，如这次用fpctrunk(3.3.1)编译TMSFNC控件时出现Internal20231102，之前的能通过编译的。用最新的fpc编译LCLTMSFNCCorePkg.lpk时出现以下错误： 在fpc源码发现以下一段文字：如果使用泛型等复杂的情况下，locals......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99