首页 > 其他分享 >kafka 配置 SASL_PLAINTEXT 校验

kafka 配置 SASL_PLAINTEXT 校验

时间:2023-02-18 10:12:32浏览次数:86  
标签:flume PLAINTEXT kafka conf SASL security server config

1.配置zookeeper

在kafka的config目录中创建 test.properties

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="test" password="user123456";
security.protocol=sasl_plaintext

 

在kafka的config目录创建 jaas-kafka-client.conf

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="test"
    password="user123456";
};

 

在kafka的bin目录中zookeeper-server-start.sh开头加上,将配置文件路径指向上面jaas-kafka-client.conf

export KAFKA_OPTS="-Djava.security.auth.login.config=/data/software/kafka_2.12-3.4.0/config/jaas-kafka-client.conf"

 

启动zookeeper

bin/zookeeper-server-start.sh config/zookeeper.properties

 

2.配置kafka

修改kafka的config目录下server.properties, 添加下面配置

listeners=SASL_PLAINTEXT://:9092
advertised.listeners=SASL_PLAINTEXT://localhost:9092
authorizer.class.name=kafka.security.authorizer.AclAuthorizer
security.inter.broker.protocol= SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
super.users=User:test

 

在kafka的config目录下新建jaas-kafka-server.conf

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="test"
    password="user123456"
    user_test="user123456";
};

 

修改kafka的bin目录下kafka-server-start.sh,在开头添加如下配置指向jaas-kafka-server.conf

export KAFKA_OPTS="-Djava.security.auth.login.config=/data/software/kafka_2.12-3.4.0/config/jaas-kafka-server.conf"

 

启动kafka

bin/kafka-server-start.sh config/server.properties

 

3.配置用户权限

给用户test分配topic权限

bin/kafka-acls.sh  --bootstrap-server localhost:9092 --add --allow-principal User:test --operation Write --operation Create --topic flume_topic1 --command-config config/test.properties

 

 

 

 创建topic

 

 4.外网访问

修改config目录下server.properties

advertised.listeners=SASL_PLAINTEXT://localhost:9092

 localhost改成服务器外网的ip

 

5.修改flume配置 

在flume的config目录新建 jaas-client.conf

KafkaClient {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="test"
    password="123456";
};

修改sink配置

# Describe/configure the sink
agent1.sinks.sink1.type = org.apache.flume.sink.kafka.KafkaSink
agent1.sinks.sink1.kafka.topic = flume_topic
agent1.sinks.sink1.kafka.bootstrap.servers = localhost:9092
agent1.sinks.sink1.kafka.producer.security.protocol = SASL_PLAINTEXT
agent1.sinks.sink1.kafka.producer.sasl.mechanism = PLAIN
agent1.sinks.sink1.kafka.producer.sasl.kerberos.service.name = kafka

在flume-env.sh添加

export JAVA_OPTS="-Djava.security.auth.login.config=/data/software/apache-flume-1.11.0-bin/conf/jaas-client.conf"

启动flume

bin/flume-ng agent --conf /data/software/apache-flume-1.11.0-bin/conf --conf-file conf/flume-kafka.conf --name agent1

 

 

参考:

https://developer.ibm.com/tutorials/kafka-authn-authz/

 

标签:flume,PLAINTEXT,kafka,conf,SASL,security,server,config
From: https://www.cnblogs.com/AwenDF/p/17130972.html

相关文章

  • Kafka启动失败异常-InconsistentClusterIdException
    一、场景通过Kafka启动命令启动Kafka无响应,每台机器的Zookeeper都已启动,但是Kafka无响应。或者出现命令刚启动的时候jps可见kafka进程,但是过一点时间后又消失不见。 二......
  • Kafka的注解KafkaListener,属性:containerFactory
     kafka的2个topic,如果2个 ConsumerConfig配置不一样。@KafkaListener(id="${groupId}",topics={"TOPIC"},containerFactory="DefinekafkaListenerContainerFacto......
  • springboot接入kafka
    1、windows下安装、启动kafka,这一步只是搭建环境​​https://blog.51cto.com/u_15595167/6026035​​2、安装kafka管理工具:kafkamanager,推荐linux下安装,windows下编译等很麻......
  • kafka单节点安装
    下载地址:https://archive.apache.org/dist/kafka/2.2.2/kafka_2.11-2.2.2.tgz 1.配置server.xml#对外监听地址listeners=PLAINTEXT://172.17.68.115:9092......
  • kafka日志数据清理策略
    vim/kafka/server.properties #日志清理策略优先级是谁先满足条件.#保留7天的日志数据log.retention.hours=168#日志数据总大小保留100G,默认-1无穷大log.retenti......
  • 部署canal同步mysql-binlog到kafka具体操作
    准备[mysqld]log-bin=mysql-bin#开启binlogbinlog-format=ROW#选择ROW模式授权canal链接MySQL账号具有作为MySQLslave的权限CREATEUSERcanalIDENTI......
  • dinky-binlog-kafka-flinksql流程处理
    准确阶段:mysql:开启mysql日志kafka:需检查服务是否正常maxwell:这里采用19版本,过新的版本对java版本要求高,我这里是java8maxwell-1.19.0maxwell操作:cd/root/tar_temp/maxwell-......
  • kafka
    1、名词介绍broker:一台kafka服务器就是一个brokerPartition:是实际存储消息的物理单元,一个Topic内部可以包含多个partition。Topic内部的partition是从0开始,顺序编号,消息......
  • 【Spring-boot-route(十四)整合Kafka】
    kafka简介kafka是用Scala和Java语言开发的,高吞吐量的分布式消息中间件。高吞吐量使它在大数据领域具有天然的优势,被广泛用来记录日志。kafka架构分析注1:图中的红色箭头表示......
  • 决战圣地玛丽乔亚Day10--kafka学习
    概念上kafka和RocketMQ的结构很类似。除了Broker、Producer、Consumer、Topic。多了一个分区的概念Partition。对于NameServer的概念,kafka用的是zookeeper来保存信息。包......