centos7直接部署添加认证的kafka

前言

测试服务器：10.255.60.149

一. 安装jdk

官网下载jdk1.8版本以上的

https://www.oracle.com/java/technologies/downloads/

测试系统版本为centos7，选择了最后一个下载后，使用rpm -ivh即可安装

二. 安装zookeeper和kafka

软件版本：kafka_2.12-2.4.0（带zookeeper)

下载链接：http://archive.apache.org/dist/kafka/2.4.0/kafka_2.12-2.4.0.tgz

将安装包上传到服务器并解压到/data目录

tar zxvf kafka_2.12-2.4.0.tgz -C /data
mv kafka_2.12-2.4.0 kafka

三. 配置SASL_PLAINTEXT账号密码认证

1.在config目录新建下面3个文件

kafka_server_jaas.conf
kafka_client_jaas.conf
kafka_zoo_jaas.conf

kafka_server_jaas.conf内容如下

KafkaServer {
    org.apache.kafka.common.security.plain.PlainLoginModule required
    username="admin"
    password="admin"
    user_admin="admin";
};
KafkaClient {
	org.apache.kafka.common.security.plain.PlainLoginModule required
		username="admin"
		password="admin";
};

Client {
	org.apache.kafka.common.security.plain.PlainLoginModule required
		username="admin"
		password="admin";
};

kafka_client_jaas.conf内容如下：

KafkaClient {  
org.apache.kafka.common.security.plain.PlainLoginModule required  
    username="admin"  
    password="admin";  
};

kafka_zoo_jaas.conf内容如下：

ZKServer{
	org.apache.kafka.common.security.plain.PlainLoginModule required
		username="admin"
		password="admin"
		user_admin="admin";
};

2.修改kafka的bin文件夹中的sh文件

1) 修改zookeeper-server-start.sh
添加下面一行，不要放在最后
export KAFKA_OPTS=" -Djava.security.auth.login.config=/data/kafka/config/kafka_zoo_jaas.conf -Dzookeeper.sasl.serverconfig=ZKServer"

2) 修改kafka-server-start.sh
添加下面一行，不要放在最后
export KAFKA_OPTS="-Djava.security.auth.login.config=/data/kafka/config/kafka_server_jaas.conf"

3) 修改kafka-console-producer.sh
添加
export KAFKA_OPTS=" -Djava.security.auth.login.config=/data/kafka/config/kafka_client_jaas.conf"

4) 修改kafka-console-consumer.sh
添加
export KAFKA_OPTS=" -Djava.security.auth.login.config=/data/kafka/config/kafka_client_jaas.conf"

3.修改config目录下文件

1）修改server.properties， 在server basics配置块中修改后的内容如下
broker.id=0
host.name=10.255.60.149
#绑定内网IP
listeners=SASL_PLAINTEXT://10.255.60.149:9092
#绑定外网IP
advertised.listeners=SASL_PLAINTEXT://10.255.60.149:9092

security.inter.broker.protocol=SASL_PLAINTEXT
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
# 配置ACL入口类
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
# 设置admin超级用户
super.users=User:admin
#设置为true，ACL机制为黑名单机制，只有黑名单中的用户无法访问
#默认为false，ACL机制为白名单机制，只有白名单中的用户可以访问
allow.everyone.if.no.acl.found=false


2) consumer.properties和 producer.properties 添加如下
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN

3)zookeeper.properties 添加如下：
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000

四. 启动服务并验证

1.启动zookeeper

/data/kafka/bin/zookeeper-server-start.sh -daemon /data/kafka/config/zookeeper.properties  

2.启动kafka

/data/kafka/bin/kafka-server-start.sh -daemon /data/kafka/config/server.properties 

3.客户端来连接

#开启生产者
/data/kafka/bin/kafka-console-producer.sh --broker-list 10.255.60.149:9092 --topic test --producer-property security.protocol=SASL_PLAINTEXT --producer-property sasl.mechanism=PLAIN

#开启消费者
/data/kafka/bin/kafka-console-consumer.sh --bootstrap-server 10.255.60.149:9092 --topic test --from-beginning --consumer-property security.protocol=SASL_PLAINTEXT --consumer-property sasl.mechanism=PLAIN

4.生产消息并在消费者中查看

生产消息

消费者查看

五. 设置systemctl启动服务

1.创建zookeeper.service

cd  /lib/systemd/system/
vim  zookeeper.service 
[Unit]
Description=Zookeeper service
After=network.target

[Service]
Type=simple
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/java/bin"
User=root
Group=root
ExecStart=/data/kafka/bin/zookeeper-server-start.sh /data/kafka/config/zookeeper.properties
ExecStop=/data/kafka/bin/zookeeper-server-stop.sh
Restart=on-failure

[Install]
WantedBy=multi-user.target

2.创建kafka.service

cd  /lib/systemd/system/
vim kafka.service 
[Unit]
Description=Apache Kafka server (broker)
After=network.target  zookeeper.service

[Service]
Type=simple
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/java/bin"
User=root
Group=root
ExecStart=/data/kafka/bin/kafka-server-start.sh /data/kafka/config/server.properties
ExecStop=/data/kafka/bin/kafka-server-stop.sh
Restart=on-failure

[Install]
WantedBy=multi-user.target

3. 刷新配置并加入开机自启

systemctl daemon-reload

systemctl enable zookeeper
systemctl enable kafka

systemctl start zookeeper
systemctl start kafka

参考文档

https://blog.csdn.net/zy517863543/article/details/103864537

https://blog.csdn.net/d1240673769/article/details/124042854

 ./kafka-console-producer.sh --topic csdn01 --broker-list 10.255.60.149:9092 --producer.config /data/kafka/config/producer.properties

输出