银河麒麟高级服务器操作系统V10 SP3安装kafka_2.12-2.3.1
1.安装环境设置
1关闭Selinux
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
[root@localhost ~]# vim /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
#SELINUX=enforcing
# SELINUXTYPE= can take one of these three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# ukmls - Multi Level Security protection.
# ukmcs -ukmcs variants of the SELinux policy.
#SELINUXTYPE=targeted
SELINUXTYPE=targeted
# SETLOCALDEFS= Check local definition changes
SETLOCALDEFS=0
|
将selinux配置文件中的“SELINUX=enforcing”更改为“SELINUX=disabled”并注释“#SELINUXTYPE=ukmcs”
| 1 2 |
[root@localhost ~]# setenforce 0 #使配置立即生效
setenforce: SELinux is disabled
|
1.2打开防火墙9092端口
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
[root@localhost ~]# vim /etc/sysconfig/iptables
# sample configuration for iptables service
# you can edit this manually or use system-config-firewall
# please do not ask us to add additional ports/services to this default configuration
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 9092 -j ACCEPT ##增加9092端口放行规则
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
|
| 1 |
[root@localhost ~]# systemctl restart iptables #重启防火墙
|
3、安装kafka
[root@localhost kafka]# find ./libs/ -name *kafka_* | head -1 | grep -o 'kafka[^ ]*' # 查看kafka版本信息
kafka_2.11-1.0.0.jar
[root@localhost 桌面]# wget https://archive.apache.org/dist/kafka/1.0.1/kafka_2.11-1.0.1.tgz #下载kafka
--2023-02-16 16:12:10-- https://archive.apache.org/dist/kafka/1.0.1/kafka_2.11-1.0.1.tgz
正在解析主机 archive.apache.org (archive.apache.org)... 138.201.131.134, 2a01:4f8:172:2ec5::2
正在连接 archive.apache.org (archive.apache.org)|138.201.131.134|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：49766096 (47M) [application/x-gzip]
正在保存至: “kafka_2.11-1.0.1.tgz”
kafka_2.11-1.0.1.tgz 100%[===============================>] 47.46M 18.8KB/s 用时 60m 58s
2023-02-16 17:13:11 (13.3 KB/s) - 已保存 “kafka_2.11-1.0.1.tgz” [49766096/49766096
| 1 |
<br>[root@localhost data]# tar -zxvf kafka_2.11-1.0.1.tgz
|
[root@localhost 桌面]# mv kafka_2.11-1.0.1 /opt/kafka
[root@localhost 桌面]# cd /opt/kafka
[root@localhost kafka]# mkdir -p kafka_data/zookeeper # zookeeper数据
[root@localhost kafka]# mkdir -p kafka_data/log/zookeeper
[root@localhost kafka]# mkdir -p kafka_data/log/kafka # kafka日志 # zookeeper日志
在kafka解压目录同一路径下：创建一个kafka_data，用于装kafka和zookeeper的log和数据。
[root@localhost ~]# cd /opt/kafka
[root@localhost kafka]# ll
总用量 48
drwxr-xr-x 3 root root 4096 10月 28 2017 bin
drwxr-xr-x 2 root root 4096 10月 28 2017 config
drwxr-xr-x 2 root root 4096 2月 20 17:09 libs
-rw-r--r-- 1 root root 28824 10月 27 2017 LICENSE
-rw-r--r-- 1 root root 336 10月 27 2017 NOTICE
drwxr-xr-x 2 root root 44 10月 28 2017 site-docs
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
<br>[root@localhost kafka]# cd config
[root@localhost config]# ll
总用量 64
-rw-r--r-- 1 root root 906 10月 27 2017 connect-console-sink.properties
-rw-r--r-- 1 root root 909 10月 27 2017 connect-console-source.properties
-rw-r--r-- 1 root root 5807 10月 27 2017 connect-distributed.properties
-rw-r--r-- 1 root root 883 10月 27 2017 connect-file-sink.properties
-rw-r--r-- 1 root root 881 10月 27 2017 connect-file-source.properties
-rw-r--r-- 1 root root 1111 10月 27 2017 connect-log4j.properties
-rw-r--r-- 1 root root 2730 10月 27 2017 connect-standalone.properties
-rw-r--r-- 1 root root 1221 10月 27 2017 consumer.properties
-rw-r--r-- 1 root root 4727 10月 27 2017 log4j.properties
-rw-r--r-- 1 root root 1919 10月 27 2017 producer.properties
-rw-r--r-- 1 root root 6852 10月 27 2017 server.properties
-rw-r--r-- 1 root root 1032 10月 27 2017 tools-log4j.properties
-rw-r--r-- 1 root root 1023 10月 27 2017 zookeeper.properties
[root@localhost config]# sudo cp server.properties server-1.properties
[root@localhost config]# sudo cp server.properties server-2.properties
[root@localhost config]# sudo chmod -R 777 /opt/kafka
[root@localhost config]# cd /opt/kafka/bin
[root@localhost bin]# nohup ./zookeeper-server-start.sh ../config/zookeeper.properties &
[1] 13293
[root@localhost bin]# nohup: 忽略输入并把输出追加到'nohup.out'
|
3.2配置kafka服务
| 1 2 |
[root@localhost kafka]# cd config
[root@localhost config]# vim server.properties<br><br>编辑修改相应的参数，内容改动和添加如下：
|
broker.id=0
port=9092 #端口号
host.name=192.168.65.130 #服务器IP地址，修改为自己的服务器IP
log.dirs=/data/kafka/kafka_data/log/kafka #日志存放路径，上面创建的目录
zookeeper.connect=localhost:2181 #zookeeper地址和端口，单机配置部署，localhost:2181
3.3配置zookeeper服务
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
[root@localhost data]# cd kafka
[root@localhost kafka]# ll
总用量 56
drwxr-xr-x 3 root root 4096 2月 22 2018 bin
drwxr-xr-x 2 root root 4096 2月 16 17:33 config
drwxr-xr-x 4 root root 4096 2月 16 17:28 kafka_data
drwxr-xr-x 2 root root 4096 2月 16 17:14 libs
-rw-r--r-- 1 root root 28824 2月 22 2018 LICENSE
-rw-r--r-- 1 root root 336 2月 22 2018 NOTICE
drwxr-xr-x 2 root root 4096 2月 22 2018 site-docs
[root@localhost kafka]# cd config
[root@localhost config]#
[root@localhost config]# vim zookeeper.properties
编辑修改相应的参数，内容改动和添加如下：
dataDir=/data/kafka/kafka_data/zookeeper
dataLogDir=/data/kafka/kafka_data/log/zookeeper
clientPort=2181
maxClientCnxns=100
tickTime=2000
initLimit=10
syncLimit=5
|
3.4启动停止服务
编辑kafka启动脚本
| 1 2 3 4 5 6 7 |
[root@localhost kafka]# vim kafka_start.sh
写入以下内容
#!/bin/sh
/data/kafka/bin/zookeeper-server-start.sh /data/kafka/config/zookeeper.properties &
sleep 3
/data/kafka/bin/kafka-server-start.sh /data/kafka/config/server.properties &
|
编辑kafka停止脚本[root@localhost kafka]# vim kafka_stop.sh
| 1 2 3 4 5 |
写入以下内容
#!/bin/sh
/data/kafka/bin/zookeeper-server-stop.sh /data/kafka/config/zookeeper.properties &
sleep 3
/data/kafka/bin/kafka-server-stop.sh /data/kafka/config/server.properties &<br>
|
为启动、停止脚本赋执行权限
[root@localhost kafka]# chmod +x kafka_start.sh kafka_stop.sh
4.kafka测试
创建生产者 topic 和 消费者 topic 简单示例