Redislabs的简单使用与benchmark测试结果
自己的理解
- 仅是理解. 没有看过源码
哨兵可以实现高可用, 但是对高吞吐是没有太大帮助的.
虽然可以实现主写从读, 但是在高并发的场景下延迟肯定会增加.
集群模式可以实现hash分片, 虽有有了一个根据key值进行hash的动作
增加了延时, 但是也增加系统容量, 在降低一定响应时间的情况下提高了系统容量.
能够扩展出更大的吞吐量. 降低redis单节点连接数过多的问题.
背景
周六一早和晚上安装了下两个节点的Redislabs
睡觉前简单整理了下redislabs 在openEuler上面的安装.
今天白天继续进行redislabs的相关工作处理
第一步 是让产品能够连的上redislabs 至少可以正常启动
第二步 可以跑一下一些基本的benchmark的方式验证一下性能
第三步 是需要进行功能和性能测试进行简单验证.
第三步我自己做不了,但是前两步应该还行. 然后基于此水这一篇文章.
第一部分: Spingboot连接redis
springboot 连接redis有很多种方式
我们产品复杂, 使用了jedis和redisson两种连接池.
所以需要在两个地方进行设置.
# 不过我感觉太过复杂并不是好事情. 容易产生误解.
jedis可以直接写. 在配置文件中:
caching-configuration:
enableRedis: true
redisManagers:
- name: default
mode: cluster
nodes: 10.110.139.191:14085,10.110.139.192:14085
max-redirects: 8
password: Yourpassword?!
# 注意 mode 是 cluster
# 注意 不是host 是 ndoes
# 注意 没有port了 是在nodes 里面直接添加上了.
第一部分: Spingboot连接redis
redisson的稍微复杂一点,需要单独处理一个文档.
yaml 文件里面需要修改一下配置:
yaml 内部的配置:
redis:
redisson:
config: classpath:redisson.yaml
然后在runtime/libs 目录下面增加
redisson.yaml
第一部分: Spingboot连接redis
- redisson.yaml 的相关配置为:
clusterServersConfig:
# 连接空闲超时,单位:毫秒 默认10000
idleConnectionTimeout: 10000
pingTimeout: 1000
# 同任何节点建立连接时的等待超时。时间单位是毫秒 默认10000
connectTimeout: 10000
# 等待节点回复命令的时间。该时间从命令发送成功时开始计时。默认3000
timeout: 3000
# 命令失败重试次数
retryAttempts: 3
# 命令重试发送时间间隔,单位:毫秒
retryInterval: 6000
# 重新连接时间间隔,单位:毫秒
reconnectionTimeout: 6000
# 执行失败最大次数
failedAttempts: 3
# 密码
password: Yourpasswords?!
# 单个连接最大订阅数量
subscriptionsPerConnection: 5
clientName: null
# loadBalancer 负载均衡算法类的选择
loadBalancer: !<org.redisson.connection.balancer.RoundRobinLoadBalancer> {}
#从节点发布和订阅连接的最小空闲连接数
slaveSubscriptionConnectionMinimumIdleSize: 1
#从节点发布和订阅连接池大小 默认值50
slaveSubscriptionConnectionPoolSize: 50
# 从节点最小空闲连接数 默认值32
slaveConnectionMinimumIdleSize: 32
# 从节点连接池大小 默认64
# 感觉需要同步修改
slaveConnectionPoolSize: 512
# 主节点最小空闲连接数 默认32
masterConnectionMinimumIdleSize: 32
# 主节点连接池大小 默认64
# 赵本帅增加默认值
masterConnectionPoolSize: 512
# 订阅操作的负载均衡模式
subscriptionMode: SLAVE
# 只在从服务器读取
readMode: SLAVE
# 集群地址
nodeAddresses:
- "redis://10.110.139.191:14085"
- "redis://10.110.139.192:14085"
# 对Redis集群节点状态扫描的时间间隔。单位是毫秒。默认1000
scanInterval: 1000
#这个线程池数量被所有RTopic对象监听器,RRemoteService调用者和RExecutorService任务共同共享。默认2
threads: 400
#这个线程池数量是在一个Redisson实例内,被其创建的所有分布式数据类型和服务,以及底层客户端所一同共享的线程池里保存的线程数量。默认2
nettyThreads: 400
# 编码方式 默认org.redisson.codec.JsonJacksonCodec
codec: !<org.redisson.codec.JsonJacksonCodec> {}
#传输模式
transportMode: NIO
# 分布式锁自动过期时间,防止死锁,默认30000
lockWatchdogTimeout: 30000
# 通过该参数来修改是否按订阅发布消息的接收顺序出来消息,如果选否将对消息实行并行处理,该参数只适用于订阅发布消息的情况, 默认true
keepPubSubOrder: true
第一部分: Spingboot连接redis
配置一定准确, 不要有错别字..
配置完成后应该就可以启动了
还可以通过 redislabs 的 8443
端口进行查询性能
第二部分: 性能验证
- 先说结论部分
1. redislabs的吞吐量跟同配置的开源版本并无大的提升.
集群模式下的损耗和shard导致吞吐量有下降.
但是企业版的着重点不在于吞吐量 而在于其他稳定性和监控等.
2. 多线程能够大幅提高吞吐量, 并且能够规避掉仅能够使用一个CPU核心的问题
今天测试了四线程的操作. PING命令的性能提升了接近3倍.MSET的性能提升2倍+
认为有较大的优化空间.
3. OpenEuler2203的5.10的内存的同配置机器的性能表现要优于CentOS7 3.10内核的虚拟机.
5.10内核的机器 最高的吞吐量能到10万. 但是 3.10内核的基本上只有7万左右.
差距比较明显主要说的是 MSET (10 keys) 的性能数据.
4. Redis5.0.4的性能明显比6.2.6的性能表现要好. 原因不明.
第二部分: 性能测试
测试方法 redis-benchmark -h -p -a 导出数据
转储到具体文件 ,然后执行如下语句进行分析
cat xxxx .log |grep -E "requests per second|====="
具体结果可以做成excel 但是大周末的 我只放原始数据算了..
既然是水 就要水到底...
第三部分: 性能测试原始数据
机器配置信息大部分为:
Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
CPU 为8C-16C
内存为 48-64G的虚拟机
磁盘为 HDD RAID6 12块 4T的盘
虚拟机操作系统分别为:
OpenEuler 2203 5.10
CentOS7 2009 3.10
第三部分: 性能测试原始数据
这里准备将元数据放出.
不进行完整的对比了
章节名字就是测试的场景.
redis 5.0.4 单线程 OpenEuler without swap
====== PING_INLINE ======
108813.92 requests per second
====== PING_BULK ======
104931.80 requests per second
====== SET ======
107874.87 requests per second
====== GET ======
105932.20 requests per second
====== INCR ======
108342.37 requests per second
====== LPUSH ======
107642.62 requests per second
====== RPUSH ======
108225.10 requests per second
====== LPOP ======
108342.37 requests per second
====== RPOP ======
105708.25 requests per second
====== SADD ======
105932.20 requests per second
====== HSET ======
109170.30 requests per second
====== SPOP ======
105708.25 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
109170.30 requests per second
====== LRANGE_100 (first 100 elements) ======
58582.31 requests per second
====== LRANGE_300 (first 300 elements) ======
23110.70 requests per second
====== LRANGE_500 (first 450 elements) ======
16641.71 requests per second
====== LRANGE_600 (first 600 elements) ======
12479.72 requests per second
====== MSET (10 keys) ======
110864.74 requests per second
6.2.6 OpenEuler with SWAP
====== PING_INLINE ======
107066.38 requests per second
====== PING_BULK ======
97560.98 requests per second
====== SET ======
106609.80 requests per second
====== GET ======
108459.87 requests per second
====== INCR ======
108342.37 requests per second
====== LPUSH ======
107296.14 requests per second
====== RPUSH ======
107411.38 requests per second
====== LPOP ======
110375.27 requests per second
====== RPOP ======
109649.12 requests per second
====== SADD ======
106382.98 requests per second
====== HSET ======
103092.78 requests per second
====== SPOP ======
105152.48 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
107642.62 requests per second
====== LRANGE_100 (first 100 elements) ======
51948.05 requests per second
====== LRANGE_300 (first 300 elements) ======
22691.17 requests per second
====== LRANGE_500 (first 450 elements) ======
17111.57 requests per second
====== LRANGE_600 (first 600 elements) ======
12597.63 requests per second
====== MSET (10 keys) ======
107066.38 requests per second
redis 6.2.6 OpenEuler 单线程 without Swap
====== PING_INLINE ======
104493.20 requests per second
====== PING_BULK ======
100200.40 requests per second
====== SET ======
105152.48 requests per second
====== GET ======
108108.11 requests per second
====== INCR ======
104058.27 requests per second
====== LPUSH ======
110253.59 requests per second
====== RPUSH ======
106269.93 requests per second
====== LPOP ======
110497.24 requests per second
====== RPOP ======
106496.27 requests per second
====== SADD ======
106496.27 requests per second
====== HSET ======
108695.65 requests per second
====== SPOP ======
106382.98 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
109289.62 requests per second
====== LRANGE_100 (first 100 elements) ======
48875.86 requests per second
====== LRANGE_300 (first 300 elements) ======
21910.60 requests per second
====== LRANGE_500 (first 450 elements) ======
16450.07 requests per second
====== LRANGE_600 (first 600 elements) ======
12280.49 requests per second
====== MSET (10 keys) ======
88417.33 requests per second
redislabs 企业版 OpenEuler 主节点测试主节点
====== PING_INLINE ======
67659.00 requests per second
====== PING_BULK ======
67980.97 requests per second
====== SET ======
63572.79 requests per second
====== GET ======
62814.07 requests per second
====== INCR ======
63331.22 requests per second
====== LPUSH ======
67385.45 requests per second
====== RPUSH ======
67842.61 requests per second
====== LPOP ======
68446.27 requests per second
====== RPOP ======
67704.80 requests per second
====== SADD ======
67704.80 requests per second
====== HSET ======
68073.52 requests per second
====== SPOP ======
67796.61 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
67842.61 requests per second
====== LRANGE_100 (first 100 elements) ======
43497.17 requests per second
====== LRANGE_300 (first 300 elements) ======
18559.76 requests per second
====== LRANGE_500 (first 450 elements) ======
13627.69 requests per second
====== LRANGE_600 (first 600 elements) ======
10636.03 requests per second
====== MSET (10 keys) ======
64641.24 requests per second
Redislabs OpenEuler 从节点测试主节点
====== PING_INLINE ======
70921.98 requests per second
====== PING_BULK ======
75357.95 requests per second
====== SET ======
83056.48 requests per second
====== GET ======
80321.28 requests per second
====== INCR ======
83056.48 requests per second
====== LPUSH ======
80321.28 requests per second
====== RPUSH ======
80710.25 requests per second
====== LPOP ======
76863.95 requests per second
====== RPOP ======
77821.02 requests per second
====== SADD ======
77519.38 requests per second
====== HSET ======
78125.00 requests per second
====== SPOP ======
74794.31 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
79113.92 requests per second
====== LRANGE_100 (first 100 elements) ======
45578.85 requests per second
====== LRANGE_300 (first 300 elements) ======
18037.52 requests per second
====== LRANGE_500 (first 450 elements) ======
13059.95 requests per second
====== LRANGE_600 (first 600 elements) ======
10339.12 requests per second
====== MSET (10 keys) ======
77220.08 requests per second
redislabs OpenEuler 从节点测试从节点
====== PING_INLINE ======
67024.13 requests per second
====== PING_BULK ======
66225.16 requests per second
====== SET ======
65530.80 requests per second
====== GET ======
68212.83 requests per second
====== INCR ======
67934.78 requests per second
====== LPUSH ======
63572.79 requests per second
====== RPUSH ======
62539.09 requests per second
====== LPOP ======
63091.48 requests per second
====== RPOP ======
61881.19 requests per second
====== SADD ======
62893.08 requests per second
====== HSET ======
62344.14 requests per second
====== SPOP ======
63091.48 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
61996.28 requests per second
====== LRANGE_100 (first 100 elements) ======
63051.70 requests per second
====== LRANGE_300 (first 300 elements) ======
62893.08 requests per second
====== LRANGE_500 (first 450 elements) ======
63331.22 requests per second
====== LRANGE_600 (first 600 elements) ======
62617.41 requests per second
====== MSET (10 keys) ======
68634.18 requests per second
6.2.6 centos7 3.10 保护模式 单节点
====== PING_INLINE ======
71073.21 requests per second
====== PING_BULK ======
69979.01 requests per second
====== SET ======
69013.11 requests per second
====== GET ======
71225.07 requests per second
====== INCR ======
72411.30 requests per second
====== LPUSH ======
74019.25 requests per second
====== RPUSH ======
74019.25 requests per second
====== LPOP ======
74183.98 requests per second
====== RPOP ======
74074.07 requests per second
====== SADD ======
71428.57 requests per second
====== HSET ======
73260.07 requests per second
====== SPOP ======
73475.39 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
74074.07 requests per second
====== LRANGE_100 (first 100 elements) ======
43802.02 requests per second
====== LRANGE_300 (first 300 elements) ======
19992.00 requests per second
====== LRANGE_500 (first 450 elements) ======
14723.20 requests per second
====== LRANGE_600 (first 600 elements) ======
11544.68 requests per second
====== MSET (10 keys) ======
71942.45 requests per second
6.2.6 centos7 3.10 非保护模式
====== PING_INLINE ======
71787.51 requests per second
====== PING_BULK ======
71073.21 requests per second
====== SET ======
71994.23 requests per second
====== GET ======
72150.07 requests per second
====== INCR ======
72202.16 requests per second
====== LPUSH ======
72098.05 requests per second
====== RPUSH ======
73099.41 requests per second
====== LPOP ======
73367.57 requests per second
====== RPOP ======
73313.78 requests per second
====== SADD ======
72516.32 requests per second
====== HSET ======
72992.70 requests per second
====== SPOP ======
71994.23 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
73529.41 requests per second
====== LRANGE_100 (first 100 elements) ======
43591.98 requests per second
====== LRANGE_300 (first 300 elements) ======
20036.06 requests per second
====== LRANGE_500 (first 450 elements) ======
14795.09 requests per second
====== LRANGE_600 (first 600 elements) ======
11498.22 requests per second
====== MSET (10 keys) ======
75815.01 requests per second
redis 6.2.6CentOS7 with out password without protected CentOS7 3.10
====== PING_INLINE ======
72621.64 requests per second
====== PING_BULK ======
72833.21 requests per second
====== SET ======
73046.02 requests per second
====== GET ======
72411.30 requests per second
====== INCR ======
74019.25 requests per second
====== LPUSH ======
73421.44 requests per second
====== RPUSH ======
73313.78 requests per second
====== LPOP ======
74294.21 requests per second
====== RPOP ======
73800.73 requests per second
====== SADD ======
72886.30 requests per second
====== HSET ======
72780.20 requests per second
====== SPOP ======
72150.07 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
73909.83 requests per second
====== LRANGE_100 (first 100 elements) ======
44091.71 requests per second
====== LRANGE_300 (first 300 elements) ======
20695.37 requests per second
====== LRANGE_500 (first 450 elements) ======
14779.78 requests per second
====== LRANGE_600 (first 600 elements) ======
11739.85 requests per second
====== MSET (10 keys) ======
75987.84 requests per second
Redis 6.2.6 centos 7 3.10 内核 四线程测试数据
====== PING_INLINE ======
throughput summary: 199203.20 requests per second
====== PING_MBULK ======
throughput summary: 199203.20 requests per second
====== SET ======
throughput summary: 199600.80 requests per second
====== GET ======
throughput summary: 199203.20 requests per second
====== INCR ======
throughput summary: 199600.80 requests per second
====== LPUSH ======
throughput summary: 199203.20 requests per second
====== RPUSH ======
throughput summary: 199203.20 requests per second
====== LPOP ======
throughput summary: 199203.20 requests per second
====== RPOP ======
throughput summary: 199203.20 requests per second
====== SADD ======
throughput summary: 199203.20 requests per second
====== HSET ======
throughput summary: 199203.20 requests per second
====== SPOP ======
throughput summary: 199203.20 requests per second
====== ZADD ======
throughput summary: 199203.20 requests per second
====== ZPOPMIN ======
throughput summary: 199203.20 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
throughput summary: 199203.20 requests per second
====== LRANGE_100 (first 100 elements) ======
throughput summary: 79744.82 requests per second
====== LRANGE_300 (first 300 elements) ======
throughput summary: 44286.98 requests per second
====== LRANGE_500 (first 500 elements) ======
throughput summary: 28457.60 requests per second
====== LRANGE_600 (first 600 elements) ======
throughput summary: 24888.00 requests per second
====== MSET (10 keys) ======
throughput summary: 132802.12 requests per second
标签:elements,benchmark,per,Redislabs,second,LRANGE,测试,requests,first
From: https://www.cnblogs.com/jinanxiaolaohu/p/16818427.html