首页 > 数据库 >redis-shake4迁移数据(蛮好用)

redis-shake4迁移数据(蛮好用)

时间:2024-08-30 10:49:07浏览次数:12  
标签:set shake4 redis cluster shake false 迁移 true

redis-shake下载地址:https://github.com/tair-opensource/RedisShake/releases

测试环境redis需要从公有云迁移到私有云,就用redis-share测试一下。

下载后解压(就两个文件,执行文件和配置文件):

 

[root@gcv-x-test-sharedb-oracle redis]# cat shake.toml
##我只需要复制db0得数据,然后目标库存到db42
function = """
shake.log(DB)
if DB == 1
then
    return
end

local db1 = 0
local db2 = 42

if DB == db1 then
  DB = db2
elseif DB == db2 then
  DB = db1
end
shake.call(DB, ARGV)

"""

##需要PSync 协议,但是一般云厂商都是禁用了SYNC和PSYNC命令
#[sync_reader]
#cluster = false            # set to true if source is a redis cluster
#address = "10.27.137.74:6379" # when cluster is true, set address to one of the cluster node
#username = ""              # keep empty if not using ACL
#password = "xxxxxx"              # keep empty if no authentication is required
#tls = false
#sync_rdb = true # set to false if you don't want to sync rdb
#sync_aof = true # set to false if you don't want to sync aof

##我只能用这个模式scan
[scan_reader]
cluster = false            # set to true if source is a redis cluster
address = "10.27.137.74:6379" # when cluster is true, set address to one of the cluster node
username = ""              # keep empty if not using ACL
password = "xxxxx"              # keep empty if no authentication is required
ksn = false                # set to true to enabled Redis keyspace notifications (KSN) subscription
tls = false

# [rdb_reader]
# filepath = "/tmp/dump.rdb"

[redis_writer]
cluster = false            # set to true if target is a redis cluster
address = "10.251.93.8:6379" # when cluster is true, set address to one of the cluster node
username = ""              # keep empty if not using ACL
password = "5xxx"              # keep empty if no authentication is required
tls = false


[advanced]
dir = "data"
ncpu = 2        # runtime.GOMAXPROCS, 0 means use runtime.NumCPU() cpu cores
pprof_port = 0  # pprof port, 0 means disable
status_port = 0 # status port, 0 means disable

# log
log_file = "shake.log"
log_level = "info"     # debug, info or warn
log_interval = 5       # in seconds

# redis-shake gets key and value from rdb file, and uses RESTORE command to
# create the key in target redis. Redis RESTORE will return a "Target key name
# is busy" error when key already exists. You can use this configuration item
# to change the default behavior of restore:
# panic:   redis-shake will stop when meet "Target key name is busy" error.
# rewrite: redis-shake will replace the key with new value.
# ignore:  redis-shake will skip restore the key when meet "Target key name is busy" error.
rdb_restore_command_behavior = "panic" # panic, rewrite or skip

# redis-shake uses pipeline to improve sending performance.
# This item limits the maximum number of commands in a pipeline.
pipeline_count_limit = 1024

# Client query buffers accumulate new commands. They are limited to a fixed
# amount by default. This amount is normally 1gb.
target_redis_client_max_querybuf_len = 1024_000_000

# In the Redis protocol, bulk requests, that are, elements representing single
# strings, are normally limited to 512 mb.
target_redis_proto_max_bulk_len = 512_000_000

# If the source is Elasticache or MemoryDB, you can set this item.
aws_psync = "" # example: aws_psync = "10.0.0.1:6379@nmfu2sl5osync,10.0.0.1:6379@xhma21xfkssync"

[module]
# The data format for BF.LOADCHUNK is not compatible in different versions. v2.6.3 <=> 20603
target_mbbloom_version = 20603

 

 

然后执行即可:

 日志:

 

工具蛮好用的,简单易懂实用。

参数和原理可看官网说明,写的蛮清楚得。https://tair-opensource.github.io/RedisShake/zh/guide/introduction.html

 

标签:set,shake4,redis,cluster,shake,false,迁移,true
From: https://www.cnblogs.com/muzisanshi/p/18388204

相关文章

  • openGauss-反向迁移gs_replicate
    openGauss-反向迁移gs_replicate特性简介支持openGauss的增量数据在线迁移至MySQL。客户价值反向迁移是指用户将源端数据库迁移到目标数据库,应用切到目标数据库后,再将目标端新产生的数据迁移回源端。反向迁移可满足用户业务迁移逃生的诉求,保持源端、目标端两个库并行运行,在目......
  • openGauss-反向全量迁移
    openGauss-反向全量迁移特性简介本特性自openGauss5.1.0版本开始引入,支持openGauss的全量数据迁移至MySQL。客户价值反向迁移可满足用户业务迁移逃生的诉求。实现openGauss数据库全量数据迁移至MySQL数据库,可最大程度保持兼容性。特性描述debeziumopengaussconnector的s......
  • Redis高可用方案:使用Keepalived实现主备双活
    注意:请确保已经安装Redis和keepalived,本文不在介绍如何安装。1、使用版本说明Redis版本:5.0.2Keepalived版本:1.3.5Linux版本:Centos7.9查看Redis版本:/usr/local/redis/bin/redis-cli-v查看Keepalived版本信息:rpm-qa|grepkeepalived或者keepalived-v2、功能实现......
  • 利用 Redisson 实现延迟消息队列:一种高效订单取消方案
    文章目录一、发送延迟消息:定时触发订单取消二、监听延迟队列:自动处理过期订单三、取消订单的实现逻辑四、总结在电商平台中,订单生成后如果长时间未被处理,我们通常需要自动取消这些订单。这种需求不仅能够提升用户体验,还能有效管理库存和资源分配。而如何实现这一需......
  • 单线程Redis:Redis为什么这么快
    1Redis是不是单线程Redis使用后台线程执行耗时的I/O操作,以免阻塞主线程bio_close_file:后台I/O线程之一,异步关闭文件bio_aof_fsync:后台I/O线程之一,异步地将AOF(AppendOnlyFile)日志同步到磁盘bio_lazy_free:异步释放内存,有些内存释放操作可能比较耗时,因此这些操......
  • Redis组件介绍(二)
    写在前面今天我们继续学习Redis。Redis常用数据类型在Redis中,每个存储的键值对中,键是String类型,值可以是不同的数据类型。Redis的索引是含头含尾的。String内存模型常用指令设置值SETkeyvalue:设置一个key/value。MSETkey1value1[key2value2...]:一......
  • 数据迁移新技能,MongoDB轻松同步至ClickHouse
    在当今数据驱动的世界中,企业的成功依赖于对数据的高效管理和精准分析。数据迁移是实现这些目标的关键环节,而选择合适的工具可以让这项工作变得更加轻松和高效。ETLCloud是一款创新的ETL(提取、转换、加载)工具,它提供了简单直观的界面和强大的功能,专门为现代数据集成挑战设计。它不......
  • 数据迁移新技能,MongoDB轻松同步至ClickHouse
    在当今数据驱动的世界中,企业的成功依赖于对数据的高效管理和精准分析。数据迁移是实现这些目标的关键环节,而选择合适的工具可以让这项工作变得更加轻松和高效。ETLCloud是一款创新的ETL(提取、转换、加载)工具,它提供了简单直观的界面和强大的功能,专门为现代数据集成挑战设计。它不......
  • Redis 数据类型详解
    Redis是一个开源的内存数据结构存储系统,广泛应用于缓存、消息队列、实时数据分析等场景。Redis提供了多种数据类型,本文将详细介绍Redis的五种主要数据类型及其应用场景,并从概述、基本操作、应用场景和数据结构等方面进行深入探讨。1.字符串(String)概述字符串是Redis......
  • Redis 数据类型详解
    Redis是一个开源的内存数据结构存储系统,广泛应用于缓存、消息队列、实时数据分析等场景。Redis提供了多种数据类型,本文将详细介绍Redis的五种主要数据类型及其应用场景,并从概述、基本操作、应用场景和数据结构等方面进行深入探讨。1.字符串(String)概述字符串是Redis......