elasticsearch 数据远程备份与还原

官方文档：

备份集群：https://www.elastic.co/guide/cn/elasticsearch/guide/current/backing-up-your-cluster.html#backing-up-your-cluster

恢复数据：https://www.elastic.co/guide/cn/elasticsearch/guide/2.x/_restoring_from_a_snapshot.html

es官方默认是备份数据到本地的，如果要备份数据到远程服务器，可以采用nfs等实现

安装nfs：https://www.cnblogs.com/rtnb/p/15584847.html

一、配置nfs

cat /etc/exports
/data/es-bak-test  172.16.0.0/16(rw,sync,no_root_squash)

挂载

mkdir /data/es-bak
临时挂载
mount -t nfs 172.16.77.166:/data/es-bak-test /data/es-bak/

永久挂载
172.16.77.166:/data/es-bak-test /data/es-bak  nfs defaults 0 0

更改权限
chown -R es:es /data/es-bak/

二、更改es配置文件，更改权限并重启

vim /data/es7/config/elasticsearch.yml

path.repo: ["/data/es-bak"]

三、备份数据

2、创建存储库

PUT  _snapshot/my_backup

{
  "type": "fs",
  "settings": {
    "location": "/data/es-bak",   # 挂载目录
    "compress": true,    # 压缩
    "chunk_size": "1g",  # 数据块大小
    "max_snapshot_bytes_per_sec": "50m",  # 快照传输速率
    "max_restore_bytes_per_sec": "50m"    # 恢复数据速率
  }
}

# 查看所有存储库

GET   _snapshot/_all

3、拍摄快照

快照拍照分为全量备份与按需备份

全量备份

PUT  _snapshot/my_backup/snapshot_1


{
  "ignore_unavailable": true,   # 快照备份报错继续
  "include_global_state": false  # 不备份集群状态信息
}

按需备份

PUT  _snapshot/my_backup/snapshot_2

{
  "indices": "lexicon_data*,kibana_sample_data_ecommerce*,monitor_data,plan_misre_data,plan_special_data",
  "ignore_unavailable": true,
  "include_global_state": false
}

如果后续需要增量备份，则直接备份即可，比如snapshot_2，那么增量直接写snapshot_3即可。

查看状态

查看单独任务
GET  _snapshot/my_backup/snapshot_2

查看所有任务
GET  _snapshot/my_backup/_all


{
      "snapshot" : "cloud-snapshot-2021.01.07-xqtcabrnt-2qkxxdhhasaa",     //快照名称
      "uuid" : "nOZxM36MScGkBIlbhjRjGg",
      "version_id" : 7100099,
      "version" : "7.10.0",                     //es版本
      "indices" : [                        //快照内包含的索引
        ".kibana_security_session_1",
        ".kibana_task_manager_1",
        ".apm-custom-link",
        ".apm-agent-configuration",
        ".kibana_1",
        ".kibana-event-log-7.10.0-000001",
        "ilm-history-3-000001",
        ".security-7",
        ".security-tokens-7"
      ],
      "data_streams" : [ ],
      "include_global_state" : true,            //带有全局状态（系统索引）
      "metadata" : {
        "policy" : "cloud-snapshot-policy"        //快照策略
      },
      "state" : "SUCCESS",                                //完成状态
      "start_time" : "2021-01-07T07:21:04.723Z",            //开始时间
      "start_time_in_millis" : 1610004064723,                
      "end_time" : "2021-01-07T07:21:11.130Z",            //结束时间
      "end_time_in_millis" : 1610004071130,
      "duration_in_millis" : 6407,
      "failures" : [ ],                                    //失败的索引
      "shards" : {
        "total" : 9,                                    //总计分片数
        "failed" : 0,
        "successful" : 9                                //成功分片数
      }
    }

四、恢复数据

官网：https://www.elastic.co/guide/cn/elasticsearch/guide/2.x/_restoring_from_a_snapshot.html

授权

chown  -R es:es /data/es-bak/

创建存储库

POST  _snapshot/my_backup

{
  "type": "fs",
  "settings": {
    "location": "/data/es-bak",   # 跟配置文件中的repo目录保持一致
    "compress": true,
    "chunk_size": "1g",
    "max_snapshot_bytes_per_sec": "50m",
    "max_restore_bytes_per_sec": "50m"
  }
}

恢复命令

POST   _snapshot/my_backup/snapshot_1/_restore

对比数据

GET  plan_special_data/_count

{ - 
  "count": 969916,
  "_shards": { - 
    "total": 2,
    "successful": 2,
    "skipped": 0,
    "failed": 0
  }
}