首页 > 其他分享 >etcd学习

etcd学习

时间：2023-03-19 10:24:40浏览次数：48  
标签：level 31 ts 学习 msg etcd go

介绍

官网

https://etcd.io

文档

https://etcd.io/docs/v3.5/quickstart

用途

etcd是Go语言编写的分布式、高可用的一致性键值存储系统，用于提供可靠的分布式键值存储、配置共享和服务发现等功能

典型使用场景

推荐数据量很小，但是更新访问频繁的情况。
常见场景：服务发现、分布式锁、分布式队列、分布式通知和协调、主备选举等。

优点

对比同类工具zookeeper
1、etcd更加稳定可靠
2、服务发现上，etcd使用的是节点租约（Lease），并且支持Group（多key）；可zookeeper使用的是临时节点，临时节点存在不少问题。
3、etcd支持稳定的watch，而不是zookeeper一样简单的单次触发watch。etcd可以存储数十万的历史变更。
4、etcd支持MVCC（多版本并发控制），因为有协同系统需要无锁操作。
5、etcd支持更大的数据规模，支持存储百万到千万级别的key。
6、etcd性能更好，在一个由3台8核节点组成的云服务器上，etcd v3版本可以做到每条数万次的写操作和数十万次的读操作。

架构

client层

etcdctl/clientv3

# help
$ etcdctl -h
# 
$ etcdctl put /name fox3
OK
$ etcdctl get /name
fox3

api

raft http
gRpcAPI
HTTP API(V2/V3)

raft

Leader Election
Log Replication
Members hip
Read Index
Learner
...

逻辑层

KVServer
Quota
Mainte nance
Apply
Auth
Compactor
treeIndex
Lease

存储层

WAL
Snapshot
boltdb

部署

单实例部署

安装预构建的二进制文件
安装 etcd 的最简单方法是从预构建的二进制文件中：
从版本下载适用于您平台的压缩存档文件， 选择版本 v3.5.0 或更高版本。
解压缩存档文件。这将生成一个包含二进制文件的目录。
将可执行二进制文件添加到路径。例如，重命名和/或移动 将二进制文件添加到路径中的目录（如 ），或添加 上一步创建的路径目录。/usr/local/bin
从外壳中，测试路径中的：etcd

# 官网脚本
ETCD_VER=v3.5.7

# choose either URL
GOOGLE_URL=https://storage.googleapis.com/etcd
GITHUB_URL=https://github.com/etcd-io/etcd/releases/download
DOWNLOAD_URL=${GOOGLE_URL}

rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
rm -rf /tmp/etcd-download-test && mkdir -p /tmp/etcd-download-test

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
tar xzvf /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /tmp/etcd-download-test --strip-components=1
rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz

/tmp/etcd-download-test/etcd --version
/tmp/etcd-download-test/etcdctl version
/tmp/etcd-download-test/etcdutl version

执行官网脚本后可以看到类似如下输出：

etcd Version: 3.5.7
Git SHA: 215b53cf3
Go Version: go1.17.13
Go OS/Arch: linux/amd64
etcdctl version: 3.5.7
API version: 3.5
etcdutl version: 3.5.7
API version: 3.5

服务启动和测试

# start a local etcd server
/tmp/etcd-download-test/etcd
{"level":"info","ts":"2023-03-18T10:31:02.704+0800","caller":"etcdmain/etcd.go:73","msg":"Running: ","args":["/tmp/etcd-download-test/etcd"]}
{"level":"warn","ts":"2023-03-18T10:31:02.704+0800","caller":"etcdmain/etcd.go:105","msg":"'data-dir' was empty; using default","data-dir":"default.etcd"}
{"level":"info","ts":"2023-03-18T10:31:02.704+0800","caller":"embed/etcd.go:124","msg":"configuring peer listeners","listen-peer-urls":["http://localhost:2380"]}
{"level":"info","ts":"2023-03-18T10:31:02.722+0800","caller":"embed/etcd.go:132","msg":"configuring client listeners","listen-client-urls":["http://localhost:2379"]}
{"level":"info","ts":"2023-03-18T10:31:02.723+0800","caller":"embed/etcd.go:306","msg":"starting an etcd server","etcd-version":"3.5.7","git-sha":"215b53cf3","go-version":"go1.17.13","go-os":"linux","go-arch":"amd64","max-cpu-set":8,"max-cpu-available":8,"member-initialized":false,"name":"default","data-dir":"default.etcd","wal-dir":"","wal-dir-dedicated":"","member-dir":"default.etcd/member","force-new-cluster":false,"heartbeat-interval":"100ms","election-timeout":"1s","initial-election-tick-advance":true,"snapshot-count":100000,"max-wals":5,"max-snapshots":5,"snapshot-catchup-entries":5000,"initial-advertise-peer-urls":["http://localhost:2380"],"listen-peer-urls":["http://localhost:2380"],"advertise-client-urls":["http://localhost:2379"],"listen-client-urls":["http://localhost:2379"],"listen-metrics-urls":[],"cors":["*"],"host-whitelist":["*"],"initial-cluster":"default=http://localhost:2380","initial-cluster-state":"new","initial-cluster-token":"etcd-cluster","quota-backend-bytes":2147483648,"max-request-bytes":1572864,"max-concurrent-streams":4294967295,"pre-vote":true,"initial-corrupt-check":false,"corrupt-check-time-interval":"0s","compact-check-time-enabled":false,"compact-check-time-interval":"1m0s","auto-compaction-mode":"periodic","auto-compaction-retention":"0s","auto-compaction-interval":"0s","discovery-url":"","discovery-proxy":"","downgrade-check-interval":"5s"}
{"level":"info","ts":"2023-03-18T10:31:02.724+0800","caller":"etcdserver/backend.go:81","msg":"opened backend db","path":"default.etcd/member/snap/db","took":"899.143µs"}
{"level":"info","ts":"2023-03-18T10:31:02.726+0800","caller":"etcdserver/raft.go:494","msg":"starting local member","local-member-id":"8e9e05c52164694d","cluster-id":"cdf818194e3a8c32"}
{"level":"info","ts":"2023-03-18T10:31:02.726+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d switched to configuration voters=()"}
{"level":"info","ts":"2023-03-18T10:31:02.726+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became follower at term 0"}
{"level":"info","ts":"2023-03-18T10:31:02.726+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"newRaft 8e9e05c52164694d [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]"}
{"level":"info","ts":"2023-03-18T10:31:02.726+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became follower at term 1"}
{"level":"info","ts":"2023-03-18T10:31:02.726+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d switched to configuration voters=(10276657743932975437)"}
{"level":"warn","ts":"2023-03-18T10:31:02.727+0800","caller":"auth/store.go:1234","msg":"simple token is not cryptographically signed"}
{"level":"info","ts":"2023-03-18T10:31:02.728+0800","caller":"mvcc/kvstore.go:393","msg":"kvstore restored","current-rev":1}
{"level":"info","ts":"2023-03-18T10:31:02.729+0800","caller":"etcdserver/quota.go:94","msg":"enabled backend quota with default value","quota-name":"v3-applier","quota-size-bytes":2147483648,"quota-size":"2.1 GB"}
{"level":"info","ts":"2023-03-18T10:31:02.729+0800","caller":"etcdserver/server.go:854","msg":"starting etcd server","local-member-id":"8e9e05c52164694d","local-server-version":"3.5.7","cluster-version":"to_be_decided"}
{"level":"info","ts":"2023-03-18T10:31:02.731+0800","caller":"embed/etcd.go:275","msg":"now serving peer/client/metrics","local-member-id":"8e9e05c52164694d","initial-advertise-peer-urls":["http://localhost:2380"],"listen-peer-urls":["http://localhost:2380"],"advertise-client-urls":["http://localhost:2379"],"listen-client-urls":["http://localhost:2379"],"listen-metrics-urls":[]}
{"level":"info","ts":"2023-03-18T10:31:02.732+0800","caller":"etcdserver/server.go:738","msg":"started as single-node; fast-forwarding election ticks","local-member-id":"8e9e05c52164694d","forward-ticks":9,"forward-duration":"900ms","election-ticks":10,"election-timeout":"1s"}
{"level":"info","ts":"2023-03-18T10:31:02.732+0800","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"default.etcd/member/snap","suffix":"snap.db","max":5,"interval":"30s"}
{"level":"info","ts":"2023-03-18T10:31:02.732+0800","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"default.etcd/member/snap","suffix":"snap","max":5,"interval":"30s"}
{"level":"info","ts":"2023-03-18T10:31:02.732+0800","caller":"fileutil/purge.go:44","msg":"started to purge file","dir":"default.etcd/member/wal","suffix":"wal","max":5,"interval":"30s"}
{"level":"info","ts":"2023-03-18T10:31:02.732+0800","caller":"embed/etcd.go:586","msg":"serving peer traffic","address":"127.0.0.1:2380"}
{"level":"info","ts":"2023-03-18T10:31:02.732+0800","caller":"embed/etcd.go:558","msg":"cmux::serve","address":"127.0.0.1:2380"}
{"level":"info","ts":"2023-03-18T10:31:02.733+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d switched to configuration voters=(10276657743932975437)"}
{"level":"info","ts":"2023-03-18T10:31:02.733+0800","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"cdf818194e3a8c32","local-member-id":"8e9e05c52164694d","added-peer-id":"8e9e05c52164694d","added-peer-peer-urls":["http://localhost:2380"]}
{"level":"info","ts":"2023-03-18T10:31:02.837+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d is starting a new election at term 1"}
{"level":"info","ts":"2023-03-18T10:31:02.837+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became pre-candidate at term 1"}
{"level":"info","ts":"2023-03-18T10:31:02.838+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d received MsgPreVoteResp from 8e9e05c52164694d at term 1"}
{"level":"info","ts":"2023-03-18T10:31:02.838+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became candidate at term 2"}
{"level":"info","ts":"2023-03-18T10:31:02.838+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 2"}
{"level":"info","ts":"2023-03-18T10:31:02.838+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became leader at term 2"}
{"level":"info","ts":"2023-03-18T10:31:02.838+0800","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 2"}
{"level":"info","ts":"2023-03-18T10:31:02.841+0800","caller":"etcdserver/server.go:2062","msg":"published local member to cluster through raft","local-member-id":"8e9e05c52164694d","local-member-attributes":"{Name:default ClientURLs:[http://localhost:2379]}","request-path":"/0/members/8e9e05c52164694d/attributes","cluster-id":"cdf818194e3a8c32","publish-timeout":"7s"}
{"level":"info","ts":"2023-03-18T10:31:02.841+0800","caller":"etcdserver/server.go:2571","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"}
{"level":"info","ts":"2023-03-18T10:31:02.842+0800","caller":"embed/serve.go:100","msg":"ready to serve client requests"}
{"level":"info","ts":"2023-03-18T10:31:02.844+0800","caller":"embed/serve.go:146","msg":"serving client traffic insecurely; this is strongly discouraged!","address":"127.0.0.1:2379"}
{"level":"info","ts":"2023-03-18T10:31:02.844+0800","caller":"etcdmain/main.go:44","msg":"notifying init daemon"}
{"level":"info","ts":"2023-03-18T10:31:02.844+0800","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"}
{"level":"info","ts":"2023-03-18T10:31:02.844+0800","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"cdf818194e3a8c32","local-member-id":"8e9e05c52164694d","cluster-version":"3.5"}
{"level":"info","ts":"2023-03-18T10:31:02.844+0800","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"}
{"level":"info","ts":"2023-03-18T10:31:02.844+0800","caller":"etcdserver/server.go:2595","msg":"cluster version is updated","cluster-version":"3.5"}

# write,read to etcd
/tmp/etcd-download-test/etcdctl --endpoints=localhost:2379 put foo bar
OK
/tmp/etcd-download-test/etcdctl --endpoints=localhost:2379 get foo
foo
bar

单机多实例部署

使用goreman创建3节点etcd集群，也可以不使用goreman，直接通过命令行方式启动三个不同的实例

# 安装goreman环境
$ go install github.com/mattn/goreman@latest
$ goreman help

$ cat local-cluster-profile
etcd1: etcd --name infra1 --listen-client-urls http://127.0.0.1:12379 --advertise-client-urls http://127.0.0.1:12379 --listen-peer-urls http://127.0.0.1:12380 --initial-advertise-peer-urls http://127.0.0.1:12380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new
etcd2: etcd --name infra2 --listen-client-urls http://127.0.0.1:22379 --advertise-client-urls http://127.0.0.1:22379 --listen-peer-urls http://127.0.0.1:22380 --initial-advertise-peer-urls http://127.0.0.1:22380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new
etcd3: etcd --name infra3 --listen-client-urls http://127.0.0.1:32379 --advertise-client-urls http://127.0.0.1:32379 --listen-peer-urls http://127.0.0.1:32380 --initial-advertise-peer-urls http://127.0.0.1:32380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new

$ goreman -f ./local-cluster-profile start

$ netstat -an|grep 2379
tcp        0      0 127.0.0.1:12379         0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:32379         0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:22379         0.0.0.0:*               LISTEN 

$ ./etcdctl --endpoints=localhost:12379 member list
8211f1d0f64f3269, started, infra1, http://127.0.0.1:12380, http://127.0.0.1:12379, false
91bc3c398fb3c146, started, infra2, http://127.0.0.1:22380, http://127.0.0.1:22379, false
fd422379fda50e48, started, infra3, http://127.0.0.1:32380, http://127.0.0.1:32379, false

生产部署

静态部署

明确知道集群中有多少节点，在启动的时候通过 --initial-cluster参数直接指定etcd的节点地址信息
和前面启动方式一样，只是分别部署在不同主机上而已。

etcd动态发现

在搭建时不知道个节点情况，可以通过已搭建etcd来辅助搭建新的etcd集群，通过已有etcd集群作为数据交互节点，然后扩展新的集群式，实现通过已有集群进行服务发现的机制。
Discovery Service Protocol，使用新的发现令牌来引导一个唯一的etcd集群。

$ curl https://discovery.etcd.io/new?size=3
https://discovery.etcd.io/bda8fb66933bfd85296f1e91f7929f99

后续在启动命令中增加--discovery https://discovery.etcd.io/bda8fb66933bfd85296f1e91f7929f99参数用来引导etcd
我觉得这个方案吧，恐怕并不大实用，部署生产etcd的很多都是内网服务器，不会允许这样去连公网。

DNS动态发现

通过DNS查询的方式获取其他节点地址信息

标签：level,31,ts,学习,msg,etcd,go
From： https://www.cnblogs.com/youxiong/p/17227461.html

相关文章

  • 「学习笔记」平衡树基础：Splay 和 Treap
    「学习笔记」平衡树基础：Splay和Treap点击查看目录目录「学习笔记」平衡树基础：Splay和Treap知识点平衡树概述Splay旋转操作Splay操作插入\(x\)查询排名为\(k\)......
  • 16、深入学习Pods
    1、概述1.1、简介Kubernetes是通过组合容器将应用放置到一个个的逻辑单元中，达到快速管理业务应用的效果，这个逻辑单元就是Pod。Pod是Kubernetes中最重要也最基本的概念......
  • 动手学深度学习－第一章引言
    在ＤＡＴＡｗｈａｌｅ的组织下，开始新的一轮深度学习，感谢组织，感谢沐神．机器学习（machinelearning，ML）是一类强大的可以从经验中学习的技术。通常采用观测数据或与环境交互的形式，机器学习算法会......
  • java学习日记20230317-多态
    多态方法和对象爱过你具有多种形态，是面向对象的第三大特征，多态是建立在封装和继承的基础上；方法的重载体现多态方法的重写体现多态对象的多态一个对象的编译类型和......
  • java学习第一天
    java学习第一天第一个代码helloworld用cmd指令编译和运行Java文件Javac加Java文件名，编译javac编译文件会生成class文件Java运行class文件（无后缀）java程序运行机......
  • 算法学习笔记(19): 树上启发式合并（DSU on tree）
    树上启发式合并DSUontree，我也不知道DSU是啥意思这是一种看似特别玄学的优化可以把树上部分问题由\(O(n^2)\)优化到\(O(n\logn)\)。例如CodeForces600E。又......
  • STM32 HAL库学习 （3） 中断！
    一、EXTI的HAL库驱动前面讲解HAL_GPIO_Init函数的时候有提到过：HAL库的EXTI外部中断的设置功能整合到HAL_GPIO_Init函数里面，而不是单独独立一个文件。所以我们......
  • Django学习day1
    django安装 -直接在终端中安装开始一个项目 -终端命令 -django-adminstartproject项目名D:\TOOL\PYCHARMCOMMUNITYEDITION2019.3.4\PYCHARMPROJECTS\DJANGOLEARN\D......
  • Kubernetes学习之基础知识
    Kubernetes是一个开源的容器编排平台，用于自动化部署、扩展和管理容器化应用程序。它提供了许多功能，例如服务发现、负载均衡、自动伸缩、滚动更新等，使得运维人员可以轻松地......
  • markdown的学习，博客创建第一天
    Markdown学习标题：//#＋标题名字一级标题//##+标题名字二级标题以此类推字体粗体 两边加**你好斜体 两边*你好粗体加斜体两边***你好删除线两......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99