首页 > 其他分享 >milvus-backup(k8s)配置文件说明

milvus-backup(k8s)配置文件说明

时间:2024-12-09 11:10:39浏览次数:11  
标签:MinIO 配置文件 S3 data k8s backup milvus minio

 

[root@yunwei-k8s-dev conf]# more backup.yaml 
# Configures the system log output.
log:
  level: info # Only supports debug, info, warn, error, panic, or fatal. Default 'info'.
  console: true # whether print log to console
  file:
    rootPath: "logs/backup.log"

http:
  simpleResponse: true

# milvus proxy address, compatible to milvus.yaml
milvus:
  address: 10.18.223.77
  port: 19530
  authorizationEnabled: true
  # tls mode values [0, 1, 2]
  # 0 is close, 1 is one-way authentication, 2 is two-way authentication.
  tlsMode: 0
  user: "root"
  password: "Milvus"

# Related configuration of minio, which is responsible for data persistence for Milvus.
minio:
  # Milvus storage configs, make them the same with milvus config
  storageType: "minio" # support storage type: local, minio, s3, aws, gcp, ali(aliyun), azure, tc(tencent)
  address: 192.168.1.246 # Address of MinIO/S3
  port: 9000   # Port of MinIO/S3
  accessKeyID: root  # accessKeyID of MinIO/S3
  secretAccessKey: 123 # MinIO/S3 encryption string
  useSSL: false # Access to MinIO/S3 with SSL
  useIAM: false
  iamEndpoint: ""
  bucketName: "milvus-dev" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
  rootPath: "milvus/milvus-dev" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance


  # Backup storage configs, the storage you want to put the backup data
  backupStorageType: "minio" # support storage type: local, minio, s3, aws, gcp, ali(aliyun), azure, tc(tencent)
  backupAddress: 192.168.1.246 # Address of MinIO/S3
  backupPort: 9000   # Port of MinIO/S3
  backupAccessKeyID: root  # accessKeyID of MinIO/S3
  backupSecretAccessKey: 123 # MinIO/S3 encryption string
  backupBucketName: "a-bucket" # Bucket name to store backup data. Backup data will store to backupBucketName/backup
RootPath
  backupRootPath: "backup" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPat
h

  # If you need to back up or restore data between two different storage systems, direct client-side copying is not 
supported. 
  # Set this option to true to enable data transfer through Milvus Backup.
  # Note: This option will be automatically set to true if `minio.storageType` and `minio.backupStorageType` differ.
  # However, if they are the same but belong to different services, you must manually set this option to `true`.
  crossStorage: "false"
  
backup:
  maxSegmentGroupSize: 2G
  parallelism: 
    # collection level parallelism to backup
    backupCollection: 4
    # thread pool to copy data. reduce it if blocks your storage's network bandwidth
    copydata: 128
    # Collection level parallelism to restore
    restoreCollection: 2
  
  # keep temporary files during restore, only use to debug 
  keepTempFiles: false
  
  # Pause GC during backup through Milvus Http API. 
  gcPause:
    enable: true
    seconds: 7200
    address: http://10.18.223.77:9091

 

  bucketName: "milvus-dev" #milvus使用的minio bucketname名称
  rootPath: "milvus/milvus-dev" # milvus使用的minio rootPath路径


  backupBucketName: "a-bucket" # 备份文件存放的bucketname名称
  backupRootPath: "backup" # 备份文件的目录


 

标签:MinIO,配置文件,S3,data,k8s,backup,milvus,minio
From: https://www.cnblogs.com/hxlasky/p/18594452

相关文章

  • Spring Boot中使用YAML配置文件
    1.YAML文件结构和语法缩进与层次YAML使用缩进来表示层级关系。每个层级的元素必须比它的父级多一个空格或Tab(推荐使用空格)。例如:server:port:8080address:localhost列表与映射列表用-开头,映射则用key:value形式。可以混合使用。myapp:features:-......
  • K8s面试系列:K8s常用 API 资源总结速记
    写在前面博文内容为K8s中常用的API资源简单总结适合对K8s有基本认知,温习,面试准备的小伙伴内容涉及:API资源组成,SSA和CSA资源操作方式,以及下面资源类别简单描述工作负载:Deployments,StatefulSets,HorizontalPodAutoscaler(HPA)…服务:Service,Ingress,E......
  • 通过配置文件对日志进行配置
    '''配置文件设置与读取因为logging的所有内容,配置项与配置的值都是相对固定的。所以可以通过配置文件的形式将日志内容进行配置与定义。然后通过logging.config.fileConfig()方法实现对内容的获取。日志配置形态是心啊的内容读取。用于创建记录器进行日志内容的生......
  • K8S钩子、探针以及控制器完整版
    一、生命周期钩子Kubernetes中的生命周期钩子(LifecycleHooks)是在容器生命周期的特定阶段执行操作的机制。通过钩子,可以在容器启动后(PostStart)或停止前(PreStop)执行一些初始化或清理工作。钩子的作用PostStart(启动后)在容器启动后立即触发执行。用于完成启动后的初始化操......
  • k8s阶段06 k8s认证体系和插件, 添加用户账号认证, kubeconfig使用, serviceaccount,
    1Kubernetes的访问控制体系认证:APIServer:https://control_host:6443/#集群外部访问控制平面节点ip+6443https://kubernetes.default.svc.cluster.local#集群内部访问集群网关:APIServer#执行安全策略:认证,鉴权,准入控制(其他地方很少用到)......
  • 什么是 Kubernetes(K8s)?
    什么是Kubernetes(K8s)?Kubernetes(简称K8s)是一个用来管理容器的开源工具,它可以自动化部署、扩展和管理容器化应用。简单来说,K8s就是一个“容器管家”,负责确保你的应用程序能够在容器中高效地运行,无论是部署1个容器,还是部署1000个容器,它都能轻松管理。通俗比喻:K8s是......
  • k8s~service和deployment中的spec.selector
    service和deployment中的spec.selector在Kubernetes中,Service和Deployment的spec.selector在使用上是有一些不同之处的,下面是对这两者的详细解释:1.Deployment中的Selector在Deployment中,spec.selector是必需的,并且通常使用matchLabels来定义选择器。例如:apiVer......
  • K8s搭建指南
    K8s指南1)k8s环境准备基本设置(前置条件)(针对master和node)#1.各个机器设置自己的域名hostnamectlset-hostnamexxxx--------------------------------------------------------------------------------------------------------------------------------#2.将SELinux设......
  • k8s-rdma-shared-dev-plugin
    文章目录前言一、创建k8s集群二、启用primarynetwork三、启用secondarynetworkk8s-rdma-shared-dev-pluginMultusCNISecondaryCNIMulti-NetworkCRD四、启用pod五、在pod中启动RoCE流量总结前言写给自己的入门篇。后续会在原理方面持续更新一、创建k8s集群k8s集......
  • crane-scheduler基于真实负载进行k8s调度
    原生 kubernetes 调度器只能基于资源的resource request 进行调度,然而Pod的真实资源使用率,往往与其所申请资源的request/limit 差异很大,导致集群负载不均的问题。crane-scheduler基于集群的真实负载数据构造了一个简单却有效的模型,作用于调度过程中的Filter与Score......