docker-compose安装minio集群

一、docker-compose安装minio单机版

直接使用docker安装单机版，可用于测试
创建.env环境文件

MINIO_PASSWORD=minio@123

创建docker-compose.yml环境文件

version: "3"
services:
  minio:
    image: minio/minio
    container_name: minio
    ports:
      - 9000:9000
      - 9001:9001
    environment:
      - MINIO_ROOT_USER=minio
      - MINIO_ROOT_PASSWORD=${MINIO_PASSWORD}
    volumes:
      - ./data:/data
    #指定容器中的目录 /data
    command: server --console-address ':9001' /data
    privileged: true
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3
    restart: always
    #network_mode: host

直接执行docker-compose up -d命令即可启动运行

二、docker-compose安装minio集群

准备三台机器用于部署minio分布式集群

192.168.131.101
192.168.131.102
192.168.131.103

1、配置hosts

在每台机器执行如下命令vim /etc/hosts，写入如下文本

echo 'minio1 192.168.131.101' >> /etc/hosts
echo 'minio2 192.168.131.102' >> /etc/hosts
echo 'minio3 192.168.131.103' >> /etc/hosts

执行命令cat /etc/hosts查看配置结果

minio1 192.168.131.101
minio2 192.168.131.102
minio3 192.168.131.103

2、创建目录

在每台机器上执行创建目录命令

mkdir -p /data/minio

3、创建yml文件

装备.env环境变量文件

cd /data/minio && echo 'MINIO_PASSWORD=minio@123' > .env

每台机器创建docker-compose.yml环境文件

version: "3"
services:
  minio:
    image: reg.harbor999.com/minio/minio
    container_name: minio-node1
    hostname: minio1
    expose:
      - "19000"
      - "19001"
    environment:
      - MINIO_ROOT_USER=minio
      - MINIO_ROOT_PASSWORD=${MINIO_PASSWORD}
    volumes:
      - ./data-1:/data1
      - ./data-2:/data2
    command: server --console-address ':19001' --address ':19000' http://minio{1...3}/data{1...2}
    privileged: true
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:19000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3
    extra_hosts:
      - minio1:192.168.131.101
      - minio2:192.168.131.102
      - minio3:192.168.131.103
    restart: always
    network_mode: host

4、nginx负载均衡（参考）此部分内容可在任意机器添加，选一台即可

在192.168.131.101上的docker-compose.yml文件中添加如下nginx服务的内容

  nginx:
    image: nginx:1.19.2-alpine
    hostname: nginx
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "9000:9090"
      - "9001:9091"
    extra_hosts:
      - minio1:192.168.131.195
      - minio2:192.168.131.194
      - minio3:192.168.131.188
    restart: always

挂载的nginx.conf配置文件

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  4096;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    keepalive_timeout  65;

    # include /etc/nginx/conf.d/*.conf;

    upstream adobe {
        server minio1:19000;
        server minio2:19000;
        server minio3:19000;
    }

    upstream console {
        ip_hash;
        server minio1:19001;
        server minio2:19001;
        server minio3:19001;
    }

    server {
        listen       9090;
        listen  [::]:9090;
        server_name  localhost;

        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;
        proxy_request_buffering off;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            chunked_transfer_encoding off;

            proxy_pass http://adobe;
        }
    }

    server {
        listen       9091;
        listen  [::]:9091;
        server_name  localhost;

        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;
        proxy_request_buffering off;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-NginX-Proxy true;

            # This is necessary to pass the correct IP to be hashed
            real_ip_header X-Real-IP;
            proxy_connect_timeout 300;

            # To support websocket
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";

            chunked_transfer_encoding off;
            proxy_pass http://console;
        }
    }
}

如自行配置nginx配置，记得开启websocket即可

			# To support websocket
			proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";

至此，准备工作完成，分别在每台机器执行docker-compose启动命令即可完成集群搭建

docker-compose up -d

扩展

1、ansible批量安装docker（ansible牛逼，不要太爽了）
2、在centos7系统中如何给docker配置代理
3、portainer管理远程docker和docker-swarm集群