生产环境minio 部署

minio分布式集群的详细搭建

1，准备工作

4台服务器 ，每台服务器 ，挂载一个硬盘

以下 不重启识别硬盘

echo "- - -" > /sys/class/scsi_host/host0/scan
 
 echo "- - -" > /sys/class/scsi_host/host1/scan
 
 echo "- - -" > /sys/class/scsi_host/host2/scan
 

1、 基础准备

192.168.40.180挂载磁盘路径：/app/minio/miniodata

192.168.40.181挂载磁盘路径：/app/minio/miniodata

192.168.40.182挂载磁盘路径：/app/minio/miniodata

192.168.40.61挂载磁盘路径：/app/minio/miniodata

生产环境强烈建议至少四台机器，这也是官方的建议要求，这样的话就可以做到挂掉一台机器集群依然可以读写，挂掉两台机器集群依然可读，本文仅以四台机器为例子说明如何搭建集群

2 下载软件包

官网
 https://min.io/download#/linux
 
 
 wget https://dl.min.io/server/minio/release/linux-amd64/minio
 
 

3 创建minio用户（密码根据实际情况设置）并增加sudo权限

groupadd -g 10002 minio;useradd -d /home/minio -g 10002 -u 10002 -m minio;echo 'minio:Huawei12#$' | chpasswd;chage -M 99999 minio;usermod -G wheel minio

1、 创建minio的data路径

mkdir -p /app/minio/miniodata
 chown -R  minio:minio  /app/minio
 

1、 上传minio.tar.gz到/home/minio下，

解压并编辑编写集群启动脚本run.sh,

其中MINIO_ACCESS_KEY、MINIO_SECRET_KEY根据实际情况设置

Minio用户编写集群启动脚本（所有节点配置文件相同）

其中，“MINIO_ACCESS_KEY”为用户名，“MINIO_SECRET_KEY”为密码，密码不能设置过于简单，不然minio会启动失败

[root@master minio]# cat run.sh 
 #!/bin/bash
 export MINIO_ACCESS_KEY=yR3eJ5mF5bK5vE2oA7aE
 export MINIO_SECRET_KEY=yH4cH8zI7gL6wG4zE0fB6xS1xN7dN2
 
 /home/minio/minio server \
 http://192.168.40.180/app/minio/miniodata  http://192.168.40.181/app/minio/miniodata  \
 http://192.168.40.182/app/minio/miniodata  http://192.168.40.61/app/minio/miniodata
 
 [root@master minio]# pwd
 /home/minio
 
 
 =========================如果修改端口 使用下面方法，各节点修改======================
 [root@master ~]# cat /home/minio/run.sh
 #!/bin/bash
 export MINIO_ACCESS_KEY=yR3eJ5mF5bK5vE2oA7aE
 export MINIO_SECRET_KEY=yH4cH8zI7gL6wG4zE0fB6xS1xN7dN2
 
 /home/minio/minio server --console-address "192.168.40.180:9001"  --address "192.168.40.180:9000" \
 http://192.168.40.180/app/minio/miniodata  http://192.168.40.181/app/minio/miniodata  \
 http://192.168.40.182/app/minio/miniodata  http://192.168.40.61/app/minio/miniodata
 
 ==========================

1、 编写服务脚本（root用户），如需要修改的话请编辑，否则保持默认值即可

其中，“WorkingDirectory”为启动脚本目录，“ExecStart”为指定集群启动脚本

[root@master minio]# cat minio.service
 [Unit]
 Description=Minio service
 Documentation=https://docs.minio.io/
  
 [Service]
 User=minio
 WorkingDirectory=/home/minio/
 ExecStart=/home/minio/run.sh
  
 Restart=on-failure
 RestartSec=5
  
 [Install]
 WantedBy=multi-user.target

将修改好的内容复制到其他节点

[minio@master ~]$ pwd
 /home/minio
 
 scp -rq *  [email protected]:/home/minio/
 scp -rq *  [email protected]:/home/minio/
 scp -rq *  [email protected]:/home/minio/
 

6、  启动测试
 1）将下载好的minio文件赋予权限
 sudo  chmod +x /home/minio/minio
 2）赋予启动脚本文件权限
 sudo chmod +x /home/minio/run.sh
 
 
 3）依次启动每个服务器的minio
 sudo cp /home/minio/minio.service /usr/lib/systemd/system/
 sudo systemctl daemon-reload
 sudo systemctl start minio
 sudo systemctl enable minio
 
 
 
 4）浏览器输入集群任意节点地址+9000端口，即可访问minio，用户名密码为前面设置的“MINIO_ACCESS_KEY”和“MINIO_SECRET_KEY”，可创建“bucket”并上传文件测试

基于nginx实现loadbalancer

官网下载最新版本
 https://nginx.org/en/download.html
 
 将nginx-1.23.1.tar.gz上传到与NGINX规划所在主机上
 新建目录：mkdir -p /app/nginx,然后上传
 解压：cd /app/nginx ; tar -zxvf nginx-1.23.1.tar.gz
 编译安装前，安装依赖包：
 yum -y install gcc gcc-c++ autoconf automake make openssl openssl-devel pcre-devel
 编译nginx：
 cd /app/nginx/nginx-1.23.1
 ./configure --prefix=/app/nginx  --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module
   (如果报找不到文件的错，则看下/app/nginx目录下的文件)
 安装：
 make && make install
 先测试下Nginx，然后如果是少文件，则创建。然后启动Nginx
 
 
 ln -s /app/nginx/sbin/nginx /usr/local/sbin/
 
 测试命令：nginx -t -c ./conf/nginx.conf
 启动命令：nginx  -c ./conf/nginx.conf
 
 
 

nginx 配置如下
 
 [root@master nginx]# cat conf/nginx.conf
 user nginx;
 worker_processes auto;
 pid /run/nginx.pid;
 events {
     worker_connections 1024;
 }
 
 
 include /usr/share/nginx/modules/*.conf;
 
 
 
 http {
     include       mime.types;
     default_type  application/octet-stream;
     sendfile        on;
     keepalive_timeout  65;
  
 upstream minio {
     server 192.168.40.180:9000;
     server 192.168.40.181:9000;
     server 192.168.40.182:9000;
     server 192.168.40.61:9000;
 }
 
 upstream console {
     ip_hash;
     server 192.168.40.180:9001;
     server 192.168.40.181:9001;
     server 192.168.40.182:9001;
     server 192.168.40.61:9001;
 }
 
 
 server {
         listen       80;
         listen  [::]:80;
         server_name  minio.sfere.local;
  
         # To allow special characters in headers
         ignore_invalid_headers off;
         # Allow any size file to be uploaded.
         # Set to a value such as 1000m; to restrict file size to a specific value
         client_max_body_size 0;
         # To disable buffering
         proxy_buffering off;
  
         location / {
             proxy_set_header Host $http_host;
             proxy_set_header X-Real-IP $remote_addr;
             proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
             proxy_set_header X-Forwarded-Proto $scheme;
  
             proxy_connect_timeout 300;
             # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
             proxy_http_version 1.1;
             proxy_set_header Connection "";
             chunked_transfer_encoding off;
  
             proxy_pass http://minio;
         }
 }
 server {
         listen       19001;
         listen  [::]:19001;
         server_name  minio.sfere.local;
  
         # To allow special characters in headers
         ignore_invalid_headers off;
         # Allow any size file to be uploaded.
         # Set to a value such as 1000m; to restrict file size to a specific value
         client_max_body_size 0;
         # To disable buffering
         proxy_buffering off;
  
         location / {
             proxy_set_header Host $http_host;
             proxy_set_header X-Real-IP $remote_addr;
             proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
             proxy_set_header X-Forwarded-Proto $scheme;
             proxy_set_header X-NginX-Proxy true;
  
             # This is necessary to pass the correct IP to be hashed
             real_ip_header X-Real-IP;
  
             proxy_connect_timeout 300;
  
             # To support websocket
             proxy_http_version 1.1;
             proxy_set_header Upgrade $http_upgrade;
             proxy_set_header Connection "upgrade";
  
             chunked_transfer_encoding off;
  
             proxy_pass http://console;
         }
 }
 
 }
 

启动测试ok

​​http://192.168.40.180:19001/​​

minio挂载在本地

minio挂载在本地
 1.安装s3fs: sudo apt install s3fs
 
  
 
 2.获取Access Key和Secret Key，修改${HOME}/.passwd-s3fs，写入：
 
 Access Key:Secret Key
  
 
 chmod 600 ${HOME}/.passwd-s3fs
  
 
  sudo vim /etc/fuse.conf，取消user_allow_other前的注释
 
  
 
 3.
 
 s3fs -o passwd_file=${HOME}/.passwd-s3fs -o url=http://minio服务器地址:9000 -o allow_other -o nonempty -o no_check_certificate -o use_path_request_style -o umask=022 桶名称 需要mount的文件夹地址
 
 黄世宇/Shiyu Huang's Personal Page：https://huangshiyu13.github.io/