www.colben.cn/content/post/minio.md

---
title: "Minio 笔记"
date: 2021-10-17T00:48:00+08:00
lastmod: 2024-04-17T20:08:00+08:00
keywords: []
tags: ["minio"]
categories: ["storage"]
---

## 部署单节点分布式存储
### 环境
- 服务器 IP: 192.168.1.10
- 自定义域名: X.X.X

### 安装 minio
- 下载
    ```bash
    curl -L -o /usr/local/bin/minio https://dl.min.io/server/minio/release/linux-amd64/minio
    curl -L -o /usr/local/bin/mc https://dl.min.io/client/mc/release/linux-amd64/mc
    chmod 0755 /usr/local/bin/{minio,mc}
    ```

- 修改 hosts，增加自定义域名解析
    ```
    192.168.1.10 X.X.X
    ```

- 创建四个目录，用于存储 minio 数据
    ```bash
    mkdir -p /minio/{1,2,3,4}
    ```

- 创建 /etc/systemd/system/minio.service，内容如下
    ```ini
    [Unit]
    Description=MinIO
    After=network.service
    Wants=network.service

    [Service]
    Environment=MINIO_ROOT_USER=XXX
    Environment=MINIO_ROOT_PASSWORD=XXXXXXXX
    ExecStart=/usr/local/bin/minio server \
        --address ":9000" \
        --console-address ":9010" \
        http://X.X.X:9000/minio/{1...4}
    #Restart=on-failure
    RestartSec=10s

    [Install]
    WantedBy=multi-user.target
    ```

### 启动 minio
- 配置开机自动启动
    ```bash
    systemctl daemon-reload
    systemctl enable minio
    ```

- 启动 minio
    ```bash
    systemctl start monio
    ```

## 部署多节点分布式存储
### 环境
自定义域名 | 网卡 IP                            | 数据盘
----       | ----                               | ----
X1.X.X     | eth0: 192.168.1.11, eth1:10.0.0.11 | /dev/sdb, /dev/sdc, /dev/sdd, /dev/sde
X2.X.X     | eth0: 192.168.1.12, eth1:10.0.0.12 | /dev/sdb, /dev/sdc, /dev/sdd, /dev/sde
X3.X.X     | eth0: 192.168.1.13, eth1:10.0.0.13 | /dev/sdb, /dev/sdc, /dev/sdd, /dev/sde
X4.X.X     | eth0: 192.168.1.14, eth1:10.0.0.14 | /dev/sdb, /dev/sdc, /dev/sdd, /dev/sde

- eth0 用于接收 api 请求
- eth1 用于集群内部通信
- 操作系统：RockyLinux 9.3

### 挂载数据盘
- 在全部服务器上执行如下操作
- 格式化硬盘，设置卷标
    ```bash
    mkfs.xfs -f /dev/sdb -L MINIO1
    mkfs.xfs -f /dev/sdc -L MINIO2
    mkfs.xfs -f /dev/sdd -L MINIO3
    mkfs.xfs -f /dev/sde -L MINIO4
    ```

- 在 /etc/fstab 中增加挂载配置
    ```
    LABEL=MINIO1 /mnt/minio1 xfs defaults,noatime 0 2
    LABEL=MINIO2 /mnt/minio2 xfs defaults,noatime 0 2
    LABEL=MINIO3 /mnt/minio3 xfs defaults,noatime 0 2
    LABEL=MINIO4 /mnt/minio4 xfs defaults,noatime 0 2
    ```

- 创建挂载目录，挂载数据盘
    ```bash
    systemctl daemon-reload
    mkdir -p /mnt/minio{1,2,3,4}
    mount -a
    ```

- 禁用 xfs 错误重试
    ```bash
    for i in sdb sdc sdd sde; do
        echo 0 > /sys/fs/xfs/$i/error/metadata/EIO/max_retries
        echo 0 > /sys/fs/xfs/$i/error/metadata/ENOSPC/max_retries
        echo 0 > /sys/fs/xfs/$i/error/metadata/default/max_retries
    done
    ```

- 为 /etc/rc.d/rc.local 增加可执行权限
    ```bash
    chmod 0755 /etc/rc.d/rc.local
    ```

- 编辑 /etc/rc.d/rc.local，增加“禁用 xfs 错误重试”的脚本
    ```bash
    for i in sdb sdc sdd sde; do
        echo 0 > /sys/fs/xfs/$i/error/metadata/EIO/max_retries
        echo 0 > /sys/fs/xfs/$i/error/metadata/ENOSPC/max_retries
        echo 0 > /sys/fs/xfs/$i/error/metadata/default/max_retries
    done
    ```

### 安装 minio
- 在全部服务器上执行如下操作
- 下载 minio rpm 包，安装
    ```bash
    curl -L -o minio.rpm https://dl.min.io/server/minio/release/linux-amd64/archive/minio-20240406052602.0.0-1.x86_64.rpm
    rpm -Uvh minio.rpm
    groupadd -r minio-user
    useradd -r -g minio-user minio-user
    chown minio-user:minio-user /mnt/minio{1..4}
    ```

- 下载 minio 客户端工具 mc
    ```bash
    curl -L -o /usr/local/bin/mc https://dl.min.io/client/mc/release/linux-amd64/mc
    chmod 0755 /usr/local/bin/mc
    ```

- 修改 hosts，增加自定义域名解析，**注意这里是集群通信，要用 eth1 网卡 ip**
    ```
    10.0.0.11 X1.X.X
    10.0.0.12 X2.X.X
    10.0.0.13 X3.X.X
    10.0.0.14 X4.X.X
    ```

- 创建 /etc/default/minio，内容如下
    ```ini
    MINIO_VOLUMES="http://X{1...4}:9000/mnt/minio{1...4}"
    MINIO_OPTS="--console-address :9001"
    MINIO_ROOT_USER=minio-admin
    MINIO_ROOT_PASSWORD=1234567890
    ```

### 启动 minio
- 在全部服务器上执行如下操作
- 启动 minio，并配置开机自动启动
    ```bash
    systemctl daemon-reload
    systemctl start monio
    systemctl enable minio
    ```

## 客户端常用操作
- 创建 alias，**注意这里是 api 调用，要用 eth0 网卡 ip**
    ```bash
    mc alias set $ALIAS_NAME http://192.168.1.X:9000 $MINIO_ROOT_USER $MINIO_ROOT_PASSWORD
    ```

- 创建 bucket
    ```bash
    mc mb $ALIAS_NAME $BUCKET_NAME/$SUB_DIR
    ```

- 创建用户
    ```bash
    mc admin user add $ALIAS_NAME $NEW_USER_NAME $NEW_PASSWORD
    ```

- 给新增用户授权
    ```bash
    mc admin policy attach $ALIAS_NAME/$BUCKET_NAME/$SUB_DIR readwrite --user $NEW_USER_NAME
    ```