www.colben.cn/content/post/redis-cluster-install.md

---
title: "CentOS7 安装 Redis 集群"
date: 2019-11-29T15:14:29+08:00
lastmod: 2019-11-29T15:14:29+08:00
tags: ["redis"]
categories: ["database"]
---

## 环境
- 操作系统: CentOS7.7，关闭 firewalld 和 selinux
- Redis: 5.0.6
- master 实例
    - 10.0.4.110:6371
    - 10.0.4.110:6372
    - 10.0.4.110:6373
- slave 实例
    - 10.0.4.111:6371
    - 10.0.4.111:6372
    - 10.0.4.111:6373

## 编译 redis server
- 在 CentOS7.7 测试服务器上下载、编译
    ```bash
    wget http://download.redis.io/releases/redis-5.0.6.tar.gz
    tar xzf redis-5.0.6.tar.gz
    cd redis-5.0.6
    make
    ```
- 抽取安装文件
    ```bash
    mkdir -p /opt/redis_cluster/{bin,conf}
    cp src/{redis-server,redis-cli,redis-benchmark} /opt/redis_cluster/bin/
    # redis-check-rdb 和 redis-check-aof 与 redis-server 校验值一致，无需复制
    ```
- 创建 /opt/redis_cluster/conf/redis.conf，常用配置如下
    ```bash
    bind 0.0.0.0
    protected-mode yes
    tcp-backlog 511
    timeout 3600
    tcp-keepalive 300
    daemonize no
    supervised no
    loglevel notice
    always-show-logo no
    save 900 1
    save 300 10
    save 60 10000
    stop-writes-on-bgsave-error yes
    rdbcompression yes
    rdbchecksum yes
    dbfilename dump.rdb
    #masterauth "123456"
    replica-serve-stale-data yes
    replica-read-only yes
    repl-diskless-sync no
    repl-diskless-sync-delay 5
    repl-disable-tcp-nodelay no
    replica-priority 100
    #requirepass "123456"
    maxclients 10000
    maxmemory 4294967296
    maxmemory-policy volatile-lru
    lazyfree-lazy-eviction no
    lazyfree-lazy-expire no
    lazyfree-lazy-server-del no
    replica-lazy-flush no
    appendonly no
    appendfilename "appendonly.aof"
    appendfsync everysec
    no-appendfsync-on-rewrite no
    auto-aof-rewrite-percentage 100
    auto-aof-rewrite-min-size 64mb
    aof-load-truncated yes
    aof-use-rdb-preamble yes
    lua-time-limit 5000
    cluster-enabled yes
    cluster-node-timeout 5000
    slowlog-log-slower-than 10000
    slowlog-max-len 128
    latency-monitor-threshold 0
    notify-keyspace-events ""
    hash-max-ziplist-entries 512
    hash-max-ziplist-value 64
    list-max-ziplist-size -2
    list-compress-depth 0
    set-max-intset-entries 512
    zset-max-ziplist-entries 128
    zset-max-ziplist-value 64
    hll-sparse-max-bytes 3000
    stream-node-max-bytes 4096
    stream-node-max-entries 100
    activerehashing yes
    client-output-buffer-limit normal 0 0 0
    client-output-buffer-limit replica 256mb 64mb 60
    client-output-buffer-limit pubsub 32mb 8mb 60
    hz 10
    dynamic-hz yes
    aof-rewrite-incremental-fsync yes
    rdb-save-incremental-fsync yes
    ```
- 创建 /opt/redis_cluster/redis@.service，配置如下
    ```ini
    [Unit]
    Description=Redis cluster -- node %i
    After=network.target

    [Service]
    ExecStartPre=/usr/bin/ls -d /var/lib/redis_cluster/%i
    ExecStart=/opt/redis_cluster/bin/redis-server /opt/redis_cluster/conf/redis.conf \
        --daemonize no \
        --port %i \
        --pidfile /var/run/redis_%i.pid \
        --logfile /var/log/redis_cluster/%i.log \
        --dir /var/lib/redis_cluster/%i \
        --cluster-config-file /var/lib/redis_cluster/%i/nodes.conf
    ExecStop=/opt/redis_cluster/bin/redis-cli -h 127.0.0.1 -p %i shutdown
    Restart=on-failure
    RestartSec=1

    [Install]
    WantedBy=multi-user.target
    ```

## 部署
- 把测试机的 /opt/redis_cluster 目录复制到 10.0.4.110 和 10.0.4.111 上
    ```bash
    scp -r /opt/redis_cluster 10.0.4.110:/opt/
    scp -r /opt/redis_cluster 10.0.4.111:/opt/
    ```
- 在 10.0.4.110 和 10.0.4.111 上都执行以下操作
    ```bash
    mv /opt/redis_cluster/redis@.service /usr/lib/systemd/system/
    systemctl daemon-reload
    mkdir -p /var/lib/redis_cluster/{6371..6373}
    mkdir -p /var/log/redis_cluster
    ```
- 在 10.0.4.110 和 10.0.4.111 上配置实例开机启动
    ```bash
    systemctl enable redis@6371
    systemctl enable redis@6372
    systemctl enable redis@6373
    ```
- 在 10.0.4.110 和 10.0.4.111 上启动全部实例
    ```bash
    systemctl start redis@6371
    systemctl start redis@6372
    systemctl start redis@6373
    ```

## 配置集群
- 在 10.0.4.110 或 10.0.4.111 上创建集群，指定三个 master 实例
    ```bash
    /opt/redis_cluster/bin/redis-cli \
        --cluster create 10.0.4.110:6371 10.0.4.110:6372 10.0.4.110:6373
    ```
- 输出如下图
![](https://colben.cn/img/dc44a6e8688218edb10ad7f2427b0525132.png)
- 同意配置，输入 "yes"，回车后输出如下图
![](https://colben.cn/img/eb3c0ccd3d0017c18d4eba3a0b776d29e21.png)
- 查看集群当前实例
    ```bash
    /opt/redis_cluster/bin/redis-cli -p 6371 cluster nodes
    ```
- 输出如下图
![](https://colben.cn/img/0bfe3ac4984fbd6973738dddb20d5bb6c72.png)
- 添加 slave 实例 10.0.4.111:6371，同步 master 10.0.4.110:6371
    ```bash
    /opt/redis_cluster/bin/redis-cli \
        --cluster add-node 10.0.4.111:6371 10.0.4.110:6371 \
        --cluster-slave \
        --cluster-master-id 5be0edaf74d1eec50bbe675f9698397023a943b7
    # 上述命令中，10.0.4.111:6371 是待添加实例，10.0.4.110:6371 是已存在集群中的任一实例
    # --cluster-slave 指定添加 slave 实例
    # --cluster-master-id 指定 10.0.4.110:6371 的 id，即本次添加实例的 master
    ```
- 输出如下图
![](https://colben.cn/img/cf50fb5da24afb930eab80100707b13efbe.png)
- 添加 slave 实例 10.0.4.111:6372，同步 master 10.0.4.110:6372
    ```bash
    /opt/redis_cluster/bin/redis-cli \
        --cluster add-node 10.0.4.111:6372 10.0.4.110:6371 \
        --cluster-slave \
        --cluster-master-id 96f2225fdd054d8a16b31726fd1eb0b66c612646
    ```
- 输出如下图
![](https://colben.cn/img/4577cd8a2c7aea037e50fb002f91bb40842.png)
- 添加 slave 实例 10.0.4.111:6373，同步 master 10.0.4.110:6373
    ```bash
    /opt/redis_cluster/bin/redis-cli \
        --cluster add-node 10.0.4.111:6373 10.0.4.110:6371 \
        --cluster-slave \
        --cluster-master-id cd60c238e66c8b4a31355bee4610444a47acef86
    ```
- 输出如下图
![](https://colben.cn/img/633adea823cad6380030826ce75a44109fb.png)
- 查看集群当前实例
    ```bash
    /opt/redis_cluster/bin/redis-cli -p 6371 cluster nodes
    ```
- 输出如下图
![](https://colben.cn/img/3d5e7c0b8e7a7e51028d11358f442947367.png)

## Docker 部署
- 安装 docker-ce，参考[这里](https://www.colben.cn/post/docker/#%E5%AE%89%E8%A3%85)
- 安装 docker-compose，参考[这里](https://www.colben.cn/post/docker/#Compose)
- 选择一个较大分区，创建 redis 集群数据目录和日志目录
    ```bsah
    mkdir -p redis_cluster/{db,log}/{6371..6376}
    ```

- 在同级目录下创建 docker-compose.yml 文件，内容如下
    ```yml
    version: "3.7"

    services:
      redis-6371:
        image: harbor.colben.cn/general/alpine-redis
        container_name: redis-6371
        restart: on-failure
        stop_grace_period: 1m
        privileged: true
        network_mode: "host"
        environment:
          _CENV_cluster-enabled: "yes"
          _CENV_port: 6371
          _CENV_maxmemory: 256MB
          # master 和 slave 实例的密码必须一致，可都为空
          _CENV_masterauth: Pass_1234
          _CENV_requirepass: Pass_1234
          # master 节点列表
          MASTER_NODES: "127.0.0.1:6371 127.0.0.1:6372 127.0.0.1:6373"
          # slave 节点列表
          # 可以为空，表示没有 slave 节点
          # 不为空时，数量必须与 master 节点数量一致，按顺序依次为 master 节点的从节点
          SLAVE_NODES: "127.0.0.1:6374 127.0.0.1:6375 127.0.0.1:6376"
        volumes:
        - type: bind
          source: ./redis_cluster/db/6371
          target: /var/lib/redis
        - type: bind
          source: ./redis_cluster/log/6371
          target: /var/log/redis

      redis-6372:
        image: harbor.colben.cn/general/alpine-redis
        container_name: redis-6372
        restart: on-failure
        stop_grace_period: 1m
        privileged: true
        network_mode: "host"
        environment:
          _CENV_cluster-enabled: "yes"
          _CENV_port: 6372
          _CENV_maxmemory: 256MB
          _CENV_masterauth: Pass_1234
          _CENV_requirepass: Pass_1234
        volumes:
        - type: bind
          source: ./redis_cluster/db/6372
          target: /var/lib/redis
        - type: bind
          source: ./redis_cluster/log/6372
          target: /var/log/redis

      redis-6373:
        image: harbor.colben.cn/general/alpine-redis
        container_name: redis-6373
        restart: on-failure
        stop_grace_period: 1m
        privileged: true
        network_mode: "host"
        environment:
          _CENV_cluster-enabled: "yes"
          _CENV_port: 6373
          _CENV_maxmemory: 256MB
          _CENV_masterauth: Pass_1234
          _CENV_requirepass: Pass_1234
        volumes:
        - type: bind
          source: ./redis_cluster/db/6373
          target: /var/lib/redis
        - type: bind
          source: ./redis_cluster/log/6373
          target: /var/log/redis

      redis-6374:
        image: harbor.colben.cn/general/alpine-redis
        container_name: redis-6374
        restart: on-failure
        stop_grace_period: 1m
        privileged: true
        network_mode: "host"
        environment:
          _CENV_cluster-enabled: "yes"
          _CENV_port: 6374
          _CENV_maxmemory: 256MB
          _CENV_masterauth: Pass_1234
          _CENV_requirepass: Pass_1234
        volumes:
        - type: bind
          source: ./redis_cluster/db/6374
          target: /var/lib/redis
        - type: bind
          source: ./redis_cluster/log/6374
          target: /var/log/redis

      redis-6375:
        image: harbor.colben.cn/general/alpine-redis
        container_name: redis-6375
        restart: on-failure
        stop_grace_period: 1m
        privileged: true
        network_mode: "host"
        environment:
          _CENV_cluster-enabled: "yes"
          _CENV_port: 6375
          _CENV_maxmemory: 256MB
          _CENV_masterauth: Pass_1234
          _CENV_requirepass: Pass_1234
        volumes:
        - type: bind
          source: ./redis_cluster/db/6375
          target: /var/lib/redis
        - type: bind
          source: ./redis_cluster/log/6375
          target: /var/log/redis

      redis-6376:
        image: harbor.colben.cn/general/alpine-redis
        container_name: redis-6376
        restart: on-failure
        stop_grace_period: 1m
        privileged: true
        network_mode: "host"
        environment:
          _CENV_cluster-enabled: "yes"
          _CENV_port: 6376
          _CENV_maxmemory: 256MB
          _CENV_masterauth: Pass_1234
          _CENV_requirepass: Pass_1234
        volumes:
        - type: bind
          source: ./redis_cluster/db/6376
          target: /var/lib/redis
        - type: bind
          source: ./redis_cluster/log/6376
          target: /var/log/redis
    ```

- 启动 redis 集群
    ```bash
    docker-compose up -d
    ```

## 参考
- [redis 官网](https://redis.io/topics/cluster-tutorial)