181 lines
4.9 KiB
Markdown
181 lines
4.9 KiB
Markdown
---
|
||
title: "RockyLinux 8.5 安装 K3S"
|
||
date: 2020-06-25T14:21:00+08:00
|
||
lastmod: 2023-09-08T12:32:00+08:00
|
||
keywords: []
|
||
tags: ["rancher", "k3s"]
|
||
categories: ["container"]
|
||
---
|
||
|
||
## 环境
|
||
|
||
角色 | 主机名 | IP | 操作系统 | 软件
|
||
---- | ---- | ---- | ---- | ----
|
||
镜像库 | k3s-170 | 10.0.4.170 | RockyLinux 8.5 | registry
|
||
数据库 | k3s-170 | 10.0.4.170 | RockyLinux 8.5 | mysql
|
||
负载均衡 | k3s-170 | 10.0.4.170 | RockyLinux 8.5 | nginx
|
||
k3s server | k3s-171 | 10.0.4.171 | RockyLinux 8.5 | k3s v1.27.4
|
||
k3s server | k3s-172 | 10.0.4.172 | RockyLinux 8.5 | k3s v1.27.4
|
||
k3s agent | k3s-173 | 10.0.4.173 | RockyLinux 8.5 | k3s v1.27.4
|
||
k3s agent | k3s-174 | 10.0.4.174 | RockyLinux 8.5 | k3s v1.27.4
|
||
k3s agent | k3s-175 | 10.0.4.175 | RockyLinux 8.5 | k3s v1.27.4
|
||
|
||
## 前期准备
|
||
- **全部服务器关闭 firewalld、selinux 和 swap，设置时间同步**
|
||
- **全部 k3s 服务器(除了k3s-170)必须设置唯一主机名**
|
||
- 在 k3s-170 上执行如下操作
|
||
- 安装 docker 环境，部署私有镜像库
|
||
- 安装 mysql，用来存储 k3s server 数据
|
||
- 安装 nginx，四层均衡负载两个 k3s server
|
||
- 下载 k3s 二进制文件和部署脚本
|
||
|
||
## 安装私有镜像库
|
||
- 在 k3s-170 上执行如下操作
|
||
- 修改 docker 配置，在 /etc/docker/daemon.json 中增加本机镜像库地址
|
||
```json
|
||
{
|
||
"insecure-registries": ["10.0.4.170:20080"]
|
||
}
|
||
```
|
||
|
||
- 安装私有镜像库 registry，监听 http 20080
|
||
- 下载镜像列表文件
|
||
```bash
|
||
curl -LO https://github.com/k3s-io/k3s/releases/download/v1.27.4%2Bk3s1/k3s-images.txt
|
||
```
|
||
|
||
- 下载 k3s 系统镜像，推送到本机私有镜像库
|
||
```bash
|
||
for image in $(cat k3s-images.txt); do
|
||
docker pull $image
|
||
docker tag $image ${image/#docker.io/10.0.4.170:20080}
|
||
docker push ${image/#docker.io/10.0.4.170:20080}
|
||
done
|
||
```
|
||
|
||
## 安装数据库
|
||
- 在 k3s-170 上执行如下操作
|
||
- 安装 mysql
|
||
- 创建 k3s 数据库
|
||
```bash
|
||
CREATE DATABASE k3s DEFAULT CHARSET UTF8MB4;
|
||
CREATE USER k3s@'%' IDENTIFIED BY 'K3s_1234';
|
||
GRANT ALL ON k3s.* TO k3s@'%';
|
||
```
|
||
|
||
## 安装负载均衡器
|
||
- 在 k3s-170 上执行如下操作
|
||
- 安装 nginx
|
||
- 配置 stream 四层转发
|
||
```
|
||
stream {
|
||
upstream k3s_servers {
|
||
server 10.0.4.171:6443;
|
||
server 10.0.4.172:6443;
|
||
}
|
||
server {
|
||
listen 6443;
|
||
proxy_pass k3s_servers;
|
||
}
|
||
}
|
||
```
|
||
|
||
## 分发 k3s 部署文件
|
||
- 在 k3s-170 上执行如下操作
|
||
- 下载安装脚本(install-k3s.sh)和可执行文件(k3s)
|
||
```bash
|
||
curl -Lo install-k3s.sh get.k3s.io
|
||
curl -LO https://github.com/k3s-io/k3s/releases/download/v1.27.4%2Bk3s1/k3s
|
||
```
|
||
|
||
- 把安装脚本和二进制文件分发到每台 k3s server 和 k3s agent 服务器上
|
||
```bash
|
||
seq -f'10.0.4.%g' 171 175 | xargs -i scp k3s install-k3s.sh {}:/usr/local/bin
|
||
```
|
||
|
||
## 安装 k3s server
|
||
- 在每台 k3s server 上执行如下操作
|
||
- 增加可执行权限
|
||
```bash
|
||
cd /usr/local/bin
|
||
chmod 0755 k3s install-k3s.sh
|
||
```
|
||
|
||
- 创建私有镜像库配置文件
|
||
```bash
|
||
mkdir -p /etc/rancher/k3s
|
||
cat >> /etc/rancher/k3s/registries.yaml <<-EOF
|
||
mirrors:
|
||
docker.io:
|
||
endpoint:
|
||
- "http://10.0.4.170:20080"
|
||
EOF
|
||
```
|
||
|
||
- 执行安装脚本
|
||
```bash
|
||
INSTALL_K3S_EXEC='server --prefer-bundled-bin --disable "coredns,servicelb,traefik,local-storage,metrics-server" --datastore-endpoint="mysql://k3s:K3s_1234@tcp(10.0.4.170:3306)/k3s"' \
|
||
INSTALL_K3S_SKIP_DOWNLOAD=true \
|
||
INSTALL_K3S_SKIP_START=true \
|
||
K3S_TOKEN=123456 \
|
||
install-k3s.sh
|
||
```
|
||
|
||
- 修改 /etc/systemd/system/k3s.service，**把 mysql 一行中的反斜杠(\\)全部去掉**，重载
|
||
```bash
|
||
systemctl daemon-reload
|
||
```
|
||
|
||
- 启动 k3s
|
||
```bash
|
||
systemctl start k3s
|
||
```
|
||
|
||
|
||
## 安装 k3s agent
|
||
- 在每台 k3s agent 上执行如下操作
|
||
- 增加可执行权限
|
||
```bash
|
||
cd /usr/local/bin
|
||
chmod 0755 k3s install-k3s.sh
|
||
```
|
||
|
||
- 创建私有镜像库配置文件
|
||
```bash
|
||
mkdir -p /etc/rancher/k3s
|
||
cat >> /etc/rancher/k3s/registries.yaml <<-EOF
|
||
mirrors:
|
||
docker.io:
|
||
endpoint:
|
||
- "http://10.0.4.170:20080"
|
||
EOF
|
||
```
|
||
|
||
- 执行安装脚本
|
||
```bash
|
||
INSTALL_K3S_EXEC='agent --prefer-bundled-bin --server "https://10.0.4.170:6443"' \
|
||
INSTALL_K3S_SKIP_DOWNLOAD=true \
|
||
K3S_TOKEN=123456 \
|
||
install-k3s.sh
|
||
```
|
||
|
||
## 常用操作
|
||
- 在 k3s server 上查看节点信息
|
||
```bash
|
||
kubectl get nodes
|
||
```
|
||
|
||
- 在 k3s server 上标记 server 节点不可调度
|
||
```bash
|
||
kubectl cordon k3s-{171..172}
|
||
```
|
||
|
||
- 在 k3s server 上设置 agent 节点的角色为 worker
|
||
```bash
|
||
kubectl label nodes k3s-{173..175} kubernetes.io/role=worker --overwrite
|
||
```
|
||
|
||
## 注意事项
|
||
- k3s 自签名 ca 证书有效期十年，客户端和服务器证书有效期一年
|
||
|