182 lines
5.4 KiB
Markdown
182 lines
5.4 KiB
Markdown
---
|
||
title: "ClickHouse 集群"
|
||
date: 2020-10-09T22:14:00+08:00
|
||
lastmod: 2020-10-09T22:14:00+08:00
|
||
tags: []
|
||
categories: ["clickhouse"]
|
||
---
|
||
|
||
# 简介
|
||
- 一个集群包含多个逻辑分片，每个逻辑分片包含多个副本节点
|
||
- 向集群内读写数据时，需依赖 Distributed 引擎表做为代理，实现数据的分发、写入、查询和路由
|
||
- ReplicatedMerge 表引擎配合 zookeeper 实现数据的复制
|
||
|
||
# 集群配置
|
||
- 节点分配
|
||
```xml
|
||
<remote_servers>
|
||
<cluster_3s2r> <!-- 集群名称 -->
|
||
<shard> <!-- 分片配置 -->
|
||
<internal_replication>true</internal_replication> <!-- 使用 ReplicatedMergeTree 引擎复制数据 -->
|
||
<weight>1<weight> <!-- 分片权重 -->
|
||
<replica>
|
||
<host>10.1.4.181</host> <!-- 副本节点的域名/IP -->
|
||
<port>9000</port> <!-- 副本节点的端口 -->
|
||
</replica>
|
||
<replica>
|
||
<host>10.1.4.182</host>
|
||
<port>9000</port>
|
||
</replica>
|
||
</shard>
|
||
<shard>
|
||
<internal_replication>true</internal_replication>
|
||
<replica>
|
||
<host>10.1.4.183</host>
|
||
<port>9000</port>
|
||
</replica>
|
||
<replica>
|
||
<host>10.1.4.184</host>
|
||
<port>9000</port>
|
||
</replica>
|
||
</shard>
|
||
<shard>
|
||
<internal_replication>true</internal_replication>
|
||
<replica>
|
||
<host>10.1.4.185</host>
|
||
<port>9000</port>
|
||
</replica>
|
||
<replica>
|
||
<host>10.1.4.186</host>
|
||
<port>9000</port>
|
||
</replica>
|
||
</shard>
|
||
</cluster_3s2r>
|
||
</remote_servers>
|
||
```
|
||
|
||
- 各节点的宏变量
|
||
- 10.0.4.181
|
||
```xml
|
||
<macros>
|
||
<shard>1</shard>
|
||
<replica>10.1.4.181</replica>
|
||
</macros>
|
||
```
|
||
- 10.0.4.182
|
||
```xml
|
||
<macros>
|
||
<shard>1</shard>
|
||
<replica>10.1.4.182</replica>
|
||
</macros>
|
||
```
|
||
- 10.0.4.183
|
||
```xml
|
||
<macros>
|
||
<shard>2</shard>
|
||
<replica>10.1.4.183</replica>
|
||
</macros>
|
||
```
|
||
- 10.0.4.184
|
||
```xml
|
||
<macros>
|
||
<shard>2</shard>
|
||
<replica>10.1.4.184</replica>
|
||
</macros>
|
||
```
|
||
- 10.0.4.185
|
||
```xml
|
||
<macros>
|
||
<shard>3</shard>
|
||
<replica>10.1.4.185</replica>
|
||
</macros>
|
||
```
|
||
- 10.0.4.186
|
||
```xml
|
||
<macros>
|
||
<shard>3</shard>
|
||
<replica>10.1.4.186</replica>
|
||
</macros>
|
||
```
|
||
|
||
- Zookeeper
|
||
```xml
|
||
<zookeeper>
|
||
<node index="1">
|
||
<host>10.0.4.101</host>
|
||
<port>2181</port>
|
||
</node>
|
||
<node index="2">
|
||
<host>10.0.4.102</host>
|
||
<port>2181</port>
|
||
</node>
|
||
<node index="3">
|
||
<host>10.0.4.103</host>
|
||
<port>2181</port>
|
||
</node>
|
||
</zookeeper>
|
||
```
|
||
|
||
- system.zookeeper: zookeeper 代理表，可通过 sql 查看 zookeeper 信息
|
||
- system.clusters: 集群信息表
|
||
|
||
# ReplicatedMergeTree 表引擎
|
||
- 引入 zookeeper 实现分布式协同，zookeeper 本身不涉及表数据传输
|
||
- 副本节点是多主架构，可在节点上执行读写操作
|
||
- 数据块: 默认 1048576 行(max_insert_block_size)
|
||
- 基本基本写入单元
|
||
- 原子性: 一个块内的数据，要么都写入成功，要么都失败
|
||
- 唯一性: 记录 hash 信息，相同的数据块会被忽略
|
||
|
||
|
||
## 创建 ReplicatedMergeTree 引擎表
|
||
- 声明
|
||
```sql
|
||
CREATE TABLE table_name_local ON CLUSTER cluster_name_2
|
||
ENGINE = ReplicatedMergeTree(
|
||
'/clickhouse/tables/{shard}/db_name/table_name_local',
|
||
'{replica}'
|
||
)
|
||
```
|
||
|
||
- table_name_local: 本地表名，推荐以 \_local 为后缀
|
||
- cluster_name_2: 在该集群内创建数据库和数据表的分片和副本
|
||
- /clickhouse/tables/ 是约定俗成的固定 zookeeper path 路径
|
||
- {shard}: 分片编号，从各自节点的宏变量中获取
|
||
- db_name: 数据库名
|
||
- {replica}: 节点域名/IP，从各自节点的宏变量中获取
|
||
|
||
# Distributed 表引擎
|
||
- 又叫分布式表，自身不存储数据，只代理数据分片
|
||
|
||
## 创建 Distributed 引擎表
|
||
- 声明
|
||
```sql
|
||
CREATE TABLE table_name_all ON CLUSTER cluster_name_1
|
||
ENGINE = Distributed(cluster_name_2, db, table, [,sharding_key])
|
||
```
|
||
|
||
- table_name_all: 分布式表名，通常以 \_all 为后缀
|
||
- ON CLUSTER: 集群操作
|
||
- cluster_name_1: 在该集群内创建分布式表 table_name_all
|
||
- cluster_name_2: 数据的分片和副本所在集群
|
||
- db: 数据库名
|
||
- table_name_local: 数据表名，即前面创建的 ReplicatedMergeTree 引擎表，通常以 \_local 为后缀
|
||
- sharding_key: 分片键，可以是整型列字段或返回整型的表达式，决定数据分配到哪些节点中
|
||
|
||
# 分布式查询
|
||
- 分布式表(Distributed)把查询转换为并行的各分片查询
|
||
- 汇总各分片的查询结果
|
||
|
||
## GlOBAL 优化查询
|
||
- 场景: 涉及到 JOIN 和 IN 时，可能会放大分布式查询
|
||
- GLOBAL 查询过程:
|
||
- 提出 IN 子句，发起分布式查询
|
||
- 汇总 IN 子句在各分片的查询结果，存入临时表(内存)
|
||
- 把这个临时表发送到其他分片节点，**考虑到该表既要内存存储，又要通过网络分发，所以数据量不宜过大**
|
||
- 在各分片节点执行完整的 SQL 语句，此时 IN 子句直接使用上一步的临时表
|
||
- 使用示例
|
||
```sql
|
||
SELECT ... WHERE ... AND ... GLOBAL IN (...)
|
||
```
|
||
|