colben/www.colben.cn
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

udpate

Browse Source
This commit is contained in:
colben
2026-04-10 19:42:34 +08:00
parent b9d55e504c
commit 08759d9323
2 changed files with 80 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
content/post/hdp2.md
View File

@@ -233,7 +233,7 @@ hbase-rs1 | 192.168.8.33/24 | - | HbaseRigionServer
hdp-slave1
```
### 部署 spark 集群
### 部署 spark-on-yarn
- 在 **hdp-X** 上执行如下操作
- 下载 spark-3.3.4-bin-hadoop2.tgz，解压
```bash
@@ -272,8 +272,10 @@ hbase-rs1 | 192.168.8.33/24 | - | HbaseRigionServer
- 创建 $SPARK_HOME/conf/spark-env.sh，参考内容如下
```bash
export JAVA_HOME=/opt/jdk
export HADOOP_CONF_DIR=${HADOOP_HOME}/etc/hadoop
export SPARK_HISTORY_OPTS="-Dspark.history.ui.port=18080 -Dspark.history.fs.logDirectory=hdfs://hdp-nn:8020/spark-logs -Dspark.history.retainedApplications=30"
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native
```
### 格式化 namenode