Hadoop HA集群搭建和配置教程

620次阅读
没有评论

Task1 高可用ZooKeeper集群部署

一、ZooKeeper安装部署

1.解压安装jdk

 [root@master ~]# tar -zxvf  /opt/software/jdk-8u152-linux-x64.tar.gz -C /usr/local/src

更改jdk的名称

[root@master ~]# mv /usr/local/src/jdk1.8.0_152/ /usr/local/src/java

2.安装ZooKeeper

解压并安装zookeeper到apps下

[root@master ~]# tar -zxf /opt/software/zookeeper-3.4.8.tar.gz -C /usr/local/src/

[root@master ~]# cd /usr/local/src/

[root@master src]# mv zookeeper-3.4.8 zookeeper

3.创建ZooKeeper数据目录

data是用来传输数据的,logs是用来记录日志的

[root@master src]# mkdir /usr/local/src/zookeeper/data

[root@master src]# mkdir /usr/local/src/zookeeper/logs

二、ZooKeeper文件参数配置

1.配置ZooKeeper环境变量

[root@master src]# cd ./zookeeper

[root@master zookeeper]# vi /etc/profile

添加如下配置:

#java environment(已配置)
export JAVA_HOME=/usr/local/src/java #JAVA_HOME指向JAVA安装目录
export PATH=$PATH:$JAVA_HOME/bin #将JAVA安装目录加入PATH路径

#zookeeper environment
export ZK_HOME=/usr/local/src/zookeeper
export PATH=$PATH:$ZK_HOME/bin

保存并退出

2.修改zoo.cfg配置文件

首先先进入到conf目录下将zoo.cfg文件拷贝过来

[root@master zookeeper]# cd conf/

[root@master conf]# ls

configuration.xsl  log4j.properties  zoo_sample.cfg

[root@master conf]# cp zoo_sample.cfg zoo.cfg

[root@master conf]# vi zoo.cfg

添加并更改如下配置:

#修改
dataDir=/usr/local/src/zookeeper/data

#增加
dataLogDir=/usr/local/src/zookeeper/logs
server.1=master:2888:3888
server.2=slave1:2888:3888
server.3=slave2:2888:3888

//上面的IP可以换成自己的主机地址,或者换成主机名,一般我们换成主机名保存并退出

3.创建myid配置文件

[root@master conf]# cd ..

[root@master zookeeper]# cd data/

[root@master data]# echo “1” > myid

三、ZooKeeper集群启动

1.分发ZooKeeper集群

[root@master data]# scp -r /usr/local/src/zookeeper/ root@slave1:/usr/local/src/

[root@master data]# scp -r /usr/local/src/zookeeper/ root@slave2:/usr/local/src/

分发环境变量并使其生效

[root@master data]# scp /etc/profile root@slave1:/etc/

[root@master data]# scp /etc/profile root@slave2:/etc/

[root@master data]# source /etc/profile

[root@slave1 ~]# source /etc/profile

[root@slave2 ~]# source /etc/profile

2.修改myid配置

master对应1,slave1对应2,slave2对应3

[root@master data]# cat myid

1

[root@slave1 ~]# cd /usr/local/src/zookeeper/data/

[root@slave1 data]# echo “2”>myid

[root@slave1 data]# cat myid

2

[root@slave2 ~]# cd /usr/local/src/zookeeper/data/

[root@slave2 data]# echo “3”>myid

[root@slave2 data]# cat myid

3

3.修改ZooKeeper安装目录的归属用户为hadoop用户。

[root@master data]# chown -R hadoop:hadoop /usr/local/src/zookeeper

[root@slave1 data] # chown -R hadoop:hadoop /usr/local/src/zookeeper

[root@slave2 data] # chown -R hadoop:hadoop /usr/local/src/zookeeper

4.启动ZooKeeper集群

关闭防火墙

[root@master data]# systemctl stop firewalld.service

[root@slave1 data]# systemctl stop firewalld.service

[root@slave2 data]# systemctl stop firewalld.service

关闭防火墙自启

[root@master data]# systemctl disable firewalld.service
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

[root@slave1 data]# systemctl disable firewalld.service
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

[root@slave2 data]# systemctl disable firewalld.service
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

同时启动三个节点的zookeeper

master:

[root@master data]# su – hadoop

[hadoop@master root]$ cd

[hadoop@master ~]$ source /etc/profile

[hadoop@master ~]$ zkServer.sh start   # ZooKeeper启动

slave1:

[root@slave1 data]# su hadoop

[hadoop@slave1 root]$ cd

[hadoop@slave1 ~]$ source /etc/profile

[hadoop@slave1 ~]$ zkServer.sh start   # ZooKeeper启动

slave2

[root@slave2 data]# su hadoop

[root@slave2 data]# cd

[hadoop@slave2 ~]$ source /etc/profile

[hadoop@slave2 ~]$ zkServer.sh start   # ZooKeeper启动

显示以下内容及成功

JMX enabled by default

Jsing. config:’ /usr/local/src/zookeeper/bin/ ../conf/zoo.cfg

starting zookeeper . .STARTED

查看状态

[hadoop@master ~]$ zkServer.sh status
JMX enabled by default
Using config: /usr/local/src/zookeeper/bin/ ../conf/zoo.cfg
Mode: fo11ower  # follower状态

[hadoop@slave1 ~]$ zkServer.sh status
JMX enab1ed by default
Using config:’ /usr/local/src/zookeeper/bin/ . ./conf/zoo.cfg
Mode:leader   # leader状态

[hadoop@slave2 ~]$ zkServer.sh status
JMX enabled by default
Using config:’ /usr/local/src/zookeeper/bin/ . ./conf/zoo.cfg
Mode: fo11ower  # follower状态

Task2 高可用ZooKeeper集群部署

一、ssh免密配置

1.创建免密(三个主机同时进行)

[root@master ~]# su – hadoop

[hadoop@master ~]$ ssh-keygen -t rsa -P  “”

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory ‘/root/.ssh’.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:12nHZiHa8tK+hgmqid/Vvq5lAxpMZFFoY4lmnkoU7s4 root@master
The key’s randomart image is:
+—[RSA 2048]—-+
|  .. .+=.        |
| .. +o*          |
| ..+ +..    . .  |
| .. oo     + + . |
| …  o S + = =  |
| o.    o.+ = +   |
|  E   ….*oo    |
|   . o.. +o+.    |
|  ..+.. .o+oo.   |
+—-[SHA256]—–+

[root@master ~]# su – hadoop

[hadoop@slave1 ~]$ ssh-keygen -t rsa -P  “”

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory ‘/root/.ssh’.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:iOZ1zxts951cUelPDLPXGY0OPrFi+HDbcOLchhKf2v0 root@slave1
The key’s randomart image is:
+—[RSA 2048]—-+
|                 |
|               .o|
|            o +oo|
|     . . . . =.==|
|    o o S * = o+=|
|   o . . & @ . oo|
|    .   . # =   o|
|         = * …o|
|        . o ..Eo.|
+—-[SHA256]—–+

[root@master ~]# su – hadoop

[hadoop@slave2 ~]$ ssh-keygen -t rsa -P  “”

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory ‘/root/.ssh’.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:LxeFYtxLWfV9ZKhh9M3vM6vwuLJl6sVoJbezjeHHf1w root@slave2
The key’s randomart image is:
+—[RSA 2048]—-+
|           .o…o|
|       . . +o..*.|
|        + =..o. *|
|       . o o.   o|
|        S.oo    .|
|         .*..  .E|
|        .ooX.  +o|
|        .+* Xo  *|
|        .+o*o+oo.|
+—-[SHA256]—–+

2.创建公钥

[hadoop@master ~]$ cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys

[hadoop@master ~]$ chmod 700 ~/.ssh/authorized_keys

3.将masterr创建的公钥发给slave1

[hadoop@master ~]# scp ~/.ssh/authorized_keys root@slave1:~/.ssh/

The authenticity of host ‘slave1 (192.168.1.7)’ can’t be established.
ECDSA key fingerprint is SHA256:Nnk2MJS3KmUzmXXzgE0DTgnq990XctFMFUV82UdgFnQ.
ECDSA key fingerprint is MD5:f3:fa:be:c7:52:1e:96:ee:1b:7d:1a:26:23:a9:66:ec.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘slave1,192.168.1.7’ (ECDSA) to the list of known hosts.
root@slave1’s password:
authorized_keys                                                                                        100%  393   319.0KB/s   00:00

4.将slave1的私钥加到公钥里

[hadoop@slave1 ~]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

5.将公钥发给slave2

[hadoop@slave1 ~]$ scp ~/.ssh/authorized_keys root@slave2:~/.ssh/

The authenticity of host ‘slave2 (192.168.1.8)’ can’t be established.
ECDSA key fingerprint is SHA256:Nnk2MJS3KmUzmXXzgE0DTgnq990XctFMFUV82UdgFnQ.
ECDSA key fingerprint is MD5:f3:fa:be:c7:52:1e:96:ee:1b:7d:1a:26:23:a9:66:ec.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘slave2,192.168.1.8’ (ECDSA) to the list of known hosts.
root@slave2’s password:
authorized_keys                                                                                        100%  786   525.0KB/s   00:00  

6.登陆测试

[hadoop@master ~]$ ssh slave1

Last login: Wed Jun 24 16:41:46 2020 from 192.168.1.7

[hadoop@slave1 ~]$ ssh slave2

Last login: Wed Jun 24 16:35:46 2020 from 192.168.1.1

[hadoop@slave2 ~]$ exit

登出

Connection to slave2 closed.

[hadoop@slave1 ~]$ exit

登出

Connection to slave1 closed.

[hadoop@master ~]$

二、Hadoop HA文件参数配置

1.解压安装Hadoop

[hadoop@master ~]$ stop-all.sh

[hadoop@master ~]$ su root

之前配置过普通集群需要先删除

[root@master ~]# rm -r -f /usr/local/src/hadoop

[root@slave1 ~]# rm -r -f /usr/local/src/hadoop

[root@slave2 ~]# rm -r -f /usr/local/src/hadoop

[root@master ~]# tar -zxvf /opt/software/hadoop-2.7.1.tar.gz -C /usr/local/src/

2.更改hadoop文件名

[root@master ~]# mv /usr/local/src/hadoop-2.7.1 /usr/local/src/hadoop

3.配置hadoop环境变量

[root@master ~]# vi /etc/profile

进行如下配置(此处需先删除第四章配置的环境变量)

#hadoop enviroment

export HADOOP_HOME=/usr/local/src/hadoop  #HADOOP_HOME指向JAVA安装目录

export HADOOP_PREFIX=$HADOOP_HOME
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_INSTALL=$HADOOP_HOME
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib:$HADOOP_COMMON_LIB_NATIVE_DIR"
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
 
#java environment
export JAVA_HOME=/usr/local/src/java #JAVA_HOME指向JAVA安装目录
export PATH=$PATH:$JAVA_HOME/bin #将JAVA安装目录加入PATH路径

#zookeeper environment
export ZK_HOME=/usr/local/src/zookeeper
export PATH=$PATH:$ZK_HOME/bin

保存并退出

4.配置hadoop-env.sh配置文件

进入到hadoop/etc/hadoop下

[root@master ~]# cd /usr/local/src/hadoop/etc/hadoop

[root@master hadoop]# vi hadoop-env.sh

在最下面添加如下配置:

export JAVA_HOME=/usr/local/src/java

5.配置core-site.xml配置文件

[root@master hadoop]# vi core-site.xml

添加如下配置:

<!-- 指定hdfs的nameservice为mycluster -->
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://mycluster</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/usr/local/src/hadoop/tmp</value>
    </property>
    <!-- 指定zookeeper地址 -->
    <property>
        <name>ha.zookeeper.quorum</name>
        <value>master:2181,slave1:2181,slave2:2181</value>
    </property>
    <!-- hadoop链接zookeeper的超时时长设置 -->
    <property>
        <name>ha.zookeeper.session-timeout.ms</name>
        <value>30000</value>
        <description>ms</description>
    </property>
    <property>
        <name>fs.trash.interval</name>
        <value>1440</value>
    </property>

保存并退出

6.配置hdfs-site.xml配置文件

[root@master hadoop]# vi hdfs-site.xml

进行如下配置:

<!-- journalnode集群之间通信的超时时间 -->
    <property>
        <name>dfs.qjournal.start-segment.timeout.ms</name>
        <value>60000</value>
    </property>
    <!--指定hdfs的nameservice为mycluster,需要和core-site.xml中的保持一致        dfs.ha.namenodes.[nameservice id]为在nameservice中的每一个NameNode设置唯一标示符。配置一个逗号分隔的NameNode ID列表。这将是被DataNode识别为所有的NameNode。如果使用"mycluster"作为nameservice ID,并且使用"master"和"slave1"作为NameNodes标示符 -->
    <property>
        <name>dfs.nameservices</name>
        <value>mycluster</value>
    </property>
        <!-- mycluster下面有两个NameNode,分别是master,slave1 -->
    <property>
        <name>dfs.ha.namenodes.mycluster</name>
        <value>master,slave1</value>
    </property>
    <!-- master的RPC通信地址 -->
    <property>
        <name>dfs.namenode.rpc-address.mycluster.master</name>
        <value>master:8020</value>
    </property>
    <!-- slave1的RPC通信地址 -->
    <property>
        <name>dfs.namenode.rpc-address.mycluster.slave1</name>
        <value>slave1:8020</value>
    </property>
     <!-- master的http通信地址 -->
    <property>
        <name>dfs.namenode.http-address.mycluster.master</name>
        <value>master:50070</value>
    </property>
    <!-- slave1的http通信地址 -->
    <property>
        <name>dfs.namenode.http-address.mycluster.slave1</name>
        <value>slave1:50070</value>
    </property>
    <!-- 指定NameNode的edits元数据的共享存储位置。也就是JournalNode列表
         该url的配置格式:qjournal://host1:port1;host2:port2;host3:port3/journalId
        journalId推荐使用nameservice,默认端口号是:8485 -->
    <property>
        <name>dfs.namenode.shared.edits.dir</name>
        <value>qjournal://master:8485;slave1:8485;slave2:8485/mycluster</value>
    </property>
    <!-- 配置失败自动切换实现方式 -->
    <property>
        <name>dfs.client.failover.proxy.provider.mycluster</name>
        <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>
    <!-- 配置隔离机制方法,多个机制用换行分割,即每个机制暂用一行 -->
    <property>
        <name>dfs.ha.fencing.methods</name>
        <value>
            sshfence
        shell(/bin/true)
        </value>
    </property>
    <property>
       <name>dfs.permissions.enabled</name>
       <value>false</value>
    </property>
    <property>
        <name>dfs.support.append</name>
        <value>true</value>
    </property>
    <!-- 使用sshfence隔离机制时需要ssh免登陆 -->
    <property>
        <name>dfs.ha.fencing.ssh.private-key-files</name>
        <value>/home/hadoop/.ssh/id_rsa</value>
    </property>
    <!-- 指定副本数 -->
    <property>
        <name>dfs.replication</name>
        <value>2</value>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>/usr/local/src/hadoop/tmp/hdfs/nn</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>/usr/local/src/hadoop/tmp/hdfs/dn</value>
    </property>
    <!-- 指定JournalNode在本地磁盘存放数据的位置 -->
    <property>
        <name>dfs.journalnode.edits.dir</name>
        <value>/usr/local/src/hadoop/tmp/hdfs/jn</value>
    </property>
    <!-- 开启NameNode失败自动切换 -->
    <property>
        <name>dfs.ha.automatic-failover.enabled</name>
        <value>true</value>
    </property>
    <!-- 启用webhdfs -->
    <property>
        <name>dfs.webhdfs.enabled</name>
        <value>true</value>
    </property>
    <!-- 配置sshfence隔离机制超时时间 -->
    <property>
        <name>dfs.ha.fencing.ssh.connect-timeout</name>
        <value>30000</value>
    </property>
    <property>
        <name>ha.failover-controller.cli-check.rpc-timeout.ms</name>
        <value>60000</value>
    </property>

保存并退出

7.配置mapred-site.xml配置文件

[root@master hadoop]# cp mapred-site.xml.template mapred-site.xml

[root@master hadoop]# vi mapred-site.xml

进行如下配置:

<!-- 指定mr框架为yarn方式 -->
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
        </property>
    <!-- 指定mapreduce jobhistory地址 -->
    <property>
        <name>mapreduce.jobhistory.address</name>
        <value>master:10020</value>
    </property>
    <!-- 任务历史服务器的web地址 -->
    <property>
        <name>mapreduce.jobhistory.webapp.address</name>
        <value>master:19888</value>
      </property>

保存并退出

8.配置yarn-site.xml配置文件

[root@master hadoop]# vi yarn-site.xml

进行如下配置:

<!-- Site specific YARN configuration properties -->
    <!-- 开启RM高可用 -->
    <property>
        <name>yarn.resourcemanager.ha.enabled</name>
        <value>true</value>
    </property>
    <!-- 指定RM的cluster id -->
    <property>
        <name>yarn.resourcemanager.cluster-id</name>
        <value>yrc</value>
    </property>
    <!-- 指定RM的名字 -->
    <property>
        <name>yarn.resourcemanager.ha.rm-ids</name>
        <value>rm1,rm2</value>
    </property>
    <!-- 分别指定RM的地址 -->
    <property>
        <name>yarn.resourcemanager.hostname.rm1</name>
        <value>master</value>
    </property>
    <property>
        <name>yarn.resourcemanager.hostname.rm2</name>
        <value>slave1</value>
    </property>
    <!-- 指定zk集群地址 -->
    <property>
        <name>yarn.resourcemanager.zk-address</name>
        <value>master:2181,slave1:2181,slave2:2181</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.log-aggregation-enable</name>
        <value>true</value>
    </property>
    <property>
        <name>yarn.log-aggregation.retain-seconds</name>
        <value>86400</value>
    </property>
    <!-- 启用自动恢复 -->
    <property>
        <name>yarn.resourcemanager.recovery.enabled</name>
        <value>true</value>
    </property>
    <!-- 制定resourcemanager的状态信息存储在zookeeper集群上 -->
    <property>
        <name>yarn.resourcemanager.store.class</name>
   <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
    </property>

保存并退出

9.配置slaves配置文件

[root@master hadoop]# vi slaves

进行如下配置:

master
slave1
slave2

10.解压包到指定目录

namenode、datanode、journalnode等存放数据的公共目录为/usr/local/src/hadoop/tmp;

 在master上执行如下:

[root@master hadoop]#  mkdir -p /usr/local/src/hadoop/tmp/hdfs/nn

[root@master hadoop]#  mkdir -p /usr/local/src/hadoop/tmp/hdfs/dn

[root@master hadoop]#  mkdir -p /usr/local/src/hadoop/tmp/hdfs/jn

[root@master hadoop]#  mkdir -p /usr/local/src/hadoop/tmp/logs

11.分发文件

[root@master hadoop]# scp -r /etc/profile root@slave1:/etc/

[root@master hadoop]# scp -r /etc/profile root@slave2:/etc/

[root@master hadoop]# scp -r /usr/local/src/hadoop root@slave1:/usr/local/src/

[root@master hadoop]# scp -r /usr/local/src/hadoop root@slave2:/usr/local/src/

12.修改目录所有者和所有者组

上述安装完成的Hadoop软件只能让root用户使用,要让hadoop用户能够运行Hadoop软件,需要将目录/usr/local/src的所有者改为hadoop用户。

[root@master ~]# chown -R hadoop:hadoop /usr/local/src/hadoop

[root@master ~]# ll /usr/local/src/

[root@slave1~]# chown -R hadoop:hadoop /usr/local/src/hadoop/ 

[root@slave2~]# chown -R hadoop:hadoop /usr/local/src/hadoop/ 

13.生效环境变量

[root@master hadoop]# su hadoop

[hadoop@master hadoop]$ source /etc/profile

[root@slave1 hadoop]# su hadoop

[hadoop@slave1 hadoop]$ source /etc/profile

[root@slave2 hadoop]# su hadoop

[hadoop@slave2 hadoop]$ source /etc/profile

到点睡觉了
版权声明:本站原创文章,由 到点睡觉了2022-01-06发表,共计16156字。
转载说明:除特殊说明外本站文章皆由CC-4.0协议发布,转载请注明出处。
评论(没有评论)