搭建HA on Zookeeper 配置要点笔记

x 搭建成功HA on Zookeeper ,特地记录下来备忘:
4台机器:
d1 = Active nn
d2 = Standby nn,zooSrv,dn
d3 = dn,zooSrv
d4 = dn.zooSrv
先安装zooSrv集群,这个就不用说了,网上有很多教程
先搭建好HA,然后重点将 on ZooKeeper配置,参考 http://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithNFS.html,最后有提到zookeeper相关的内容。


要在HA上启用ZooKeeper,主要改两个文件:
configure hdfs-site.xml
<property>
  <name>dfs.ha.automatic-failover.enabled</name>
    <value>true</value>
      <description>
          Whether automatic failover is enabled. See the HDFS High
              Availability documentation for details on automatic HA
                  configuration.
                    </description>
< /property>

< property>
  <name>dfs.client.failover.proxy.provider.hadoop-test</name>
    <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
    </property>
< !-- use sshfence default or use shell script -->
< property>
  <name>dfs.ha.fencing.methods</name>
    <value>sshfence</value>
    </property>
< property>
  <name>dfs.ha.fencing.ssh.private-key-files</name>
    <value>/home/hd2/.ssh/id_rsa</value>
    </property>
< property>
  <name>dfs.ha.fencing.ssh.connect-timeout</name>
    <value>30000</value>
    </property>



This specifies that the cluster should be set up for automatic failover. In your core-site.xml file, add:

< property>
  <name>fs.defaultFS</name>
  <!--
    <value>hdfs://d1:8020</value>
  这里注意不要指定具体的namenode,要指定nameservice-id,我在hdfs-site.xml配置的是hadoop-test:8020  -->
    <value>hdfs://hadoop-test:8020</value>
      <description>The name of the default file system.  A URI whose
        scheme and authority determine the FileSystem implementation.  The
          uri's scheme determines the config property (fs.SCHEME.impl) naming
            the FileSystem implementation class.  The uri's authority is used to
              determine the host, port, etc. for a filesystem.</description>
              </property>
< property>
   <name>ha.zookeeper.quorum</name>
      <value>d2:2181,d3:2181,d4:2181</value>
       </property>

具体执行命令

init zookeeper:
        $ hdfs zkfc -formatZK

启动HA on ZooKeeper
starting cluster exec : start-dfs.sh
or
manually exec : hadoop-daemon.sh start zkfc ,分别在 d1 和 d2 主备namenode上

HA的 process:
1. boot journalnode
2. Ann format and boot
3. Standby -bootstrapStandby and start
4. start zkfc autoselect active node


验证效果:
jps 找到d1的namenode进程号,kill 后 ,观察页面,d2上的状态由Standby变成Active
可以再次启动d1的namenode:hadoop-daemon.sh start namenode ,查看页面由Active变成Standby

0 个评论

要回复文章请先登录注册