博客

参考：

Hadoop3.0完全分布式集群搭建方法（CentOS7+Hadoop3.2.0）

我在虚拟机上安装了3台centos7,并且以普通用户安装hadoop于 /usr/local下

1.在首次启动hadoop,格式化 hadoop namenode -format  时 ，提示如下错误：

错误总结：

（1）/tmp 文件夹下权限不足，开放 /tmp 文件夹权限即可 sudo chmod -R 777 /tmp

（2） 以普通用户在hadoop01上启动start-dfs.sh时，在hadoop01上可以查到namnode进程，但是在hadoop02和hadoop03上无法查到datanode进程，因为我将配置文件中的文件夹位置定义在 /data下，子节点无权限在根目录下建立此文件夹，手动创建/data文件夹并赋予足够的权限即可。

（3）多次格式化hdfs后，无法启动datanode,原因为namenode和datanode的ID不一致导致无法启动datanode，可以选择修改ID或者删除dfs文件夹（前提是里面没有重要文件）。参考：

https://blog.csdn.net/qq_42239069/article/details/83513948

Hadoop的启动和停止说明

sbin/start-all.sh 启动所有的Hadoop守护进程。包括NameNode、 Secondary NameNode、DataNode、ResourceManager、NodeManager

sbin/stop-all.sh 停止所有的Hadoop守护进程。包括NameNode、 Secondary NameNode、DataNode、ResourceManager、NodeManager

sbin/start-dfs.sh 启动Hadoop HDFS守护进程NameNode、SecondaryNameNode、DataNode

sbin/stop-dfs.sh 停止Hadoop HDFS守护进程NameNode、SecondaryNameNode和DataNode

sbin/hadoop-daemons.sh start namenode 单独启动NameNode守护进程

sbin/hadoop-daemons.sh stop namenode 单独停止NameNode守护进程

sbin/hadoop-daemons.sh start datanode 单独启动DataNode守护进程

sbin/hadoop-daemons.sh stop datanode 单独停止DataNode守护进程

sbin/hadoop-daemons.sh start secondarynamenode 单独启动SecondaryNameNode守护进程

sbin/hadoop-daemons.sh stop secondarynamenode 单独停止SecondaryNameNode守护进程

sbin/start-yarn.sh 启动ResourceManager、NodeManager

sbin/stop-yarn.sh 停止ResourceManager、NodeManager

sbin/yarn-daemon.sh start resourcemanager 单独启动ResourceManager

sbin/yarn-daemons.sh start nodemanager  单独启动NodeManager

sbin/yarn-daemon.sh stop resourcemanager 单独停止ResourceManager

sbin/yarn-daemons.sh stopnodemanager  单独停止NodeManager

sbin/mr-jobhistory-daemon.sh start historyserver 手动启动jobhistory

sbin/mr-jobhistory-daemon.sh stop historyserver 手动停止jobhistory

首次启动错误如下，对应第一个问题：

WARNING: Use of this script to execute namenode is deprecated.
WARNING: Attempting to execute replacement "hdfs namenode" instead.

/usr/local/hadoop-3.2.0/libexec/hadoop-functions.sh: line 1801: /tmp/hadoop-hadoop-namenode.pid: Permission denied
ERROR:  Cannot write namenode pid /tmp/hadoop-hadoop-namenode.pid.
2019-06-28 22:34:38,804 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = hadoop01/192.168.161.128
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 3.2.0
STARTUP_MSG:   classpath = /usr/local/hadoop-3.2.0/etc/hadoop:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/accessors-smart-1.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/asm-5.0.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/avro-1.7.7.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-codec-1.11.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-io-2.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-lang3-3.7.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-net-3.6.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/commons-text-1.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/curator-client-2.12.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/curator-framework-2.12.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/curator-recipes-2.12.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/dnsjava-2.1.7.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/gson-2.2.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/guava-11.0.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/hadoop-annotations-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/hadoop-auth-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/httpclient-4.5.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/httpcore-4.4.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jackson-annotations-2.9.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jackson-core-2.9.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jackson-databind-2.9.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jersey-core-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jersey-json-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jersey-server-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jersey-servlet-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jsch-0.1.54.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/json-smart-2.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jsr305-3.0.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerb-client-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerb-common-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerb-core-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerb-server-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerb-util-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerby-config-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerby-util-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/netty-3.10.5.Final.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/re2j-1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/snappy-java-1.0.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/stax2-api-3.1.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/token-provider-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/xz-1.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/zookeeper-3.4.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/lib/metrics-core-3.2.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/hadoop-common-3.2.0-tests.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/hadoop-common-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/hadoop-nfs-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/common/hadoop-kms-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/okio-1.6.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/hadoop-auth-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/json-smart-2.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/asm-5.0.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/curator-framework-2.12.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/curator-client-2.12.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-io-2.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/hadoop-annotations-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-net-3.6.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jettison-1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-lang3-3.7.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-text-1.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/avro-1.7.7.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/paranamer-2.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/xz-1.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/gson-2.2.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/curator-recipes-2.12.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jackson-databind-2.9.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jackson-annotations-2.9.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/jackson-core-2.9.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-3.2.0-tests.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-client-3.2.0-tests.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-client-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.0-tests.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.0-tests.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.2.0-tests.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/fst-2.50.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/guice-4.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.9.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.9.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.9.5.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/java-util-1.9.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/jersey-client-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/json-io-2.5.1.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/objenesis-1.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-api-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-client-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-common-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-registry-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-server-common-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-server-router-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-services-api-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-services-core-3.2.0.jar:/usr/local/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-submarine-3.2.0.jar
STARTUP_MSG:   build = https://github.com/apache/hadoop.git -r e97acb3bd8f3befd27418996fa5d4b50bf2e17bf; compiled by 'sunilg' on 2019-01-08T06:08Z
STARTUP_MSG:   java = 1.8.0_211
************************************************************/
2019-06-28 22:34:38,824 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2019-06-28 22:34:38,943 INFO namenode.NameNode: createNameNode [-format]
Formatting using clusterid: CID-ce09df50-fd33-42ca-a5d7-c4ec876baee8
2019-06-28 22:34:39,757 INFO namenode.FSEditLog: Edit logging is async:true
2019-06-28 22:34:39,780 INFO namenode.FSNamesystem: KeyProvider: null
2019-06-28 22:34:39,780 INFO namenode.FSNamesystem: fsLock is fair: true
2019-06-28 22:34:39,781 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2019-06-28 22:34:39,785 INFO namenode.FSNamesystem: fsOwner             = hadoop (auth:SIMPLE)
2019-06-28 22:34:39,785 INFO namenode.FSNamesystem: supergroup          = supergroup
2019-06-28 22:34:39,785 INFO namenode.FSNamesystem: isPermissionEnabled = true
2019-06-28 22:34:39,786 INFO namenode.FSNamesystem: HA Enabled: false
2019-06-28 22:34:39,840 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2019-06-28 22:34:39,853 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2019-06-28 22:34:39,853 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
2019-06-28 22:34:39,869 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2019-06-28 22:34:39,870 INFO blockmanagement.BlockManager: The block deletion will start around 2019 Jun 28 22:34:39
2019-06-28 22:34:39,871 INFO util.GSet: Computing capacity for map BlocksMap
2019-06-28 22:34:39,871 INFO util.GSet: VM type       = 64-bit
2019-06-28 22:34:39,873 INFO util.GSet: 2.0% max memory 235.9 MB = 4.7 MB
2019-06-28 22:34:39,873 INFO util.GSet: capacity      = 2^19 = 524288 entries
2019-06-28 22:34:39,886 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled
2019-06-28 22:34:39,886 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
2019-06-28 22:34:39,899 INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS
2019-06-28 22:34:39,899 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
2019-06-28 22:34:39,899 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
2019-06-28 22:34:39,899 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2019-06-28 22:34:39,899 INFO blockmanagement.BlockManager: defaultReplication         = 2
2019-06-28 22:34:39,900 INFO blockmanagement.BlockManager: maxReplication             = 512
2019-06-28 22:34:39,900 INFO blockmanagement.BlockManager: minReplication             = 1
2019-06-28 22:34:39,900 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
2019-06-28 22:34:39,900 INFO blockmanagement.BlockManager: redundancyRecheckInterval  = 3000ms
2019-06-28 22:34:39,900 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
2019-06-28 22:34:39,900 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
2019-06-28 22:34:40,002 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911
2019-06-28 22:34:40,002 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215
2019-06-28 22:34:40,002 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215
2019-06-28 22:34:40,002 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215
2019-06-28 22:34:40,011 INFO util.GSet: Computing capacity for map INodeMap
2019-06-28 22:34:40,011 INFO util.GSet: VM type       = 64-bit
2019-06-28 22:34:40,011 INFO util.GSet: 1.0% max memory 235.9 MB = 2.4 MB
2019-06-28 22:34:40,011 INFO util.GSet: capacity      = 2^18 = 262144 entries
2019-06-28 22:34:40,013 INFO namenode.FSDirectory: ACLs enabled? false
2019-06-28 22:34:40,013 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
2019-06-28 22:34:40,013 INFO namenode.FSDirectory: XAttrs enabled? true
2019-06-28 22:34:40,013 INFO namenode.NameNode: Caching file names occurring more than 10 times
2019-06-28 22:34:40,017 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536
2019-06-28 22:34:40,018 INFO snapshot.SnapshotManager: SkipList is disabled
2019-06-28 22:34:40,021 INFO util.GSet: Computing capacity for map cachedBlocks
2019-06-28 22:34:40,021 INFO util.GSet: VM type       = 64-bit
2019-06-28 22:34:40,022 INFO util.GSet: 0.25% max memory 235.9 MB = 603.8 KB
2019-06-28 22:34:40,022 INFO util.GSet: capacity      = 2^16 = 65536 entries
2019-06-28 22:34:40,031 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2019-06-28 22:34:40,031 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2019-06-28 22:34:40,031 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2019-06-28 22:34:40,039 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
2019-06-28 22:34:40,039 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2019-06-28 22:34:40,041 INFO util.GSet: Computing capacity for map NameNodeRetryCache
2019-06-28 22:34:40,041 INFO util.GSet: VM type       = 64-bit
2019-06-28 22:34:40,042 INFO util.GSet: 0.029999999329447746% max memory 235.9 MB = 72.5 KB
2019-06-28 22:34:40,042 INFO util.GSet: capacity      = 2^13 = 8192 entries
Re-format filesystem in Storage Directory root= /home/hadoop/data/hadoop/dfs/name; location= null ? (Y or N) Y
2019-06-28 22:34:46,904 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1265790810-192.168.161.128-1561775686890
2019-06-28 22:34:46,904 INFO common.Storage: Will remove files: [/home/hadoop/data/hadoop/dfs/name/current/VERSION, /home/hadoop/data/hadoop/dfs/name/current/seen_txid, /home/hadoop/data/hadoop/dfs/name/current/fsimage_0000000000000000000.md5, /home/hadoop/data/hadoop/dfs/name/current/fsimage_0000000000000000000]
2019-06-28 22:34:47,017 INFO common.Storage: Storage directory /home/hadoop/data/hadoop/dfs/name has been successfully formatted.
2019-06-28 22:34:47,028 INFO namenode.FSImageFormatProtobuf: Saving image file /home/hadoop/data/hadoop/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
2019-06-28 22:34:47,213 INFO namenode.FSImageFormatProtobuf: Image file /home/hadoop/data/hadoop/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 401 bytes saved in 0 seconds .
2019-06-28 22:34:47,220 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
2019-06-28 22:34:47,225 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at hadoop01/192.168.161.128
************************************************************/