arrow_back Install Hadoop 3.3.0 on Windows 10 Step by Step Guide

comment Comments
Raymond Raymond #413 access_time 4 years ago more_vert

Have you downloaded the winutils native libraries (Step 3 of this article)? It seems is is not there or is not the correct version for your system.

Can you run the following commands:

winutils

It should print out the following text:

Usage: winutils [command] ...
Provide basic command line utilities for Hadoop on Windows.

The available commands and their usages are:

chmod          Change file mode bits.

Usage: chmod [OPTION] OCTAL-MODE [FILE]
   or: chmod [OPTION] MODE [FILE]
Change the mode of the FILE to MODE.

   -R: change files and directories recursively

Each MODE is of the form '[ugoa]*([-+=]([rwxX]*|[ugo]))+'.


chown          Change file owner.

Usage: chown [OWNER][:[GROUP]] [FILE]
Change the owner and/or group of the FILE to OWNER and/or GROUP.

Note:
On Linux, if a colon but no group name follows the user name, the group of
the files is changed to that user's login group. Windows has no concept of
a user's login group. So we do not change the group owner in this case.


groups         List user groups.
.....


format_quote

person Carl access_time 4 years ago

thanks for posting clear instructions, failed at starting the dfs

first line didn't look too encouraging!

DEPRECATED: Use of this script to execute hdfs command is deprecated.

Instead use the hdfs command for it.

2021-02-12 13:06:34,694 INFO namenode.NameNode: STARTUP_MSG:

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = DESKTOP-3UP7473/192.168.93.209

STARTUP_MSG:   args = []

STARTUP_MSG:   version = 3.3.0

STARTUP_MSG:   classpath = D:\big-data\hadoop-3.3.0\etc\hadoop;D:\big-data\hadoop-3.3.0\share\hadoop\common;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\accessors-smart-1.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\animal-sniffer-annotations-1.17.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\asm-5.0.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\audience-annotations-0.5.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\avro-1.7.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\checker-qual-2.5.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-beanutils-1.9.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-cli-1.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-codec-1.11.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-collections-3.2.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-compress-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-configuration2-2.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-daemon-1.0.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-io-2.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-lang3-3.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-logging-1.1.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-math3-3.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-net-3.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-text-1.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\curator-client-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\curator-framework-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\curator-recipes-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\dnsjava-2.1.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\failureaccess-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\gson-2.2.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\guava-27.0-jre.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\hadoop-annotations-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\hadoop-auth-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\hadoop-shaded-protobuf_3_7-1.0.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\htrace-core4-4.1.0-incubating.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\httpclient-4.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\httpcore-4.4.10.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\j2objc-annotations-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-annotations-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-core-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-core-asl-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-databind-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-jaxrs-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-mapper-asl-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-xc-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\javax.activation-api-1.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\javax.servlet-api-3.1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jaxb-api-2.2.11.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jaxb-impl-2.2.3-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jcip-annotations-1.0-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jersey-core-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jersey-json-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jersey-server-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jersey-servlet-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jettison-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-http-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-io-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-security-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-server-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-servlet-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-util-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-webapp-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-xml-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jsch-0.1.55.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\json-smart-2.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jsp-api-2.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jsr305-3.0.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jsr311-api-1.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jul-to-slf4j-1.7.25.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-admin-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-client-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-common-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-core-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-crypto-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-identity-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-server-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-simplekdc-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-util-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-asn1-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-config-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-pkix-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-util-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-xdr-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\log4j-1.2.17.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\metrics-core-3.2.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\netty-3.10.6.Final.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\nimbus-jose-jwt-7.9.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\paranamer-2.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\protobuf-java-2.5.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\re2j-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\slf4j-api-1.7.25.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\slf4j-log4j12-1.7.25.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\snappy-java-1.0.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\stax2-api-3.1.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\token-provider-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\woodstox-core-5.0.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\zookeeper-3.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\zookeeper-jute-3.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-common-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-common-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-kms-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-nfs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-registry-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\accessors-smart-1.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\animal-sniffer-annotations-1.17.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\asm-5.0.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\audience-annotations-0.5.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\avro-1.7.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\checker-qual-2.5.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-beanutils-1.9.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-cli-1.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-codec-1.11.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-collections-3.2.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-compress-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-configuration2-2.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-daemon-1.0.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-io-2.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-lang3-3.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-logging-1.1.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-math3-3.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-net-3.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-text-1.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\curator-client-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\curator-framework-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\curator-recipes-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\dnsjava-2.1.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\failureaccess-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\gson-2.2.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\guava-27.0-jre.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\hadoop-annotations-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\hadoop-auth-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\hadoop-shaded-protobuf_3_7-1.0.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\htrace-core4-4.1.0-incubating.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\httpclient-4.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\httpcore-4.4.10.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\j2objc-annotations-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-annotations-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-core-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-core-asl-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-databind-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-jaxrs-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-mapper-asl-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-xc-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\javax.activation-api-1.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\javax.servlet-api-3.1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jaxb-api-2.2.11.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jaxb-impl-2.2.3-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jcip-annotations-1.0-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jersey-core-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jersey-json-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jersey-server-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jersey-servlet-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jettison-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-http-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-io-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-security-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-server-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-servlet-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-util-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-util-ajax-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-webapp-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-xml-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jsch-0.1.55.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\json-simple-1.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\json-smart-2.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jsr305-3.0.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jsr311-api-1.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-admin-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-client-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-common-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-core-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-crypto-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-identity-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-server-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-simplekdc-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-util-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-asn1-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-config-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-pkix-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-util-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-xdr-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\leveldbjni-all-1.8.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\log4j-1.2.17.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\netty-3.10.6.Final.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\netty-all-4.1.50.Final.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\nimbus-jose-jwt-7.9.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\okhttp-2.7.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\okio-1.6.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\paranamer-2.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\protobuf-java-2.5.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\re2j-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\snappy-java-1.0.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\stax2-api-3.1.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\token-provider-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\woodstox-core-5.0.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\zookeeper-3.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\zookeeper-jute-3.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-client-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-client-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-httpfs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-native-client-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-native-client-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-nfs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-rbf-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-rbf-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\aopalliance-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\asm-analysis-7.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\asm-commons-7.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\asm-tree-7.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\bcpkix-jdk15on-1.60.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\bcprov-jdk15on-1.60.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\ehcache-3.3.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\fst-2.50.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\geronimo-jcache_1.0_spec-1.0-alpha-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\guice-4.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\guice-servlet-4.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\HikariCP-java7-2.4.12.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jackson-jaxrs-base-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jackson-jaxrs-json-provider-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jackson-module-jaxb-annotations-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jakarta.activation-api-1.2.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jakarta.xml.bind-api-2.3.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\java-util-1.9.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax-websocket-client-impl-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax-websocket-server-impl-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax.inject-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax.websocket-api-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax.websocket-client-api-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jersey-client-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jersey-guice-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jetty-annotations-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jetty-client-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jetty-jndi-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jetty-plus-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jline-3.9.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jna-5.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\json-io-2.5.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\metrics-core-3.2.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\mssql-jdbc-6.2.1.jre7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\objenesis-2.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\snakeyaml-1.16.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\swagger-annotations-1.5.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-api-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-client-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-common-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-server-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-servlet-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-api-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-applications-distributedshell-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-applications-mawo-core-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-applications-unmanaged-am-launcher-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-client-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-common-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-registry-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-applicationhistoryservice-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-common-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-nodemanager-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-resourcemanager-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-router-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-sharedcachemanager-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-tests-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-timeline-pluginstorage-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-web-proxy-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-services-api-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-services-core-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-app-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-common-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-core-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-hs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-hs-plugins-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-jobclient-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-jobclient-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-nativetask-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-shuffle-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-uploader-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-examples-3.3.0.jar

STARTUP_MSG:   build = https://gitbox.apache.org/repos/asf/hadoop.git -r aa96f1871bfd858f9bac59cf2a81ec470da649af; compiled by 'brahma' on 2020-07-06T18:44Z

STARTUP_MSG:   java = 1.8.0_281

************************************************************/

2021-02-12 13:06:34,768 INFO namenode.NameNode: createNameNode []

2021-02-12 13:06:34,860 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties

2021-02-12 13:06:34,963 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).

2021-02-12 13:06:34,963 INFO impl.MetricsSystemImpl: NameNode metrics system started

2021-02-12 13:06:35,138 INFO namenode.NameNodeUtils: fs.defaultFS is hdfs://0.0.0.0:19000

2021-02-12 13:06:35,139 INFO namenode.NameNode: Clients should use 0.0.0.0:19000 to access this namenode/service.

2021-02-12 13:06:35,162 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

2021-02-12 13:06:35,242 INFO util.JvmPauseMonitor: Starting JVM pause monitor

2021-02-12 13:06:35,257 INFO hdfs.DFSUtil: Filter initializers set : org.apache.hadoop.http.lib.StaticUserWebFilter,org.apache.hadoop.hdfs.web.AuthFilterInitializer

2021-02-12 13:06:35,261 INFO hdfs.DFSUtil: Starting Web-server for hdfs at: http://0.0.0.0:9870

2021-02-12 13:06:35,269 INFO util.log: Logging initialized @887ms to org.eclipse.jetty.util.log.Slf4jLog

2021-02-12 13:06:35,334 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.

2021-02-12 13:06:35,456 INFO http.HttpRequestLog: Http request log for http.requests.namenode is not defined

2021-02-12 13:06:35,463 INFO http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)

2021-02-12 13:06:35,464 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs

2021-02-12 13:06:35,464 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs

2021-02-12 13:06:35,465 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static

2021-02-12 13:06:35,467 INFO http.HttpServer2: Added filter AuthFilter (class=org.apache.hadoop.hdfs.web.AuthFilter) to context hdfs

2021-02-12 13:06:35,467 INFO http.HttpServer2: Added filter AuthFilter (class=org.apache.hadoop.hdfs.web.AuthFilter) to context logs

2021-02-12 13:06:35,467 INFO http.HttpServer2: Added filter AuthFilter (class=org.apache.hadoop.hdfs.web.AuthFilter) to context static

2021-02-12 13:06:35,485 INFO http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*

2021-02-12 13:06:35,491 INFO http.HttpServer2: Jetty bound to port 9870

2021-02-12 13:06:35,492 INFO server.Server: jetty-9.4.20.v20190813; built: 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm 1.8.0_281-b09

2021-02-12 13:06:35,510 INFO server.session: DefaultSessionIdManager workerName=node0

2021-02-12 13:06:35,510 INFO server.session: No SessionScavenger set, using defaults

2021-02-12 13:06:35,511 INFO server.session: node0 Scavenging every 600000ms

2021-02-12 13:06:35,520 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.

2021-02-12 13:06:35,522 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@4d1c005e{logs,/logs,file:///D:/big-data/hadoop-3.3.0/logs/,AVAILABLE}

2021-02-12 13:06:35,522 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@59402b8f{static,/static,file:///D:/big-data/hadoop-3.3.0/share/hadoop/hdfs/webapps/static/,AVAILABLE}

2021-02-12 13:06:35,564 INFO util.TypeUtil: JVM Runtime does not support Modules

2021-02-12 13:06:35,570 INFO handler.ContextHandler: Started o.e.j.w.WebAppContext@1130520d{hdfs,/,file:///D:/big-data/hadoop-3.3.0/share/hadoop/hdfs/webapps/hdfs/,AVAILABLE}{file:/D:/big-data/hadoop-3.3.0/share/hadoop/hdfs/webapps/hdfs}

2021-02-12 13:06:35,577 INFO server.AbstractConnector: Started ServerConnector@d554c5f{HTTP/1.1,[http/1.1]}{0.0.0.0:9870}

2021-02-12 13:06:35,577 INFO server.Server: Started @1196ms

2021-02-12 13:06:35,872 WARN namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!

2021-02-12 13:06:35,872 WARN namenode.FSNamesystem: Only one namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of data loss due to lack of redundant storage directories!

2021-02-12 13:06:35,903 INFO namenode.FSEditLog: Edit logging is async:true

2021-02-12 13:06:35,921 INFO namenode.FSNamesystem: KeyProvider: null

2021-02-12 13:06:35,922 INFO namenode.FSNamesystem: fsLock is fair: true

2021-02-12 13:06:35,922 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false

2021-02-12 13:06:35,927 INFO namenode.FSNamesystem: fsOwner                = carlr (auth:SIMPLE)

2021-02-12 13:06:35,927 INFO namenode.FSNamesystem: supergroup             = supergroup

2021-02-12 13:06:35,927 INFO namenode.FSNamesystem: isPermissionEnabled    = true

2021-02-12 13:06:35,927 INFO namenode.FSNamesystem: isStoragePolicyEnabled = true

2021-02-12 13:06:35,928 INFO namenode.FSNamesystem: HA Enabled: false

2021-02-12 13:06:35,952 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling

2021-02-12 13:06:35,957 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000

2021-02-12 13:06:35,957 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true

2021-02-12 13:06:35,960 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000

2021-02-12 13:06:35,960 INFO blockmanagement.BlockManager: The block deletion will start around 2021 Feb 12 13:06:35

2021-02-12 13:06:35,961 INFO util.GSet: Computing capacity for map BlocksMap

2021-02-12 13:06:35,961 INFO util.GSet: VM type       = 64-bit

2021-02-12 13:06:35,963 INFO util.GSet: 2.0% max memory 889 MB = 17.8 MB

2021-02-12 13:06:35,963 INFO util.GSet: capacity      = 2^21 = 2097152 entries

2021-02-12 13:06:35,968 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled

2021-02-12 13:06:35,969 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false

2021-02-12 13:06:35,973 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.999

2021-02-12 13:06:35,973 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0

2021-02-12 13:06:35,974 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000

2021-02-12 13:06:35,975 INFO blockmanagement.BlockManager: defaultReplication         = 1

2021-02-12 13:06:35,975 INFO blockmanagement.BlockManager: maxReplication             = 512

2021-02-12 13:06:35,975 INFO blockmanagement.BlockManager: minReplication             = 1

2021-02-12 13:06:35,976 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2

2021-02-12 13:06:35,976 INFO blockmanagement.BlockManager: redundancyRecheckInterval  = 3000ms

2021-02-12 13:06:35,976 INFO blockmanagement.BlockManager: encryptDataTransfer        = false

2021-02-12 13:06:35,977 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000

2021-02-12 13:06:35,995 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911

2021-02-12 13:06:35,995 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215

2021-02-12 13:06:35,996 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215

2021-02-12 13:06:35,996 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215

2021-02-12 13:06:36,006 INFO util.GSet: Computing capacity for map INodeMap

2021-02-12 13:06:36,006 INFO util.GSet: VM type       = 64-bit

2021-02-12 13:06:36,007 INFO util.GSet: 1.0% max memory 889 MB = 8.9 MB

2021-02-12 13:06:36,007 INFO util.GSet: capacity      = 2^20 = 1048576 entries

2021-02-12 13:06:36,007 INFO namenode.FSDirectory: ACLs enabled? true

2021-02-12 13:06:36,008 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true

2021-02-12 13:06:36,008 INFO namenode.FSDirectory: XAttrs enabled? true

2021-02-12 13:06:36,008 INFO namenode.NameNode: Caching file names occurring more than 10 times

2021-02-12 13:06:36,012 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536

2021-02-12 13:06:36,013 INFO snapshot.SnapshotManager: SkipList is disabled

2021-02-12 13:06:36,016 INFO util.GSet: Computing capacity for map cachedBlocks

2021-02-12 13:06:36,016 INFO util.GSet: VM type       = 64-bit

2021-02-12 13:06:36,017 INFO util.GSet: 0.25% max memory 889 MB = 2.2 MB

2021-02-12 13:06:36,017 INFO util.GSet: capacity      = 2^18 = 262144 entries

2021-02-12 13:06:36,039 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10

2021-02-12 13:06:36,039 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10

2021-02-12 13:06:36,040 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25

2021-02-12 13:06:36,042 INFO namenode.FSNamesystem: Retry cache on namenode is enabled

2021-02-12 13:06:36,042 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis

2021-02-12 13:06:36,044 INFO util.GSet: Computing capacity for map NameNodeRetryCache

2021-02-12 13:06:36,044 INFO util.GSet: VM type       = 64-bit

2021-02-12 13:06:36,044 INFO util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB

2021-02-12 13:06:36,045 INFO util.GSet: capacity      = 2^15 = 32768 entries

2021-02-12 13:06:36,055 ERROR namenode.NameNode: Failed to start namenode.

java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z

        at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)

        at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:793)

        at org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:1240)

        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:690)

        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:642)

        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:386)

        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:242)

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1197)

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:779)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:673)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:760)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1014)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:987)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1756)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1821)

2021-02-12 13:06:36,057 INFO util.ExitUtil: Exiting with status 1: java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z

2021-02-12 13:06:36,059 INFO namenode.NameNode: SHUTDOWN_MSG:

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at DESKTOP-3UP7473/192.168.93.209

************************************************************/


C:\Windows\system32>

C Carl Rees #410 access_time 4 years ago more_vert

thanks for posting clear instructions, failed at starting the dfs

first line didn't look too encouraging!

DEPRECATED: Use of this script to execute hdfs command is deprecated.

Instead use the hdfs command for it.

2021-02-12 13:06:34,694 INFO namenode.NameNode: STARTUP_MSG:

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = DESKTOP-3UP7473/192.168.93.209

STARTUP_MSG:   args = []

STARTUP_MSG:   version = 3.3.0

STARTUP_MSG:   classpath = D:\big-data\hadoop-3.3.0\etc\hadoop;D:\big-data\hadoop-3.3.0\share\hadoop\common;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\accessors-smart-1.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\animal-sniffer-annotations-1.17.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\asm-5.0.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\audience-annotations-0.5.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\avro-1.7.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\checker-qual-2.5.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-beanutils-1.9.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-cli-1.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-codec-1.11.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-collections-3.2.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-compress-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-configuration2-2.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-daemon-1.0.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-io-2.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-lang3-3.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-logging-1.1.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-math3-3.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-net-3.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\commons-text-1.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\curator-client-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\curator-framework-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\curator-recipes-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\dnsjava-2.1.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\failureaccess-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\gson-2.2.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\guava-27.0-jre.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\hadoop-annotations-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\hadoop-auth-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\hadoop-shaded-protobuf_3_7-1.0.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\htrace-core4-4.1.0-incubating.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\httpclient-4.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\httpcore-4.4.10.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\j2objc-annotations-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-annotations-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-core-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-core-asl-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-databind-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-jaxrs-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-mapper-asl-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jackson-xc-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\javax.activation-api-1.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\javax.servlet-api-3.1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jaxb-api-2.2.11.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jaxb-impl-2.2.3-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jcip-annotations-1.0-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jersey-core-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jersey-json-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jersey-server-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jersey-servlet-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jettison-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-http-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-io-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-security-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-server-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-servlet-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-util-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-webapp-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jetty-xml-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jsch-0.1.55.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\json-smart-2.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jsp-api-2.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jsr305-3.0.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jsr311-api-1.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\jul-to-slf4j-1.7.25.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-admin-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-client-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-common-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-core-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-crypto-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-identity-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-server-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-simplekdc-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerb-util-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-asn1-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-config-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-pkix-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-util-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\kerby-xdr-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\log4j-1.2.17.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\metrics-core-3.2.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\netty-3.10.6.Final.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\nimbus-jose-jwt-7.9.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\paranamer-2.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\protobuf-java-2.5.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\re2j-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\slf4j-api-1.7.25.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\slf4j-log4j12-1.7.25.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\snappy-java-1.0.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\stax2-api-3.1.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\token-provider-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\woodstox-core-5.0.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\zookeeper-3.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\lib\zookeeper-jute-3.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-common-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-common-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-kms-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-nfs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\common\hadoop-registry-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\accessors-smart-1.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\animal-sniffer-annotations-1.17.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\asm-5.0.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\audience-annotations-0.5.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\avro-1.7.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\checker-qual-2.5.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-beanutils-1.9.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-cli-1.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-codec-1.11.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-collections-3.2.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-compress-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-configuration2-2.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-daemon-1.0.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-io-2.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-lang3-3.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-logging-1.1.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-math3-3.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-net-3.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\commons-text-1.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\curator-client-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\curator-framework-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\curator-recipes-4.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\dnsjava-2.1.7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\failureaccess-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\gson-2.2.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\guava-27.0-jre.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\hadoop-annotations-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\hadoop-auth-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\hadoop-shaded-protobuf_3_7-1.0.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\htrace-core4-4.1.0-incubating.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\httpclient-4.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\httpcore-4.4.10.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\j2objc-annotations-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-annotations-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-core-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-core-asl-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-databind-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-jaxrs-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-mapper-asl-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jackson-xc-1.9.13.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\javax.activation-api-1.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\javax.servlet-api-3.1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jaxb-api-2.2.11.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jaxb-impl-2.2.3-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jcip-annotations-1.0-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jersey-core-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jersey-json-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jersey-server-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jersey-servlet-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jettison-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-http-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-io-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-security-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-server-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-servlet-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-util-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-util-ajax-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-webapp-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jetty-xml-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jsch-0.1.55.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\json-simple-1.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\json-smart-2.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jsr305-3.0.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\jsr311-api-1.1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-admin-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-client-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-common-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-core-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-crypto-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-identity-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-server-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-simplekdc-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerb-util-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-asn1-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-config-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-pkix-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-util-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\kerby-xdr-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\leveldbjni-all-1.8.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\log4j-1.2.17.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\netty-3.10.6.Final.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\netty-all-4.1.50.Final.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\nimbus-jose-jwt-7.9.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\okhttp-2.7.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\okio-1.6.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\paranamer-2.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\protobuf-java-2.5.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\re2j-1.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\snappy-java-1.0.5.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\stax2-api-3.1.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\token-provider-1.0.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\woodstox-core-5.0.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\zookeeper-3.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\lib\zookeeper-jute-3.5.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-client-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-client-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-httpfs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-native-client-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-native-client-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-nfs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-rbf-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\hdfs\hadoop-hdfs-rbf-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\aopalliance-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\asm-analysis-7.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\asm-commons-7.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\asm-tree-7.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\bcpkix-jdk15on-1.60.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\bcprov-jdk15on-1.60.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\ehcache-3.3.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\fst-2.50.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\geronimo-jcache_1.0_spec-1.0-alpha-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\guice-4.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\guice-servlet-4.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\HikariCP-java7-2.4.12.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jackson-jaxrs-base-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jackson-jaxrs-json-provider-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jackson-module-jaxb-annotations-2.10.3.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jakarta.activation-api-1.2.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jakarta.xml.bind-api-2.3.2.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\java-util-1.9.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax-websocket-client-impl-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax-websocket-server-impl-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax.inject-1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax.websocket-api-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\javax.websocket-client-api-1.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jersey-client-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jersey-guice-1.19.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jetty-annotations-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jetty-client-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jetty-jndi-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jetty-plus-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jline-3.9.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\jna-5.2.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\json-io-2.5.1.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\metrics-core-3.2.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\mssql-jdbc-6.2.1.jre7.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\objenesis-2.6.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\snakeyaml-1.16.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\swagger-annotations-1.5.4.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-api-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-client-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-common-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-server-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\lib\websocket-servlet-9.4.20.v20190813.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-api-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-applications-distributedshell-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-applications-mawo-core-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-applications-unmanaged-am-launcher-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-client-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-common-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-registry-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-applicationhistoryservice-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-common-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-nodemanager-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-resourcemanager-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-router-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-sharedcachemanager-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-tests-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-timeline-pluginstorage-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-server-web-proxy-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-services-api-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\yarn\hadoop-yarn-services-core-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-app-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-common-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-core-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-hs-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-hs-plugins-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-jobclient-3.3.0-tests.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-jobclient-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-nativetask-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-shuffle-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-client-uploader-3.3.0.jar;D:\big-data\hadoop-3.3.0\share\hadoop\mapreduce\hadoop-mapreduce-examples-3.3.0.jar

STARTUP_MSG:   build = https://gitbox.apache.org/repos/asf/hadoop.git -r aa96f1871bfd858f9bac59cf2a81ec470da649af; compiled by 'brahma' on 2020-07-06T18:44Z

STARTUP_MSG:   java = 1.8.0_281

************************************************************/

2021-02-12 13:06:34,768 INFO namenode.NameNode: createNameNode []

2021-02-12 13:06:34,860 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties

2021-02-12 13:06:34,963 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).

2021-02-12 13:06:34,963 INFO impl.MetricsSystemImpl: NameNode metrics system started

2021-02-12 13:06:35,138 INFO namenode.NameNodeUtils: fs.defaultFS is hdfs://0.0.0.0:19000

2021-02-12 13:06:35,139 INFO namenode.NameNode: Clients should use 0.0.0.0:19000 to access this namenode/service.

2021-02-12 13:06:35,162 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

2021-02-12 13:06:35,242 INFO util.JvmPauseMonitor: Starting JVM pause monitor

2021-02-12 13:06:35,257 INFO hdfs.DFSUtil: Filter initializers set : org.apache.hadoop.http.lib.StaticUserWebFilter,org.apache.hadoop.hdfs.web.AuthFilterInitializer

2021-02-12 13:06:35,261 INFO hdfs.DFSUtil: Starting Web-server for hdfs at: http://0.0.0.0:9870

2021-02-12 13:06:35,269 INFO util.log: Logging initialized @887ms to org.eclipse.jetty.util.log.Slf4jLog

2021-02-12 13:06:35,334 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.

2021-02-12 13:06:35,456 INFO http.HttpRequestLog: Http request log for http.requests.namenode is not defined

2021-02-12 13:06:35,463 INFO http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)

2021-02-12 13:06:35,464 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs

2021-02-12 13:06:35,464 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs

2021-02-12 13:06:35,465 INFO http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static

2021-02-12 13:06:35,467 INFO http.HttpServer2: Added filter AuthFilter (class=org.apache.hadoop.hdfs.web.AuthFilter) to context hdfs

2021-02-12 13:06:35,467 INFO http.HttpServer2: Added filter AuthFilter (class=org.apache.hadoop.hdfs.web.AuthFilter) to context logs

2021-02-12 13:06:35,467 INFO http.HttpServer2: Added filter AuthFilter (class=org.apache.hadoop.hdfs.web.AuthFilter) to context static

2021-02-12 13:06:35,485 INFO http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*

2021-02-12 13:06:35,491 INFO http.HttpServer2: Jetty bound to port 9870

2021-02-12 13:06:35,492 INFO server.Server: jetty-9.4.20.v20190813; built: 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm 1.8.0_281-b09

2021-02-12 13:06:35,510 INFO server.session: DefaultSessionIdManager workerName=node0

2021-02-12 13:06:35,510 INFO server.session: No SessionScavenger set, using defaults

2021-02-12 13:06:35,511 INFO server.session: node0 Scavenging every 600000ms

2021-02-12 13:06:35,520 INFO server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.

2021-02-12 13:06:35,522 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@4d1c005e{logs,/logs,file:///D:/big-data/hadoop-3.3.0/logs/,AVAILABLE}

2021-02-12 13:06:35,522 INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@59402b8f{static,/static,file:///D:/big-data/hadoop-3.3.0/share/hadoop/hdfs/webapps/static/,AVAILABLE}

2021-02-12 13:06:35,564 INFO util.TypeUtil: JVM Runtime does not support Modules

2021-02-12 13:06:35,570 INFO handler.ContextHandler: Started o.e.j.w.WebAppContext@1130520d{hdfs,/,file:///D:/big-data/hadoop-3.3.0/share/hadoop/hdfs/webapps/hdfs/,AVAILABLE}{file:/D:/big-data/hadoop-3.3.0/share/hadoop/hdfs/webapps/hdfs}

2021-02-12 13:06:35,577 INFO server.AbstractConnector: Started ServerConnector@d554c5f{HTTP/1.1,[http/1.1]}{0.0.0.0:9870}

2021-02-12 13:06:35,577 INFO server.Server: Started @1196ms

2021-02-12 13:06:35,872 WARN namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!

2021-02-12 13:06:35,872 WARN namenode.FSNamesystem: Only one namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of data loss due to lack of redundant storage directories!

2021-02-12 13:06:35,903 INFO namenode.FSEditLog: Edit logging is async:true

2021-02-12 13:06:35,921 INFO namenode.FSNamesystem: KeyProvider: null

2021-02-12 13:06:35,922 INFO namenode.FSNamesystem: fsLock is fair: true

2021-02-12 13:06:35,922 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false

2021-02-12 13:06:35,927 INFO namenode.FSNamesystem: fsOwner                = carlr (auth:SIMPLE)

2021-02-12 13:06:35,927 INFO namenode.FSNamesystem: supergroup             = supergroup

2021-02-12 13:06:35,927 INFO namenode.FSNamesystem: isPermissionEnabled    = true

2021-02-12 13:06:35,927 INFO namenode.FSNamesystem: isStoragePolicyEnabled = true

2021-02-12 13:06:35,928 INFO namenode.FSNamesystem: HA Enabled: false

2021-02-12 13:06:35,952 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling

2021-02-12 13:06:35,957 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000

2021-02-12 13:06:35,957 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true

2021-02-12 13:06:35,960 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000

2021-02-12 13:06:35,960 INFO blockmanagement.BlockManager: The block deletion will start around 2021 Feb 12 13:06:35

2021-02-12 13:06:35,961 INFO util.GSet: Computing capacity for map BlocksMap

2021-02-12 13:06:35,961 INFO util.GSet: VM type       = 64-bit

2021-02-12 13:06:35,963 INFO util.GSet: 2.0% max memory 889 MB = 17.8 MB

2021-02-12 13:06:35,963 INFO util.GSet: capacity      = 2^21 = 2097152 entries

2021-02-12 13:06:35,968 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled

2021-02-12 13:06:35,969 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false

2021-02-12 13:06:35,973 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.999

2021-02-12 13:06:35,973 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0

2021-02-12 13:06:35,974 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000

2021-02-12 13:06:35,975 INFO blockmanagement.BlockManager: defaultReplication         = 1

2021-02-12 13:06:35,975 INFO blockmanagement.BlockManager: maxReplication             = 512

2021-02-12 13:06:35,975 INFO blockmanagement.BlockManager: minReplication             = 1

2021-02-12 13:06:35,976 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2

2021-02-12 13:06:35,976 INFO blockmanagement.BlockManager: redundancyRecheckInterval  = 3000ms

2021-02-12 13:06:35,976 INFO blockmanagement.BlockManager: encryptDataTransfer        = false

2021-02-12 13:06:35,977 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000

2021-02-12 13:06:35,995 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911

2021-02-12 13:06:35,995 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215

2021-02-12 13:06:35,996 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215

2021-02-12 13:06:35,996 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215

2021-02-12 13:06:36,006 INFO util.GSet: Computing capacity for map INodeMap

2021-02-12 13:06:36,006 INFO util.GSet: VM type       = 64-bit

2021-02-12 13:06:36,007 INFO util.GSet: 1.0% max memory 889 MB = 8.9 MB

2021-02-12 13:06:36,007 INFO util.GSet: capacity      = 2^20 = 1048576 entries

2021-02-12 13:06:36,007 INFO namenode.FSDirectory: ACLs enabled? true

2021-02-12 13:06:36,008 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true

2021-02-12 13:06:36,008 INFO namenode.FSDirectory: XAttrs enabled? true

2021-02-12 13:06:36,008 INFO namenode.NameNode: Caching file names occurring more than 10 times

2021-02-12 13:06:36,012 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536

2021-02-12 13:06:36,013 INFO snapshot.SnapshotManager: SkipList is disabled

2021-02-12 13:06:36,016 INFO util.GSet: Computing capacity for map cachedBlocks

2021-02-12 13:06:36,016 INFO util.GSet: VM type       = 64-bit

2021-02-12 13:06:36,017 INFO util.GSet: 0.25% max memory 889 MB = 2.2 MB

2021-02-12 13:06:36,017 INFO util.GSet: capacity      = 2^18 = 262144 entries

2021-02-12 13:06:36,039 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10

2021-02-12 13:06:36,039 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10

2021-02-12 13:06:36,040 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25

2021-02-12 13:06:36,042 INFO namenode.FSNamesystem: Retry cache on namenode is enabled

2021-02-12 13:06:36,042 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis

2021-02-12 13:06:36,044 INFO util.GSet: Computing capacity for map NameNodeRetryCache

2021-02-12 13:06:36,044 INFO util.GSet: VM type       = 64-bit

2021-02-12 13:06:36,044 INFO util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB

2021-02-12 13:06:36,045 INFO util.GSet: capacity      = 2^15 = 32768 entries

2021-02-12 13:06:36,055 ERROR namenode.NameNode: Failed to start namenode.

java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z

        at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)

        at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:793)

        at org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:1240)

        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:690)

        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:642)

        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:386)

        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:242)

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1197)

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:779)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:673)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:760)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1014)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:987)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1756)

        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1821)

2021-02-12 13:06:36,057 INFO util.ExitUtil: Exiting with status 1: java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z

2021-02-12 13:06:36,059 INFO namenode.NameNode: SHUTDOWN_MSG:

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at DESKTOP-3UP7473/192.168.93.209

************************************************************/


C:\Windows\system32>

Raymond Raymond #361 access_time 4 years ago more_vert

Hi Cosmo,

Did you finish the installation successfully?

PS download should work if you follow the steps exactly. It may not work if there is network proxy, etc. For that scenario, please directly from GitHub.

format_quote

person cosmo access_time 4 years ago

Hi all, thanks very much for this guide!!

just for your information:

1) install jdk under folder without space (e.g. c:\java\jdk1.8.0_271) otherwise it doesn't work

2) under path command use \bin (/bin doesn't work)

3) download winutils from github, don't use PS script (it doesn't work)

4) Yarn nodemanager return an error: 

2020-11-10 17:02:12,840 ERROR util.SysInfoWindows: ExitCodeException exitCode=1: PdhAddCounter \Network Interface(*)\Bytes Received/Sec failed with 0xc0000bb8.
Error in GetDiskAndNetwork. Err:1
       at org.apache.hadoop.util.Shell.run(Shell.java:901)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213)
        at org.apache.hadoop.util.SysInfoWindows.getSystemInfoInfoFromShell(SysInfoWindows.java:86)
        at org.apache.hadoop.util.SysInfoWindows.refreshIfNeeded(SysInfoWindows.java:101)
        at org.apache.hadoop.util.SysInfoWindows.getNumVCoresUsed(SysInfoWindows.java:212)
        at org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.getNumVCoresUsed(ResourceCalculatorPlugin.java:138)
        at org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl$MonitoringThread.run(NodeResourceMonitorImpl.java:148)
       at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)


C cosmo capodiferro #360 access_time 4 years ago more_vert

Hi all, thanks very much for this guide!!

just for your information:

1) install jdk under folder without space (e.g. c:\java\jdk1.8.0_271) otherwise it doesn't work

2) under path command use \bin (/bin doesn't work)

3) download winutils from github, don't use PS script (it doesn't work)

4) Yarn nodemanager return an error: 

2020-11-10 17:02:12,840 ERROR util.SysInfoWindows: ExitCodeException exitCode=1: PdhAddCounter \Network Interface(*)\Bytes Received/Sec failed with 0xc0000bb8.
Error in GetDiskAndNetwork. Err:1
       at org.apache.hadoop.util.Shell.run(Shell.java:901)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213)
        at org.apache.hadoop.util.SysInfoWindows.getSystemInfoInfoFromShell(SysInfoWindows.java:86)
        at org.apache.hadoop.util.SysInfoWindows.refreshIfNeeded(SysInfoWindows.java:101)
        at org.apache.hadoop.util.SysInfoWindows.getNumVCoresUsed(SysInfoWindows.java:212)
        at org.apache.hadoop.yarn.util.ResourceCalculatorPlugin.getNumVCoresUsed(ResourceCalculatorPlugin.java:138)
        at org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorImpl$MonitoringThread.run(NodeResourceMonitorImpl.java:148)
       at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)


Raymond Raymond #350 access_time 4 years ago more_vert

Did you follow this installation guide to install Hadoop 3.3.0 on Windows 10? I would suggest you follow this guide to avoid any unexpected errors. This guide was tested in brand new Windows 10 environment.

From the error message, it looks like winutils.exe program is not compatible with Windows. Can you run winutils.exe directly in Command Prompt or PowerShell?

Are your Windows 32bit? and is your winutils.exe 64 bit?


format_quote

person carla access_time 4 years ago

I am new to hadoop. I try the hadoop3.3.0 and hadoop3.2.1 and use the winutils.The version of winutils is correct. But when I run a simple spark application , I get the error --->Cannot run program "Hadoop\hadoop-3.2.1\bin\winutils.exe. CreateProcess error=216, This version of %1 is not compatible with Windows.

Do you know how to solve it? thank you!

C carla feng #349 access_time 4 years ago more_vert

I am new to hadoop. I try the hadoop3.3.0 and hadoop3.2.1 and use the winutils.The version of winutils is correct. But when I run a simple spark application , I get the error --->Cannot run program "Hadoop\hadoop-3.2.1\bin\winutils.exe. CreateProcess error=216, This version of %1 is not compatible with Windows.

Do you know how to solve it? thank you!

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts