Fix for Hadoop 3.2.1 namenode format issue on Windows 10

event 2020-01-25 visibility 9,054 comment 0 insights
more_vert
insights Stats
Fix for Hadoop 3.2.1 namenode format issue on Windows 10
Raymond Raymond Hadoop, Hive & HBase

Articles about Apache Hadoop, Hive and HBase installation, performance tuning and general tutorials.

*The yellow elephant logo is a registered trademark of Apache Hadoop.

Issue

When installing Hadoop 3.2.1 on Windows 10,  you may encounter the following error when trying to format HDFS  namnode:

ERROR namenode.NameNode: Failed to start namenode.

The error happens when running the following command in Command Prompt:

hdfs namenode -format
2020-01-18 13:36:03,021 ERROR namenode.NameNode: Failed to start namenode.
java.lang.UnsupportedOperationException
        at java.nio.file.Files.setPosixFilePermissions(Files.java:2044)
        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452)
        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591)
        at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1649)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1759)
2020-01-18 13:36:03,025 INFO util.ExitUtil: Exiting with status 1: java.lang.UnsupportedOperationException

Root cause

Refer to the following official issue tracker about the details of the root cause:

HDFS-14890

This issue will only be fixed on Hadoop 3.3.0 or 3.2.0 releases.

If you want to install 3.2.1, follow instructions below.

Resolution

I've provided detailed steps about installing Hadoop 3.2.1 on Windows 10  in the following article with fix about this issue:

Latest Hadoop 3.2.1 Installation on Windows 10 Step by Step Guide

Fix the issue only

I've uploaded the updated JAR file into the following location. Please download it from the following link:

https://github.com/FahaoTang/big-data/blob/master/hadoop-hdfs-3.2.1.jar  

And then rename the file name hadoop-hdfs-3.2.1.jar to hadoop-hdfs-3.2.1.bk in folder %HADOOP_HOME%\share\hadoop\hdfs.

Copy the downloaded hadoop-hdfs-3.2.1.jar to folder %HADOOP_HOME%\share\hadoop\hdfs.

warning This is just a temporary fix before the official fix is published. There is no guarantee this temporary fix won't cause any new issue. 

Refer to this article for more details about how to build a native Windows Hadoop: Compile and Build Hadoop 3.2.1 on Windows 10 Guide.

If you have any other questions, feel free to add a comment. 

More from Kontext
comment Comments
No comments yet.

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts