arrow_back Install Hadoop 3.2.1 on Windows 10 Step by Step Guide

comment Comments
O Orland Espiritu #1526 access_time 3 years ago more_vert

Hi.

What does this mean? here's the link to the full output https://www.dropbox.com/s/00rjsiyu8ezdf2w/yarn%20node%20manager.txt?dl=0

This is my output for the hive metastore ,  it showing warnings and no access to hiveserver2

https://www.dropbox.com/s/ec16lpp8d0tz1n9/--servicemetastoreoutput.txt?dl=0

2021-10-19 13:39:44,152 WARN nativeio.NativeIO: NativeIO.getStat error (3): The system cannot find the path specified.

 -- file path: tmp/hadoop-User/nm-local-dir/filecache

2021-10-19 13:39:44,219 WARN nativeio.NativeIO: NativeIO.getStat error (3): The system cannot find the path specified.

 -- file path: tmp/hadoop-User/nm-local-dir/usercache

2021-10-19 13:39:44,285 WARN nativeio.NativeIO: NativeIO.getStat error (3): The system cannot find the path specified.

 -- file path: tmp/hadoop-User/nm-local-dir/nmPrivate

Raymond Raymond #1497 access_time 3 years ago more_vert

Hi,

Can you please add more details so that I can help you?

For Hadoop build related questions, can you publish here:

Compile and Build Hadoop 3.2.1 on Windows 10 Guide - Hadoop Forum - Kontext

This article is about installing Hadoop with a pre-compiled binary package.


format_quote

person Antonio access_time 3 years ago

Hi, I'm using your guide with Hadoop 3.2.2 but when I compile the maven project with the command


mvn package  -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true



there is an error



Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (pre-dist) on project hadoop-project-dist: Command execution failed.:



any idea about how to solve it?

thanks

A Antonio Nappi #1496 access_time 3 years ago more_vert

Hi, I'm using your guide with Hadoop 3.2.2 but when I compile the maven project with the command


mvn package  -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true



there is an error



Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (pre-dist) on project hadoop-project-dist: Command execution failed.:



any idea about how to solve it?

thanks

Raymond Raymond #1492 access_time 3 years ago more_vert

You are welcome. Hope it helps. 

format_quote

person Mohamad access_time 3 years ago

Thank You 

Have a good Life

M Mohamad KonteratChi #1491 access_time 3 years ago more_vert

Thank You 

Have a good Life

Raymond Raymond #412 access_time 4 years ago more_vert

The error you got indicates that Hadoop required JAR libraries are not loaded properly.

The fix I provided was for missing the right winutil native libs not JAR libs. Make sure your JAVA_HOME, HADDOP_HOME environment variables and other configurations are done properly. If you are not following all the steps in my article, I would suggest you follow this article. 

format_quote

person Morar access_time 4 years ago

After I use the fix you mentioned namenode format still doesn't work

hdfs namenode -format

Error: Could not find or load main class org.apache.hadoop.hdfs.server.namenode.NameNode

M Morar Ciprian #411 access_time 4 years ago more_vert

After I use the fix you mentioned namenode format still doesn't work

hdfs namenode -format

Error: Could not find or load main class org.apache.hadoop.hdfs.server.namenode.NameNode

Raymond Raymond #347 access_time 4 years ago more_vert

Hi Ankit,

Yes, please refer to this series: https://kontext.tech/tag/big-data-on-windows-10.

It includes installation guide of Spark, Hive, Sqoop and Zeppelin on Windows. 

-Raymond

format_quote

person Ankit access_time 4 years ago

Hi Raymond, By any chance, have you also installed hive and other hadoop softwares, on a windows machine. If yes, would it be possible for you to share the installation guides? Regards, Ankit
A Ankit Tiwari #346 access_time 4 years ago more_vert
Hi Raymond, By any chance, have you also installed hive and other hadoop softwares, on a windows machine. If yes, would it be possible for you to share the installation guides? Regards, Ankit
A Ankit Tiwari #344 access_time 4 years ago more_vert

Hi Raymond,

Thanks for your reply :)

Looks like it may have happened due to mixing of different versions of Hadoop libraries. I started Hadoop installation by watching few YouTube videos (for older versions of Hadoop) where they later provided a modified version of bin folder. But my name-node was failing continuously. When I googled for the error message, I found your link and then followed the installation steps and could install Hadoop 3.2.1.

Thank you for taking out time to reply and clarify my confusion.

Regards,

Ankit

format_quote

person Raymond access_time 4 years ago

I'm glad it helped you. For the issue you mentioned, I don't think it is related to this tutorial. If you follow all the steps in this tutorial, you will not get that issue as this has been tested out by quite a few different people. I've also tested the steps in a new Windows 10 environment too. 

It seems you may have mixed different versions of Hadoop libraries when doing the installation.

This installation guide is only for Hadoop 3.2.1. Can you confirm whether you exactly followed all the steps in this guide and also was using Hadoop 3.2.1 release for installation?


Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts