Hadoop 3.3.1 winutils

Raymond Raymond event 2021-09-27 visibility 32,621 comment 5
more_vert

Winutils is required when installing Hadoop on Windows environment.

Hadoop 3.3.1 winutils

I've compiled Hadoop 3.3.1 on Windows 10 using CMake and Visual Studio (MSVC x64).

Follow these two steps to download it:

  1. https://github.com/kontext-tech/winutils
  2. Go to subfolder hadoop-3.3.1/bin to download the binaries. 
warning These binaries are provided for testing and learning purposes and no guarantees are provided. Please don't use it on production environments. 

Verify the tool

You can run the winutils.exe program in your Windows PC to make sure it works:

20210927122931-image.png

Feedback

If you encounter any issues, please comment here.

More from Kontext
comment Comments
C Craig Todd

Craig access_time 3 years ago link more_vert

Thanks so much for this post and even more so your post about how to compile Hadoop 3.2.1 winutils.  I followed those instructions to the letter with great success (starting with a blank Windows 10 VM is definitely a critical starting point!).  I have just tried following the same steps to compile 3.3.1 but ran into additional compiler errors on the native HDFS libraries.  By chance, did you have to make any tweaks to any files similar to ones you made for 3.2.1 file src\main\native\libhdfs-tests\test_libhdfs_threaded.c?  Or any other environment/tool changes you may have made from 3.2.1?  Thanks for any feedback you might have.

Raymond Raymond

Raymond access_time 3 years ago link more_vert

Hi Craig,

For 3.3.1, it is not easy to compile on Windows. I tried to get it work but unfortunately there were too many errors I encountered. Eventually, I only complied the native HDFS project and reused the JARs from binary package provided by ASF. 

C Craig Todd

Craig access_time 3 years ago link more_vert

Ah, understood.  Luckily, it seems the 3.2.1 winutils files work just fine with the 3.3.1 stock files, so that's good enough for me.  Thanks for the quick reply, cheers!

D Dinesh P

Dinesh access_time 4 years ago link more_vert

Hi - can you please add back the original Apache License to the gitbug (all repos forked from https://github.com/steveloughran/winutils/blob/master/ which is on Apache 2.0)

Raymond Raymond

Raymond access_time 4 years ago link more_vert

Thanks for the suggestion. I've added the Hadoop project license file in as Hadoop project actually has different license for different part of the project. 

https://github.com/kontext-tech/winutils/blob/master/LICENSE.txt 

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts