Raymond Raymond

Hadoop 3.3.1 winutils

event 2021-09-27 visibility 29,482 comment 5 insights toc
more_vert
insights Stats

Winutils is required when installing Hadoop on Windows environment.

Hadoop 3.3.1 winutils

I've compiled Hadoop 3.3.1 on Windows 10 using CMake and Visual Studio (MSVC x64).

Follow these two steps to download it:

  1. https://github.com/kontext-tech/winutils
  2. Go to subfolder hadoop-3.3.1/bin to download the binaries. 
warning These binaries are provided for testing and learning purposes and no guarantees are provided. Please don't use it on production environments. 

Verify the tool

You can run the winutils.exe program in your Windows PC to make sure it works:

20210927122931-image.png

Feedback

If you encounter any issues, please comment here.

More from Kontext
comment Comments
C Craig Todd #1623 access_time 3 years ago more_vert

Ah, understood.  Luckily, it seems the 3.2.1 winutils files work just fine with the 3.3.1 stock files, so that's good enough for me.  Thanks for the quick reply, cheers!

format_quote

person Raymond access_time 3 years ago

Hi Craig,

For 3.3.1, it is not easy to compile on Windows. I tried to get it work but unfortunately there were too many errors I encountered. Eventually, I only complied the native HDFS project and reused the JARs from binary package provided by ASF. 

Raymond Raymond #1622 access_time 3 years ago more_vert

Hi Craig,

For 3.3.1, it is not easy to compile on Windows. I tried to get it work but unfortunately there were too many errors I encountered. Eventually, I only complied the native HDFS project and reused the JARs from binary package provided by ASF. 

format_quote

person Craig access_time 3 years ago

Thanks so much for this post and even more so your post about how to compile Hadoop 3.2.1 winutils.  I followed those instructions to the letter with great success (starting with a blank Windows 10 VM is definitely a critical starting point!).  I have just tried following the same steps to compile 3.3.1 but ran into additional compiler errors on the native HDFS libraries.  By chance, did you have to make any tweaks to any files similar to ones you made for 3.2.1 file src\main\native\libhdfs-tests\test_libhdfs_threaded.c?  Or any other environment/tool changes you may have made from 3.2.1?  Thanks for any feedback you might have.

C Craig Todd #1621 access_time 3 years ago more_vert

Thanks so much for this post and even more so your post about how to compile Hadoop 3.2.1 winutils.  I followed those instructions to the letter with great success (starting with a blank Windows 10 VM is definitely a critical starting point!).  I have just tried following the same steps to compile 3.3.1 but ran into additional compiler errors on the native HDFS libraries.  By chance, did you have to make any tweaks to any files similar to ones you made for 3.2.1 file src\main\native\libhdfs-tests\test_libhdfs_threaded.c?  Or any other environment/tool changes you may have made from 3.2.1?  Thanks for any feedback you might have.

Raymond Raymond #1530 access_time 3 years ago more_vert

Thanks for the suggestion. I've added the Hadoop project license file in as Hadoop project actually has different license for different part of the project. 

https://github.com/kontext-tech/winutils/blob/master/LICENSE.txt 

format_quote

person Dinesh access_time 3 years ago

Hi - can you please add back the original Apache License to the gitbug (all repos forked from https://github.com/steveloughran/winutils/blob/master/ which is on Apache 2.0)

D Dinesh P #1529 access_time 3 years ago more_vert

Hi - can you please add back the original Apache License to the gitbug (all repos forked from https://github.com/steveloughran/winutils/blob/master/ which is on Apache 2.0)

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts