arrow_back Compile and Build Hadoop 3.2.1 on Windows 10 Guide

comment Comments

11-13 of 13

access_time 3 years ago more_vert

#374 Re: Compile and Build Hadoop 3.2.1 on Windows 10 Guide

Hi Jin,

I don't think Maven clean is the approach to resolve a build problem.

mvn package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true -X

Can you clean up the generated files and rerun your build with additional -X option? In this way you can see all the detailed errors. Usually the detailed error message will be able to guide you to resolve the problem.

-Raymond

format_quote

person Jin access_time 3 years ago
Re: Compile and Build Hadoop 3.2.1 on Windows 10 Guide

Hi Raymond - can you advise? I got to the last step and got stuck with the build, encountering this error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.2.0:shade (default) on project hadoop-client-minicluster: Error creating shaded jar: duplicate entry: META-INF/services/org.apache.hadoop.shaded.org.eclipse.jetty.http.HttpFieldPreEncoder -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hadoop-client-minicluster

Searched online and this site said to use "clean" so I ran the following code:

mvn clean package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true

And the build did come back as successful. However, i don't see Hadoop-3.2.1 at C:/hdp/hadoop/hadoop-dist/target.

Perhaps using clean is not the right way? Any thoughts on how to fix this issue?

Thanks!

reply Reply

access_time 3 years ago more_vert

#373 Re: Compile and Build Hadoop 3.2.1 on Windows 10 Guide

Hi Raymond - can you advise? I got to the last step and got stuck with the build, encountering this error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.2.0:shade (default) on project hadoop-client-minicluster: Error creating shaded jar: duplicate entry: META-INF/services/org.apache.hadoop.shaded.org.eclipse.jetty.http.HttpFieldPreEncoder -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hadoop-client-minicluster

Searched online and this site said to use "clean" so I ran the following code:

mvn clean package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true

And the build did come back as successful. However, i don't see Hadoop-3.2.1 at C:/hdp/hadoop/hadoop-dist/target.

Perhaps using clean is not the right way? Any thoughts on how to fix this issue?

Thanks!

reply Reply

access_time 4 years ago more_vert

#319 Re: Compile and Build Hadoop 3.2.1 on Windows 10 Guide

To build Hadoop 3.3.0, follow build instructions:

https://github.com/apache/hadoop/blob/rel/release-3.3.0/BUILDING.txt

Most of these steps are similar to this post.

reply Reply

11-13 of 13

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

toc Table of contents

Prerequisites

insights Stats

warning Please login first to view stats information.

tag Big Data Tools on Windows 10

article Install Hadoop 3.3.0 on Windows 10 Step by Step Guide article Install Apache Spark 3.0.0 on Windows 10 article Install Hadoop 3.0.0 on Windows (Single Node) article Install Zeppelin 0.7.3 on Windows article Install Apache Sqoop in Windows Read more (11)

tagwindows10