arrow_back Compile and Build Hadoop 3.2.1 on Windows 10 Guide
person Jin access_time 4 years ago
Hi Raymond - can you advise? I got to the last step and got stuck with the build, encountering this error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.2.0:shade (default) on project hadoop-client-minicluster: Error creating shaded jar: duplicate entry: META-INF/services/org.apache.hadoop.shaded.org.eclipse.jetty.http.HttpFieldPreEncoder -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :hadoop-client-minicluster
Searched online and this site said to use "clean" so I ran the following code:
mvn clean package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true
And the build did come back as successful. However, i don't see Hadoop-3.2.1 at C:/hdp/hadoop/hadoop-dist/target.
Perhaps using clean is not the right way? Any thoughts on how to fix this issue?
Thanks!
Hi Raymond - can you advise? I got to the last step and got stuck with the build, encountering this error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.2.0:shade (default) on project hadoop-client-minicluster: Error creating shaded jar: duplicate entry: META-INF/services/org.apache.hadoop.shaded.org.eclipse.jetty.http.HttpFieldPreEncoder -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :hadoop-client-minicluster
Searched online and this site said to use "clean" so I ran the following code:
mvn clean package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true
And the build did come back as successful. However, i don't see Hadoop-3.2.1 at C:/hdp/hadoop/hadoop-dist/target.
Perhaps using clean is not the right way? Any thoughts on how to fix this issue?
Thanks!
To build Hadoop 3.3.0, follow build instructions:
https://github.com/apache/hadoop/blob/rel/release-3.3.0/BUILDING.txt
Most of these steps are similar to this post.
Hi Jin,
I don't think Maven clean is the approach to resolve a build problem.