CMake Build Error - Could not Find OpenSSL on Windows 10
Background
When I was building Hadoop 3.3.0 on Windows 10, I hit one error about OpenSSL:
CMake Error at C:/Program Files/CMake/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message): Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY OPENSSL_INCLUDE_DIR)
This error was raised by the following line in CMakeLists.txt:
find_package(OpenSSL REQUIRED)
The following is my environment setup:
-- Building for: Visual Studio 16 2019 -- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.18363. -- The C compiler identification is MSVC 19.26.28806.0 -- The CXX compiler identification is MSVC 19.26.28806.0
CMake version
CMake version is 3.18.0.
C:\hdp\cmaketest>cmake --version cmake version 3.18.0 CMake suite maintained and supported by Kitware (kitware.com/cmake).
Microsoft Visual C++ version
MSVC 19.26.28806.0
OpenSSL version
OpenSSL is installed on Windows using Microsoft vcpkg (a package manager for C++). The version installed in OpenSSL 1.1.18.
C:\hdp\cmaketest>openssl version OpenSSL 1.1.1g 21 Apr 2020
Reproduce the issue
Follow the steps below can reproduce this issue.
- Create project folder C:\hdp\cmaketest
- Create file CMakeLists.txt in the project folder with the following content:
cmake_minimum_required(VERSION 3.10) # set the project name project(Test) # Find package find_package(OpenSSL REQUIRED)
- Run CMake command:
cmake .
The error will occur as the following screenshot shows:
Fix this issue
In my system, OpenSSL is installed via vcpkg and installation folder is: C:/vcpkg/installed/x64-windows-static.
Thus to fix this issue, I just need to change the command line to the following:
cmake . -DOPENSSL_ROOT_DIR="C:/vcpkg/installed/x64-windows-static"
The above command line tells CMake that OpenSSL root folder is in the above directory. If you OpenSSL is installed with different triplet or location, please update the command line accordingly.
Reference
Which Hadoop version are you building? I recommend using Docker to build the latest Hadoop: Build Latest Hadoop on Windows 10 natively via Docker (kontext.tech)
It usually means there is no CMAKE_PREFIX_PATH set for your cmake command when building HDFS native client.
I Builing Hadoop-3.3.1,I can't use docker in company. So i try to build with native win.
Did you use vcpkg to install openssl and other required libs?
No I install it directly
Try install vcpkg and use it to install the dependent lib like OpenSSL.
https://kontext.tech/article/594/microsoft-vcpkg-c-library-manager
If it still doesn’t work, you need use maven debug mode (-X) to see if your maven command has parameters about cmake prefix path or OpenSSL path.
I download vcpkg,and install openssl.but it still got error. And I build with -X,here is the error message:
Execute:Java13CommandLauncher: Executing 'cmake' with arguments:
'C:\code\hadoop-compile\hadoop-hdfs-project\hadoop-hdfs-native-client/src/'
'-DGENERATED_JAVAH=C:\code\hadoop-compile\hadoop-hdfs-project\hadoop-hdfs-native-client\target/native/javah'
'-DJVM_ARCH_DATA_MODEL=64'
'-DHADOOP_BUILD=1'
'-DREQUIRE_FUSE=false'
'-DREQUIRE_VALGRIND=false'
'-A'
'x64'
The ' characters around the executable and arguments are
not part of the command.
[exec] CMake Warning (dev) in CMakeLists.txt:
[exec] No project() command is present. The top-level CMakeLists.txt file must
[exec] contain a li-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19043.
[exec] teral, direct call to the project() command. Add a line of
[exec] code such as
[exec]
[exec] project(ProjectName)
The command was generated by the following line in the pom.xml file:
Can you try adding the following argument into your Maven build tool too?
-Dnative_cmake_args="-DOPENSSL_ROOT_DIR=C:/vcpkg/installed/x64-windows-static"
Please change the path to the location where those C++ packages installed in your computer. Under that path, there should include sub folders like bin, lib, shared, etc.
BTW did you run vcpkg integrate install command?
Hi,
I have encounter same problem, but from your article could not able to figure out how did you manage to fix the issue. Can you please elaborate little bit?
Thanking you in advance.
Cheers
Execute:Java13CommandLauncher: Executing 'cmake' with arguments:
'C:\Bigdata\hadoop-src\hadoop-hdfs-project\hadoop-hdfs-native-client/src/'
'-DGENERATED_JAVAH=C:\Bigdata\hadoop-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target/native/javah'
'-DJVM_ARCH_DATA_MODEL=64'
'-DHADOOP_BUILD=1'
'-DREQUIRE_FUSE=false'
'-DREQUIRE_VALGRIND=false'
'-A'
'x64'
The ' characters around the executable and arguments are
not part of the command.
[exec] CMake Warning (dev) in CMakeLists.txt:
[exec] No project() command is present. The top-level CMakeLists.txt file must
[exec] contain a li-- Building for: Visual Studio 16 2019
[exec] -- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.18363.
[exec] teral, direct call to the project() command. Add a line of
[exec] code such as
[exec]
[exec] project(ProjectName)
[exec]
[exec] near the top of the file, but after cmake_minimum_required().
[exec]
[exec] CMake is pretending there is a "project(Project)" command on the first
[exec] line.
[exec] This warning is for project developers. Use -Wno-dev to suppress it.
[exec]
[exec] -- The C compiler identification is MSVC 19.28.29336.0
[exec] -- The CXX compiler identification is MSVC 19.28.29336.0
[exec] -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe
[exec] -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe -- works
[exec] -- Detecting C compiler ABI info
[exec] -- Detecting C compiler ABI info - done
[exec] -- Detecting C compile features
[exec] -- Detecting C compile features - done
[exec] -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe
[exec] -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/Hostx64/x64/cl.exe -- works
[exec] -- Detecting CXX compiler ABI info
[exec] -- Detecting CXX compiler ABI info - done
[exec] -- Detecting CXX compile features
[exec] -- Detecting CXX compile features - done
[exec] -- Performing Test HAVE_BETTER_TLS
[exec] -- Performing Test HAVE_BETTER_TLS - Failed
[exec] -- Performing Test HAVE_INTEL_SSE_INTRINSICS
[exec] -- Performing Test HAVE_INTEL_SSE_INTRINSICS - Success
[exec] -- Looking for dlopen in dl
[exec] -- Looking for dlopen in dl - not found
[exec] -- Found Java: C:/Java/jdk1.8.0_152/bin/java.exe (found version "1.8.0_152")
[exec] -- Found JNI: C:/Java/jdk1.8.0_152/lib/jawt.lib
[exec] OPENSSL_LIBRARY=OPENSSL_LIBRARY-NOTFOUND
[exec] Cannot find a usable OpenSSL library. OPENSSL_LIBRARY=OPENSSL_LIBRARY-NOTFOUND, OPENSSL_INCLUDE_DIR=C:/Bigdata/openssl-1.1.1i/include, CUSTOM_OPENSSL_LIB=, CUSTOM_OPENSSL_PREFIX=, CUSTOM_OPENSSL_INCLUDE=
[exec] -- Performing Test THREAD_LOCAL_SUPPORTED
[exec] -- Performing Test THREAD_LOCAL_SUPPORTED - Success
[exec] -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
[exec] CMake Error at C:/Bigdata/CMake/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
[exec] Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
[exec] system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY
[exec] OPENSSL_INCLUDE_DIR)
[exec] Call Stack (most recent call first):
[exec] C:/Bigdata/CMake/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
[exec] C:/Bigdata/CMake/share/cmake-3.16/Modules/FindOpenSSL.cmake:449 (find_package_handle_standard_args)
[exec] main/native/libhdfspp/CMakeLists.txt:44 (find_package)
[exec]
[exec]
[exec] -- Configuring incomplete, errors occurred!
[exec] See also "C:/Bigdata/hadoop-src/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/CMakeFiles/CMakeOutput.log".
[exec] See also "C:/Bigdata/hadoop-src/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/CMakeFiles/CMakeError.log".
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Apache Hadoop Main 3.3.0:
[INFO]
[INFO] Apache Hadoop Main ................................. SUCCESS [01:43 min]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [ 7.160 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ 4.774 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ 4.831 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 3.847 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 8.451 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 9.165 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 3.979 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [ 12.633 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 6.696 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [03:10 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 8.547 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [ 12.081 s]
[INFO] Apache Hadoop Registry ............................. SUCCESS [ 11.141 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [ 1.184 s]
[INFO] Apache Hadoop HDFS Client .......................... SUCCESS [ 58.089 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [02:09 min]
[INFO] Apache Hadoop HDFS Native Client ................... FAILURE [ 25.773 s]
[INFO] Apache Hadoop HttpFS ............................... SKIPPED
[INFO] Apache Hadoop HDFS-NFS ............................. SKIPPED
[INFO] Apache Hadoop HDFS-RBF ............................. SKIPPED
[INFO] Apache Hadoop HDFS Project ......................... SKIPPED
[INFO] Apache Hadoop YARN ................................. SKIPPED
[INFO] Apache Hadoop YARN API ............................. SKIPPED
[INFO] Apache Hadoop YARN Common .......................... SKIPPED
[INFO] Apache Hadoop YARN Server .......................... SKIPPED
[INFO] Apache Hadoop YARN Server Common ................... SKIPPED
[INFO] Apache Hadoop YARN NodeManager ..................... SKIPPED
[INFO] Apache Hadoop YARN Web Proxy ....................... SKIPPED
[INFO] Apache Hadoop YARN ApplicationHistoryService ....... SKIPPED
[INFO] Apache Hadoop YARN Timeline Service ................ SKIPPED
[INFO] Apache Hadoop YARN ResourceManager ................. SKIPPED
[INFO] Apache Hadoop YARN Server Tests .................... SKIPPED
[INFO] Apache Hadoop YARN Client .......................... SKIPPED
[INFO] Apache Hadoop YARN SharedCacheManager .............. SKIPPED
[INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SKIPPED
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SKIPPED
[INFO] Apache Hadoop YARN TimelineService HBase Common .... SKIPPED
[INFO] Apache Hadoop YARN TimelineService HBase Client .... SKIPPED
[INFO] Apache Hadoop YARN TimelineService HBase Servers ... SKIPPED
[INFO] Apache Hadoop YARN TimelineService HBase Server 1.2 SKIPPED
[INFO] Apache Hadoop YARN TimelineService HBase tests ..... SKIPPED
[INFO] Apache Hadoop YARN Router .......................... SKIPPED
[INFO] Apache Hadoop YARN TimelineService DocumentStore ... SKIPPED
[INFO] Apache Hadoop YARN Applications .................... SKIPPED
[INFO] Apache Hadoop YARN DistributedShell ................ SKIPPED
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SKIPPED
[INFO] Apache Hadoop MapReduce Client ..................... SKIPPED
[INFO] Apache Hadoop MapReduce Core ....................... SKIPPED
[INFO] Apache Hadoop MapReduce Common ..................... SKIPPED
[INFO] Apache Hadoop MapReduce Shuffle .................... SKIPPED
[INFO] Apache Hadoop MapReduce App ........................ SKIPPED
[INFO] Apache Hadoop MapReduce HistoryServer .............. SKIPPED
[INFO] Apache Hadoop MapReduce JobClient .................. SKIPPED
[INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED
[INFO] Apache Hadoop YARN Services ........................ SKIPPED
[INFO] Apache Hadoop YARN Services Core ................... SKIPPED
[INFO] Apache Hadoop YARN Services API .................... SKIPPED
[INFO] Apache Hadoop YARN Application Catalog ............. SKIPPED
[INFO] Apache Hadoop YARN Application Catalog Webapp ...... SKIPPED
[INFO] Apache Hadoop YARN Application Catalog Docker Image SKIPPED
[INFO] Apache Hadoop YARN Application MaWo ................ SKIPPED
[INFO] Apache Hadoop YARN Application MaWo Core ........... SKIPPED
[INFO] Apache Hadoop YARN Site ............................ SKIPPED
[INFO] Apache Hadoop YARN Registry ........................ SKIPPED
[INFO] Apache Hadoop YARN UI .............................. SKIPPED
[INFO] Apache Hadoop YARN CSI ............................. SKIPPED
[INFO] Apache Hadoop YARN Project ......................... SKIPPED
[INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SKIPPED
[INFO] Apache Hadoop MapReduce NativeTask ................. SKIPPED
[INFO] Apache Hadoop MapReduce Uploader ................... SKIPPED
[INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
[INFO] Apache Hadoop MapReduce ............................ SKIPPED
[INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
[INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
[INFO] Apache Hadoop Client Aggregator .................... SKIPPED
[INFO] Apache Hadoop Dynamometer Workload Simulator ....... SKIPPED
[INFO] Apache Hadoop Dynamometer Cluster Simulator ........ SKIPPED
[INFO] Apache Hadoop Dynamometer Block Listing Generator .. SKIPPED
[INFO] Apache Hadoop Dynamometer Dist ..................... SKIPPED
[INFO] Apache Hadoop Dynamometer .......................... SKIPPED
[INFO] Apache Hadoop Archives ............................. SKIPPED
[INFO] Apache Hadoop Archive Logs ......................... SKIPPED
[INFO] Apache Hadoop Rumen ................................ SKIPPED
[INFO] Apache Hadoop Gridmix .............................. SKIPPED
[INFO] Apache Hadoop Data Join ............................ SKIPPED
[INFO] Apache Hadoop Extras ............................... SKIPPED
[INFO] Apache Hadoop Pipes ................................ SKIPPED
[INFO] Apache Hadoop OpenStack support .................... SKIPPED
[INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
[INFO] Apache Hadoop Kafka Library support ................ SKIPPED
[INFO] Apache Hadoop Azure support ........................ SKIPPED
[INFO] Apache Hadoop Aliyun OSS support ................... SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
[INFO] Apache Hadoop Resource Estimator Service ........... SKIPPED
[INFO] Apache Hadoop Azure Data Lake support .............. SKIPPED
[INFO] Apache Hadoop Image Generation Tool ................ SKIPPED
[INFO] Apache Hadoop Tools Dist ........................... SKIPPED
[INFO] Apache Hadoop Tools ................................ SKIPPED
[INFO] Apache Hadoop Client API ........................... SKIPPED
[INFO] Apache Hadoop Client Runtime ....................... SKIPPED
[INFO] Apache Hadoop Client Packaging Invariants .......... SKIPPED
[INFO] Apache Hadoop Client Test Minicluster .............. SKIPPED
[INFO] Apache Hadoop Client Packaging Invariants for Test . SKIPPED
[INFO] Apache Hadoop Client Packaging Integration Tests ... SKIPPED
[INFO] Apache Hadoop Distribution ......................... SKIPPED
[INFO] Apache Hadoop Client Modules ....................... SKIPPED
[INFO] Apache Hadoop Tencent COS Support .................. SKIPPED
[INFO] Apache Hadoop Cloud Storage ........................ SKIPPED
[INFO] Apache Hadoop Cloud Storage Project ................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10:07 min
[INFO] Finished at: 2021-01-16T11:02:15+11:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs-native-client: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec failonerror="true" dir="C:\Bigdata\hadoop-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target/native" executable="cmake">... @ 5:135 in C:\Bigdata\hadoop-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target\antrun\build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs-native-client: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec failonerror="true" dir="C:\Bigdata\hadoop-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target/native" executable="cmake">... @ 5:135 in C:\Bigdata\hadoop-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target\antrun\build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec failonerror="true" dir="C:\Bigdata\hadoop-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target/native" executable="cmake">... @ 5:135 in C:\Bigdata\hadoop-src\hadoop-hdfs-project\hadoop-hdfs-native-client\target\antrun\build-main.xml
at org.apache.maven.plugin.antrun.AntRunMojo.execute (AntRunMojo.java:355)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.tools.ant.BuildException: exec returned: 1
at org.apache.tools.ant.taskdefs.ExecTask.runExecute (ExecTask.java:646)
at org.apache.tools.ant.taskdefs.ExecTask.runExec (ExecTask.java:672)
at org.apache.tools.ant.taskdefs.ExecTask.execute (ExecTask.java:498)
at org.apache.tools.ant.UnknownElement.execute (UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor31.invoke (Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.apache.tools.ant.dispatch.DispatchUtils.execute (DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform (Task.java:348)
at org.apache.tools.ant.Target.execute (Target.java:390)
at org.apache.tools.ant.Target.performTasks (Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets (Project.java:1399)
at org.apache.tools.ant.Project.executeTarget (Project.java:1368)
at org.apache.maven.plugin.antrun.AntRunMojo.execute (AntRunMojo.java:327)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :hadoop-hdfs-native-client
I have provided the detailed steps here about how to install vcpkg on Windows and also to install openssl: Microsoft vcpkg C++ Library Manager
For fixing issues about missing OpenSSL, you can install it via vcpkg.
About vcpkg, follow this page to install it: microsoft/vcpkg: C++ Library Manager for Windows, Linux, and MacOS (github.com)
Thanks a lot for replying, and very much appreciated.
I installed the OpenSSL as suggested by you. Added the OPENSSL_ROOT_DIR in system path, still getting same error.
C:\Bigdata\vcpkg\vcpkg>vcpkg install openssl
Computing installation plan...
A suitable version of cmake was not found (required v3.19.2). Downloading portable cmake v3.19.2...
Downloading cmake...
https://github.com/Kitware/CMake/releases/download/v3.19.2/cmake-3.19.2-win32-x86.zip -> C:\Bigdata\vcpkg\vcpkg\downloads\cmake-3.19.2-win32-x86.zip
Extracting cmake...
A suitable version of 7zip was not found (required v18.1.0). Downloading portable 7zip v18.1.0...
Downloading 7zip...
https://www.nuget.org/api/v2/package/7-Zip.CommandLine/18.1.0 -> C:\Bigdata\vcpkg\vcpkg\downloads\7-zip.commandline.18.1.0.nupkg
Extracting 7zip...
A suitable version of nuget was not found (required v5.5.1). Downloading portable nuget v5.5.1...
Downloading nuget...
https://dist.nuget.org/win-x86-commandline/v5.5.1/nuget.exe -> C:\Bigdata\vcpkg\vcpkg\downloads\22ea847d-nuget.exe
The following packages will be built and installed:
openssl[core]:x86-windows -> 1.1.1i
Detecting compiler hash for triplet x86-windows...
A suitable version of powershell-core was not found (required v7.1.0). Downloading portable powershell-core v7.1.0...
Downloading powershell-core...
https://github.com/PowerShell/PowerShell/releases/download/v7.1.0/PowerShell-7.1.0-win-x86.zip -> C:\Bigdata\vcpkg\vcpkg\downloads\PowerShell-7.1.0-win-x86.zip
Extracting powershell-core...
Could not locate cached archive: C:\Users\hiten\AppData\Local\vcpkg\archives\cf\cf5e781eafe65b7336760f75bbf784ae37ef62d3.zip
Starting package 1/1: openssl:x86-windows
Building package openssl[core]:x86-windows...
-- Downloading https://www.openssl.org/source/openssl-1.1.1i.tar.gz -> openssl-1.1.1i.tar.gz...
-- Downloading https://strawberry.perl.bot/download/5.30.0.1/strawberry-perl-5.30.0.1-32bit.zip -> strawberry-perl-5.30.0.1-32bit.zip...
-- Downloading https://strawberry.perl.bot/download/5.30.0.1/strawberry-perl-5.30.0.1-32bit.zip... Failed. Status: 22;"HTTP response code said error"
-- Downloading http://strawberryperl.com/download/5.30.0.1/strawberry-perl-5.30.0.1-32bit.zip -> strawberry-perl-5.30.0.1-32bit.zip...
-- Extracting source C:/Bigdata/vcpkg/vcpkg/downloads/openssl-1.1.1i.tar.gz
-- Using source at C:/Bigdata/vcpkg/vcpkg/buildtrees/openssl/src/openssl-1-ae932e0835.clean
-- Downloading https://www.nasm.us/pub/nasm/releasebuilds/2.15.05/win32/nasm-2.15.05-win32.zip -> nasm-2.15.05-win32.zip...
-- Downloading https://download.qt.io/official_releases/jom/jom_1_1_3.zip -> jom_1_1_3.zip...
-- Copying openssl release source files...
-- Copying openssl release source files... done
-- Configure x86-windows-rel
-- Configure x86-windows-rel done
-- Build x86-windows-rel
-- Build x86-windows-rel done
-- Copying openssl debug source files...
-- Copying openssl debug source files... done
-- Configure x86-windows-dbg
-- Configure x86-windows-dbg done
-- Build x86-windows-dbg
-- Build x86-windows-dbg done
-- Installing: C:/Bigdata/vcpkg/vcpkg/packages/openssl_x86-windows/share/openssl/copyright
-- Installing: C:/Bigdata/vcpkg/vcpkg/packages/openssl_x86-windows/share/openssl/usage
-- Performing post-build validation
-- Performing post-build validation done
Stored binary cache: C:\Users\hiten\AppData\Local\vcpkg\archives\cf\cf5e781eafe65b7336760f75bbf784ae37ef62d3.zip
Building package openssl[core]:x86-windows... done
Installing package openssl[core]:x86-windows...
Installing package openssl[core]:x86-windows... done
Elapsed time for package openssl:x86-windows: 24.95 min
Total elapsed time: 34.21 min
The package openssl is compatible with built-in CMake targets:
find_package(OpenSSL REQUIRED)
target_link_libraries(main PRIVATE OpenSSL::SSL OpenSSL::Crypto)
Did you run the test CMake project (Not the HDFS project) successfully?
From the logs, I can see you are installing the x86 version while in my article, I mentioned to use x64 triplet.
https://kontext.tech/column/c_and_cplusplus/594/microsoft-vcpkg-c-library-manager
Thanks once again for quick reply.
I followed the steps from the above URL. Still having issues. I found out issues with the cmake.txt file C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src"", where in line number 142-144, if I commented out it is working fine.
#set(OPENSSL_LIBRARY "")
#set(OPENSSL_INCLUDE_DIR "")
#set(OPENSSL_SOURCE_FILES "")
But got new error message,
main:
[exec] CMake Warning (dev) in CMakeLists.txt:
[exec] No project() command is present. The top-level CMakeLists.txt file must
[exec] contain a li-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.18363.
[exec] teral, direct call to the project() command. Add a line of
[exec] code such as
[exec]
[exec] project(ProjectName)
[exec]
[exec] near the top of the file, but after cmake_minimum_required().
[exec]
[exec] CMake is pretending there is a "project(Project)" command on the first
[exec] line.
[exec] This warning is for project developers. Use -Wno-dev to suppress it.
[exec]
[exec] CUSTOM_OPENSSL_PREFIX =
[exec] -- Performing Test THREAD_LOCAL_SUPPORTED
[exec] Cannot find a usable OpenSSL library. OPENSSL_LIBRARY=OPENSSL_LIBRARY-NOTFOUND, OPENSSL_INCLUDE_DIR=C:/Bigdata/vcpkg/vcpkg/packages/openssl_x64-windows/include, CUSTOM_OPENSSL_LIB=, CUSTOM_OPENSSL_PREFIX=, CUSTOM_OPENSSL_INCLUDE=
[exec] -- Performing Test THREAD_LOCAL_SUPPORTED - Success
[exec] -- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
[exec] -- Could NOT find CYRUS_SASL (missing: CYRUS_SASL_SHARED_LIB CYRUS_SASL_INCLUDE_DIR)
[exec] -- Could NOT find GSASL (missing: GSASL_LIBRARIES GSASL_INCLUDE_DIR)
[exec] -- Performing Test THREAD_LOCAL_SUPPORTED
[exec] -- Performing Test THREAD_LOCAL_SUPPORTED - Success
[exec] -- Performing Test PROTOC_IS_COMPATIBLE
[exec] -- Performing Test PROTOC_IS_COMPATIBLE - Failed
[exec] -- valgrind location: MEMORYCHECK_COMMAND-NOTFOUND
[exec] -- Compiling with NO SASL SUPPORT
[exec] -- Syncing C:/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/include/*.h* to C:/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/main/native/libhdfspp/extern/include
[exec] -- Syncing C:/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/*.proto to C:/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/main/native/libhdfspp/extern/proto/hdfs
[exec] -- Syncing C:/hadoop/hadoop-common-project/hadoop-common/src/main/proto/*.proto to C:/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/main/native/libhdfspp/extern/proto/hadoop
[exec] -- Syncing C:/hadoop/hadoop-common-project/hadoop-common/src/test/proto/*.proto to C:/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/main/native/libhdfspp/extern/proto/hadoop_test
[exec] CMake Warning at main/native/libhdfspp/CMakeLists.txt:86 (message):
[exec] WARNING: the Protocol Buffers Library and the Libhdfs++ Library must both
[exec] be compiled with the same (or compatible) compiler. Normally only the same
[exec] major versions of the same compiler are compatible with each other.
[exec]
[exec]
[exec] -- Non-Linux system detected. Will not build FUSE client.
[exec] -- Configuring incomplete, errors occurred!
[exec] See also "C:/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/CMakeFiles/CMakeOutput.log".
[exec] See also "C:/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/CMakeFiles/CMakeError.log".
[exec] CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
[exec] Please set them or make sure they are set and tested correctly in the CMake files:
[exec] Protobuf_PROTOC_LIBRARY
[exec] linked by target "protoc-gen-hrpc" in directory C:/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/proto
[exec]
Hi, your build failed because CMake cannot find protobuf.
I can repeat your errors using a test project including a CMakeLists.txt file with the following content:
cmake_minimum_required(VERSION 3.10) # set the project name project(Test) # Find package find_package(OpenSSL REQUIRED) find_package(Protobuf REQUIRED)
The build failed with the following error:
C:\hdp\cmaketest>cmake . -- Selecting Windows SDK version to target Windows 10.0.18363. CMake Error at C:/Program Files/CMake/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:218 (message): Could NOT find Protobuf (missing: Protobuf_LIBRARIES Protobuf_INCLUDE_DIR) Call Stack (most recent call first): C:/Program Files/CMake/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:582 (_FPHSA_FAILURE_MESSAGE) C:/Program Files/CMake/share/cmake-3.19/Modules/FindProtobuf.cmake:628 (FIND_PACKAGE_HANDLE_STANDARD_ARGS) CMakeLists.txt:8 (find_package) -- Configuring incomplete, errors occurred! See also "C:/hdp/cmaketest/CMakeFiles/CMakeOutput.log". See also "C:/hdp/cmaketest/CMakeFiles/CMakeError.log".
I installed protobuf library using vcpkg:
C:\dev\vcpkg>vcpkg install protobuf
By default, triplet is set as x64-windows:
If you don't have this environment variable (VCPKG_DEFAULT_TRIPLET) setup, you can also directly run the following command to install:
C:\dev\vcpkg>vcpkg install protobuf:x64-windows
Wait until the installation is done:
Remember to setup an environment variable CMAKE_PREFIX_PATH like the following so that CMake knows where to find those installed libraries:
Alternatively, you will need change HDFS native project pom file cmake command to add the path there.
Remember to clean the cached CMake files. You can use Git to clean all the generated files.
And then rerun the test project and it will become successful:
C:\hdp\cmaketest>cmake . -- Building for: Visual Studio 14 2015 -- Selecting Windows SDK version to target Windows 10.0.18363. -- The C compiler identification is MSVC 19.0.24215.1 -- The CXX compiler identification is MSVC 19.0.24215.1 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/cl.exe - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found OpenSSL: C:/dev/vcpkg/installed/x64-windows/lib/libcrypto.lib (found version "1.1.1i") -- Found Protobuf: C:/dev/vcpkg/installed/x64-windows/lib/libprotobuf.lib (found version "3.14.0") -- Configuring done -- Generating done -- Build files have been written to: C:/hdp/cmaketest
As you can see Protobuf is now found.
And then you can continue to build your Hadoop project. Good luck!
If you still cannot resolve the problem, have a look at the CMake error log file:
{Your Hadoop Folder}/hadoop-hdfs-project/hadoop-hdfs-native-client/target/native/CMakeFiles/CMakeError.log
Thanks a lot again for your help and support !
I tried with your steps, and sample CMake file able to build. Now there is no failure from source generation, but got into different error as compilation error, bit strange since I have not changed any source, might be some setting I am missing.
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj" (default target) (5) ->
[exec] (ClCompile target) ->
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error_code.ipp(40,15): error C2694: 'const char *asio::detail::system_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error.ipp(96,15): error C2694: 'const char *asio::error::detail::misc_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error_code.ipp(40,15): error C2694: 'const char *asio::detail::system_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error.ipp(96,15): error C2694: 'const char *asio::error::detail::misc_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\configuration.cc(37,10): fatal error C1083: Cannot open include file: 'strings.h': No such file or directory [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error_code.ipp(40,15): error C2694: 'const char *asio::detail::system_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error.ipp(96,15): error C2694: 'const char *asio::error::detail::misc_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\configuration_loader.cc(23,10): fatal error C1083: Cannot open include file: 'strings.h': No such file or directory [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error_code.ipp(40,15): error C2694: 'const char *asio::detail::system_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error.ipp(96,15): error C2694: 'const char *asio::error::detail::misc_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error_code.ipp(40,15): error C2694: 'const char *asio::detail::system_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error.ipp(96,15): error C2694: 'const char *asio::error::detail::misc_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error_code.ipp(40,15): error C2694: 'const char *asio::detail::system_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error.ipp(96,15): error C2694: 'const char *asio::error::detail::misc_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error_code.ipp(40,15): error C2694: 'const char *asio::detail::system_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error.ipp(96,15): error C2694: 'const char *asio::error::detail::misc_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error_code.ipp(40,15): error C2694: 'const char *asio::detail::system_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\asio-1.10.2\include\asio/impl/error.ipp(96,15): error C2694: 'const char *asio::error::detail::misc_category::name(void) const': overriding virtual function has less restrictive exception specification than base class virtual member function 'const char *std::error_category::name(void) noexcept const' [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\statinfo.cc(41,28): error C2065: 'S_IRUSR': undeclared identifier [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\statinfo.cc(42,28): error C2065: 'S_IWUSR': undeclared identifier [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\statinfo.cc(43,28): error C2065: 'S_IXUSR': undeclared identifier [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\statinfo.cc(44,28): error C2065: 'S_IRGRP': undeclared identifier [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\statinfo.cc(45,28): error C2065: 'S_IWGRP': undeclared identifier [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\statinfo.cc(46,28): error C2065: 'S_IXGRP': undeclared identifier [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\statinfo.cc(47,28): error C2065: 'S_IROTH': undeclared identifier [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\statinfo.cc(48,28): error C2065: 'S_IWOTH': undeclared identifier [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\common\statinfo.cc(49,28): error C2065: 'S_IXOTH': undeclared identifier [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common_obj.vcxproj]
[exec]
[exec]
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\ALL_BUILD.vcxproj" (default target) (1) ->
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\bindings\c\bindings_c_obj.vcxproj" (default target) (3) ->
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\common\common.vcxproj" (default target) (4) ->
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\third_party\uriparser2\uriparser2_obj.vcxproj" (default target) (6) ->
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\uriparser2\uriparser2\uriparser2.c(74,23): error C2057: expected constant expression [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\third_party\uriparser2\uriparser2_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\uriparser2\uriparser2\uriparser2.c(74,23): error C2466: cannot allocate an array of constant size 0 [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\third_party\uriparser2\uriparser2_obj.vcxproj]
[exec] C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\third_party\uriparser2\uriparser2\uriparser2.c(74,24): error C2133: 'buffer': unknown size [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\third_party\uriparser2\uriparser2_obj.vcxproj]
[exec]
[exec]
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\ALL_BUILD.vcxproj" (default target) (1) ->
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\bindings\c\bindings_c_obj.vcxproj" (default target) (3) ->
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\fs\fs.vcxproj" (default target) (7) ->
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\fs\fs_obj.vcxproj" (default target) (8) ->
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\proto\proto.vcxproj" (default target) (9) ->
[exec] "C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\proto\proto_obj.vcxproj" (default target) (10) ->
[exec] (CustomBuild target) ->
[exec] C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(238,5): error MSB8066: Custom build for 'C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\datatransfer.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\ClientDatanodeProtocol.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\ClientNamenodeProtocol.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\acl.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\encryption.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\erasurecoding.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\hdfs.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\inotify.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\xattr.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\ReconfigurationProtocol.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\IpcConnectionContext.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\ProtobufRpcEngine.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\RpcHeader.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\Security.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\HAServiceProtocol.pb.h.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\CMakeFiles\22ba60f9d122a6b94e9fab13c793b5f3\ClientNamenodeProtocol.hrpc.inl.rule;C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\src\main\native\libhdfspp\lib\proto\CMakeLists.txt' exited with code 1. [C:\hadoop\hadoop-hdfs-project\hadoop-hdfs-native-client\target\native\main\native\libhdfspp\lib\proto\proto_obj.vcxproj]
I am glad you have come to this step.
The error is probably because of some of the C/C++ code are not compatible with Visual C++ unfortunately (differences between VC++ and GNU C/C++). You need to manually make some adjustments.
Refer to section II Issues Fix on the build article where I shared one example of fixing some of the errors:
Compile and Build Hadoop 3.2.1 on Windows 10 Guide
Hopefully it will resolve your problem. As your build environment might not be the same as mine, for example your VC++ compiler version might be different from mine, you may encounter some other issues. If that happens, you will need to resolve them one by one. But if you've followed exact setup in my article, you should only encounter that code error I mentioned.
Thanks for your reply. And big thanks to guide me through my journey.
I uninstalled all, started from scratch with instructions from your page, and the issues you have mentioned I faced exactly same. After fixing those, able to build. Now I am ready to deploy and run.
Thanks once again, and have a great day.
You are welcome, Hitendra. I'm glad it is working!
In the past, there were also quite a few users that could not built correctly because they didn't follow exactly about the steps in my articles. For all the installation guides I published, I've tested them on a sandbox to ensure it works. So usually if you keep following the steps, it should work.
Cheers,
Raymond
it could build with openssl
but when i build in hadoop,I can't find openssl.