Sqoop Installation on Windows 10 using Windows Subsystem for Linux
This page summarizes the steps required to install Apache Sqoop (v1.4.7) in Windows 10 environment via Windows Subsystem for Linux (WSL).
Prerequisites
If you have already installed Hadoop 3.2.0 in WSL, ignore the following steps as you don’t need to install it again.
Follow the following pages to install WSL in a system or non-system drive on your Windows 10 and also install Hadoop 3.2.0 in the WSL.
- Install Windows Subsystem for Linux on a Non-System Drive
- Install Hadoop 3.2.0 on Windows 10 using Windows Subsystem for Linux (WSL)
Now let’s start to install Apache Sqoop v1.4.7.
Download binary package
Go to Sqoop official website to find a download URL for Sqoop 1.
In the mirror site, there are usually two binaries available:
- sqoop-[version].bin__hadoop-2.6.0.tar.gz
- sqoop-[version].tar.gz
We are after the first one.
To me, the recommended location is http://apache.mirror.digitalpacific.com.au/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz.
Download the package using the following command:
wget http://apache.mirror.digitalpacific.com.au/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
Unzip the binary package
Unpack the binary using the following command:
tar -xvzf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C ~/hadoop
Setup environment variables
Setup SQOOP_HOME environment variables and also add the bin subfolder into PATH variable.
Run the following command to change .bashrc file:
vi ~/.bashrc
Add the following lines to the end of the file:
export SQOOP_HOME=~/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0
export PATH=$SQOOP_HOME/bin:$PATH
Source the modified file to make it effective:
source ~/.bashrc
Test configurations
Run the following command to test the configurations:
$SQOOP_HOME/bin/configure-sqoop
Example output:
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
Print out version
The above warning messages are printed out because we have not configured HBase, HCAT, Accumulo or Zookeeper in the WSL distro yet.
$ sqoop version
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
/home/tangr/hadoop/hadoop-3.2.0/libexec/hadoop-functions.sh: line 2364: HADOOP_ORG.APACHE.SQOOP.SQOOP_USER: bad substitution
/home/tangr/hadoop/hadoop-3.2.0/libexec/hadoop-functions.sh: line 2459: HADOOP_ORG.APACHE.SQOOP.SQOOP_OPTS: bad substitution
2019-05-19 12:04:18,429 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017