This page summarizes the steps required to install Apache Sqoop (v1.4.7) in Windows 10 environment via Windows Subsystem for Linux (WSL).

Prerequisites

If you have already installed Hadoop 3.2.0 in WSL, ignore the following steps as you don’t need to install it again.

Follow  the following pages to install WSL in a system or non-system drive on your Windows 10 and also install Hadoop 3.2.0 in the WSL.

Now let’s start to install Apache Sqoop v1.4.7.

Download binary package

Go to Sqoop official website to find a download URL for Sqoop 1.

In the mirror site, there are usually two binaries available:

  • sqoop-[version].bin__hadoop-2.6.0.tar.gz
  • sqoop-[version].tar.gz

We are after the first one.

To me, the recommended location is http://apache.mirror.digitalpacific.com.au/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz.

Download the package using the following command:

wget http://apache.mirror.digitalpacific.com.au/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

Unzip the binary package

Unpack the binary using the following command:

tar -xvzf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C ~/hadoop

Setup environment variables

Setup SQOOP_HOME environment variables and also add the bin subfolder into PATH variable.

Run the following command to change .bashrc file:

vi ~/.bashrc

Add the following lines to the end of the file:

export SQOOP_HOME=~/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0                                                             

export PATH=$SQOOP_HOME/bin:$PATH

Source the modified file to make it effective:

source  ~/.bashrc

Test configurations

Run the following command to test the configurations:

$SQOOP_HOME/bin/configure-sqoop

Example output:

Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

Print out version

The above warning messages are printed out because we have not configured HBase, HCAT, Accumulo or Zookeeper in the WSL distro yet.

$ sqoop version
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
/home/tangr/hadoop/hadoop-3.2.0/libexec/hadoop-functions.sh: line 2364: HADOOP_ORG.APACHE.SQOOP.SQOOP_USER: bad substitution
/home/tangr/hadoop/hadoop-3.2.0/libexec/hadoop-functions.sh: line 2459: HADOOP_ORG.APACHE.SQOOP.SQOOP_OPTS: bad substitution
2019-05-19 12:04:18,429 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017

info Last modified by Raymond at 9 months ago * This page is subject to Site terms.

More from Kontext

local_offer hadoop local_offer linux local_offer WSL

visibility 14200
thumb_up 8
access_time 2 years ago

In my previous post , I showed how to configure a single node Hadoop instance on Windows 10. The steps are not too difficult to follow if you have Java programming backgr...

open_in_new Hadoop

local_offer zeppelin local_offer spark local_offer hadoop local_offer linux local_offer sqoop local_offer hive local_offer WSL

visibility 1093
thumb_up 0
access_time 2 years ago

This page summarizes the installation guides about big data tools on Windows through Windows Subsystem for Linux (WSL). ...

open_in_new Sqoop

local_offer spark local_offer linux local_offer WSL

visibility 5205
thumb_up 0
access_time 2 years ago

This pages summarizes the steps to install the latest version 2.4.3 of Apache Spark on Windows 10 via Windows Subsystem for Linux (WSL). Prerequisites Follow either of the following pages to install WSL in a system or non-system drive on your Windows 10. ...

open_in_new Spark + PySpark

local_offer zeppelin local_offer spark local_offer linux local_offer WSL

visibility 2059
thumb_up 0
access_time 2 years ago

This page summarizes the steps to install Zeppelin version 0.7.3 on Windows 10 via Windows Subsystem for Linux (WSL). Version 0.8.1 When running Zeppelin in Ubuntu, the server may pick up one host address that is not accessible, for example 169.254.148.100, and the the remote interprete...

open_in_new Zeppelin

info About author

comment Comments (2)

comment Add comment

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

R
Raymondarrow_drop_down
@Sekhar You can refer to the user documentation link about sqoop import command. You need to pass in your JDBC url, user name, password, source table, etc.
format_quote

person Sekhar access_time 9 months ago
Re: Sqoop Installation on Windows 10 using Windows Subsystem for Linux

Thanks for the detailed steps for installing Hadoop, Sqoop. I was trying to sqoop data from mysql and getting different kind of errors. I followed your instructions to install on WSL. Would it be possible to provide steps/examples to sqoop data from any derby/mysql. Thank you.. 
reply Reply
account_circle Sekhar
Thanks for the detailed steps for installing Hadoop, Sqoop. I was trying to sqoop data from mysql and getting different kind of errors. I followed your instructions to install on WSL. Would it be possible to provide steps/examples to sqoop data from any derby/mysql. Thank you.. 
reply Reply

Dark theme mode

Dark theme mode is available on Kontext.

Learn more arrow_forward

Kontext Column

Created for everyone to publish data, programming and cloud related articles. Follow three steps to create your columns.


Learn more arrow_forward