Sqoop Installation on Windows 10 using Windows Subsystem for Linux

access_time 2 years ago visibility1398 comment 2

This page summarizes the steps required to install Apache Sqoop (v1.4.7) in Windows 10 environment via Windows Subsystem for Linux (WSL).

Prerequisites

If you have already installed Hadoop 3.2.0 in WSL, ignore the following steps as you don’t need to install it again.

Follow  the following pages to install WSL in a system or non-system drive on your Windows 10 and also install Hadoop 3.2.0 in the WSL.

Now let’s start to install Apache Sqoop v1.4.7.

Download binary package

Go to Sqoop official website to find a download URL for Sqoop 1.

In the mirror site, there are usually two binaries available:

  • sqoop-[version].bin__hadoop-2.6.0.tar.gz
  • sqoop-[version].tar.gz

We are after the first one.

To me, the recommended location is http://apache.mirror.digitalpacific.com.au/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz.

Download the package using the following command:

wget http://apache.mirror.digitalpacific.com.au/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

Unzip the binary package

Unpack the binary using the following command:

tar -xvzf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C ~/hadoop

Setup environment variables

Setup SQOOP_HOME environment variables and also add the bin subfolder into PATH variable.

Run the following command to change .bashrc file:

vi ~/.bashrc

Add the following lines to the end of the file:

export SQOOP_HOME=~/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0                                                             

export PATH=$SQOOP_HOME/bin:$PATH

Source the modified file to make it effective:

source  ~/.bashrc

Test configurations

Run the following command to test the configurations:

$SQOOP_HOME/bin/configure-sqoop

Example output:

Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

Print out version

The above warning messages are printed out because we have not configured HBase, HCAT, Accumulo or Zookeeper in the WSL distro yet.

$ sqoop version
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /home/tangr/hadoop/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
/home/tangr/hadoop/hadoop-3.2.0/libexec/hadoop-functions.sh: line 2364: HADOOP_ORG.APACHE.SQOOP.SQOOP_USER: bad substitution
/home/tangr/hadoop/hadoop-3.2.0/libexec/hadoop-functions.sh: line 2459: HADOOP_ORG.APACHE.SQOOP.SQOOP_OPTS: bad substitution
2019-05-19 12:04:18,429 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017

info Last modified by Administrator at 2 months ago copyright This page is subject to Site terms.
Like this article?
Share on

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

Kontext Column

Created for everyone to publish data, programming and cloud related articles.
Follow three steps to create your columns.


Learn more arrow_forward

More from Kontext

local_offer linux local_offer hadoop local_offer hdfs local_offer yarn local_offer big-data-on-linux

visibility 608
thumb_up 0
access_time 2 months ago

This article provides step-by-step guidance to install Hadoop 3.3.0 on Linux such as Debian, Ubuntu, Red Hat, openSUSE, etc.  Hadoop 3.3.0 was released on July 14 2020. It is the first release of Apache Hadoop 3.3 line. There are significant changes compared with Hadoop 3.2.0, such as ...

Install Hadoop 3.3.0 on Windows 10 using WSL

local_offer linux local_offer hadoop local_offer WSL local_offer big-data-on-wsl

visibility 1274
thumb_up 4
access_time 2 months ago

Hadoop 3.3.0 was released on July 14 2020. It is the first release of Apache Hadoop 3.3 line. There are significant changes compared with Hadoop 3.2.0, such as Java 11 runtime support, protobuf upgrade to 3.7.1, scheduling of opportunistic containers, non-volatile SCM support in HDFS ...

Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

local_offer hadoop local_offer hive local_offer WSL local_offer big-data-on-wsl

visibility 4334
thumb_up 0
access_time 2 years ago

Previously, I demonstrated how to configured Apache Hive 3.0.0 on Windows 10. Apache Hive 3.0.0 Installation on Windows 10 Step by Step Guide On this page, I’m going to show you how to install the latest version Apache Hive 3.1.1 on Windows 10 using Windows Subsystem for Linux (WSL) Ubuntu ...

About column

Apache Sqoop, a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases.

rss_feed Subscribe RSS