This site uses cookies to deliver our services. By using this site, you acknowledge that you have read and understand our Cookie and Privacy policy. Your use of Kontext website is subject to this policy. Allow Cookies and Dismiss

Password Security Solution for Sqoop

13 views 0 comments last modified about 13 days ago Raymond Tang

lite-log hadoop sqoop

In Sqoop, there are multiple approaches to pass in passwords for RDBMS.

Options

Option 1 - clear password through --password argument

sqoop [subcommand] --username user --password pwd

This is the weakest approach as password is exposed directly in the command line.

Option 2 - interactive password through -P argument

sqoop [subcommand] --username user -P

Password needs to be manually input interactively. You cannot use this approach to schedule the job.

Option 3 - storing password in file through --password-file argument

sqoop [subcommand] --username user --password-file mypasswordfile.path

Password is still clearly stored in a file which is weak though better than option 1.

Recommended option - Hadoop credential

From Hadoop 2.2.0, we can use hadoop credential command to create password alias and then use it in Sqoop or other tools.

Generate the password in Java key store

Java key store is one of the supported providers.

#Store the password in HDFS

hadoop credential create mydatabase.password -provider jceks://hdfs/user/hue/mypwd.jceks

# Store the password locally

hadoop credential create mydatabase.password -provider jceks://file/home/user/mypwd.jceks

Use the password alias

sqoop [subcommand] \
-Dhadoop.security.credential.provider.path=jceks://hdfs/user/hue/mypwd.jceks\
--verbose \
--username user \
--password-alias mydatabase.password \

….

In this way, clear password is not exposed directly.

Related pages

Install Hadoop 3.0.0 in Windows (Single Node)

9221 views   14 comments last modified about 11 months ago

This page summarizes the steps to install Hadoop 3.0.0 in your Windows environment. Reference page: https://wiki.apache.org/hadoop/Hadoop2OnWindows ...

View detail

Resolve Hadoop RemoteException - Name node is in safe mode

171 views   0 comments last modified about 9 months ago

In Safe Mode, the HDFS cluster is read-only. After completion of block replication maintenance activity, the name node leaves safe mode automatically. If you try to delete files in safe mode, the following exception may raise: org.apache.hadoop.ipc.RemoteException(org.apac...

View detail

Configure Sqoop in a Edge Node of Hadoop Cluster

822 views   0 comments last modified about 9 months ago

This page continues with the following documentation about configuring a Hadoop multi-nodes cluster via adding a new edge node to configure administration or client tools. ...

View detail

Configure Sqoop in a Edge Node of Hadoop Cluster

822 views   0 comments last modified about 9 months ago

This page continues with the following documentation about configuring a Hadoop multi-nodes cluster via adding a new edge node to configure administration or client tools. ...

View detail

Configure YARN and MapReduce Resources in Hadoop Cluster

513 views   0 comments last modified about 9 months ago

When configuring YARN and MapReduce in Hadoop cluster, it is very important to configure the memory and virtual processors correctly. If the configurations are incorrect, the nodes may not be able to start properly and the applications may not be able to run successfully. For example...

View detail

Configure Hadoop 3.1.0 in a Multi Node Cluster

3331 views   0 comments last modified about 9 months ago

Previously, I summarized the steps to install Hadoop in a single node Windows machine. Install Hadoop 3.0.0 in Windows (Single Node) In this page, I...

View detail

Add comment

Please login first to add comments.  Log in New user?  Register

Comments (0)

No comments yet.

Contacts

  • enquiry[at]kontext.tech

Subscribe