In Sqoop, there are multiple approaches to pass in passwords for RDBMS.
sqoop [subcommand] --username user --password pwd
This is the weakest approach as password is exposed directly in the command line.
sqoop [subcommand] --username user -P
Password needs to be manually input interactively. You cannot use this approach to schedule the job.
sqoop [subcommand] --username user --password-file mypasswordfile.path
Password is still clearly stored in a file which is weak though better than option 1.
From Hadoop 2.2.0, we can use hadoop credential command to create password alias and then use it in Sqoop or other tools.
Generate the password in Java key store
Java key store is one of the supported providers.
#Store the password in HDFS
hadoop credential create mydatabase.password -provider jceks://hdfs/user/hue/mypwd.jceks
# Store the password locally
hadoop credential create mydatabase.password -provider jceks://file/home/user/mypwd.jceks
sqoop [subcommand] \
--username user \
--password-alias mydatabase.password \
In this way, clear password is not exposed directly.
In Safe Mode, the HDFS cluster is read-only. After completion of block replication maintenance activity, the name node leaves safe mode automatically. If you try to delete files in safe mode, the following exception may raise: org.apache.hadoop.ipc.RemoteException(org.apac...View detail
When configuring YARN and MapReduce in Hadoop cluster, it is very important to configure the memory and virtual processors correctly. If the configurations are incorrect, the nodes may not be able to start properly and the applications may not be able to run successfully. For example...View detail