access_time 3 years ago languageEnglish
more_vert

Copy Files from Hadoop HDFS to Local

visibility 2,486 comment 0

Copy file from HDFS to local

Use the following command:

hadoop fs [-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]

For example, copy a file from /hdfs-file.txt in HDFS to local /tmp/ using the following command:

hadoop fs -copyToLocal /hdfs-file.txt /tmp/hdfs-file.txt

HDFS CLI

If you forgot any HDFS commands, you can use the following command to list all of them:

hadoop fs

The output will generate all the commands (not all of them are implemented yet):

hadoop fs
Usage: hadoop fs [generic options]
         [-appendToFile <localsrc> ... <dst>]
         [-cat [-ignoreCrc] <src> ...]
         [-checksum <src> ...]
         [-chgrp [-R] GROUP PATH...]
         [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
         [-chown [-R] [OWNER][:[GROUP]] PATH...]
         [-copyFromLocal [-f] [-p] [-l] [-d] [-t <thread count>] <localsrc> ... <dst>]
         [-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
         [-count [-q] [-h] [-v] [-t [<storage type>]] [-u] [-x] [-e] <path> ...]
         [-cp [-f] [-p | -p[topax]] [-d] <src> ... <dst>]
         [-createSnapshot <snapshotDir> [<snapshotName>]]
         [-deleteSnapshot <snapshotDir> <snapshotName>]
         [-df [-h] [<path> ...]]
         [-du [-s] [-h] [-v] [-x] <path> ...]
         [-expunge]
         [-find <path> ... <expression> ...]
         [-get [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
         [-getfacl [-R] <path>]
         [-getfattr [-R] {-n name | -d} [-e en] <path>]
         [-getmerge [-nl] [-skip-empty-file] <src> <localdst>]
         [-help [cmd ...]]
         [-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [-e] [<path> ...]]
         [-mkdir [-p] <path> ...]
         [-moveFromLocal <localsrc> ... <dst>]
         [-moveToLocal <src> <localdst>]
         [-mv <src> ... <dst>]
         [-put [-f] [-p] [-l] [-d] <localsrc> ... <dst>]
         [-renameSnapshot <snapshotDir> <oldName> <newName>]
         [-rm [-f] [-r|-R] [-skipTrash] [-safely] <src> ...]
         [-rmdir [--ignore-fail-on-non-empty] <dir> ...]
         [-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
         [-setfattr {-n name [-v value] | -x name} <path>]
         [-setrep [-R] [-w] <rep> <path> ...]
         [-stat [format] <path> ...]
         [-tail [-f] <file>]
         [-test -[defsz] <path>]
         [-text [-ignoreCrc] <src> ...]
         [-touchz <path> ...]
         [-truncate [-w] <length> <path> ...]
         [-usage [cmd ...]]

Generic options supported are:
-conf <configuration file>        specify an application configuration file
-D <property=value>               define a value for a given property
-fs <file:///|hdfs://namenode:port> specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.
-jt <local|resourcemanager:port>  specify a ResourceManager
-files <file1,...>                specify a comma-separated list of files to be copied to the map reduce cluster
-libjars <jar1,...>               specify a comma-separated list of jar files to be included in the classpath
-archives <archive1,...>          specify a comma-separated list of archives to be unarchived on the compute machines

The general command line syntax is:
command [genericOptions] [commandOptions]

If you want to view the detailed syntax for any command, you can try the following command:

hadoop fs -help [command]

For example, run command ‘hadoop fs -help copyToLocal’ will generate the following output:

hadoop fs -help copyToLocal
-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst> :
   Identical to the -get command.

info Last modified by Raymond 3 years ago copyright This page is subject to Site terms.
Like this article?
Share on

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

Follow Kontext

Get our latest updates on LinkedIn.

Want to contribute on Kontext to help others?

Learn more

More from Kontext

Apache Hive 3.1.2 Installation on Linux Guide
visibility 1920
thumb_up 0
access_time 6 months ago
visibility 775
thumb_up 0
access_time 3 years ago