Copy file from HDFS to local

Use the following command:

hadoop fs [-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]

For example, copy a file from /hdfs-file.txt in HDFS to local /tmp/ using the following command:

hadoop fs -copyToLocal /hdfs-file.txt /tmp/hdfs-file.txt

HDFS CLI

If you forgot any HDFS commands, you can use the following command to list all of them:

hadoop fs

The output will generate all the commands (not all of them are implemented yet):

hadoop fs
Usage: hadoop fs [generic options]
         [-appendToFile <localsrc> ... <dst>]
         [-cat [-ignoreCrc] <src> ...]
         [-checksum <src> ...]
         [-chgrp [-R] GROUP PATH...]
         [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
         [-chown [-R] [OWNER][:[GROUP]] PATH...]
         [-copyFromLocal [-f] [-p] [-l] [-d] [-t <thread count>] <localsrc> ... <dst>]
         [-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
         [-count [-q] [-h] [-v] [-t [<storage type>]] [-u] [-x] [-e] <path> ...]
         [-cp [-f] [-p | -p[topax]] [-d] <src> ... <dst>]
         [-createSnapshot <snapshotDir> [<snapshotName>]]
         [-deleteSnapshot <snapshotDir> <snapshotName>]
         [-df [-h] [<path> ...]]
         [-du [-s] [-h] [-v] [-x] <path> ...]
         [-expunge]
         [-find <path> ... <expression> ...]
         [-get [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
         [-getfacl [-R] <path>]
         [-getfattr [-R] {-n name | -d} [-e en] <path>]
         [-getmerge [-nl] [-skip-empty-file] <src> <localdst>]
         [-help [cmd ...]]
         [-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [-e] [<path> ...]]
         [-mkdir [-p] <path> ...]
         [-moveFromLocal <localsrc> ... <dst>]
         [-moveToLocal <src> <localdst>]
         [-mv <src> ... <dst>]
         [-put [-f] [-p] [-l] [-d] <localsrc> ... <dst>]
         [-renameSnapshot <snapshotDir> <oldName> <newName>]
         [-rm [-f] [-r|-R] [-skipTrash] [-safely] <src> ...]
         [-rmdir [--ignore-fail-on-non-empty] <dir> ...]
         [-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
         [-setfattr {-n name [-v value] | -x name} <path>]
         [-setrep [-R] [-w] <rep> <path> ...]
         [-stat [format] <path> ...]
         [-tail [-f] <file>]
         [-test -[defsz] <path>]
         [-text [-ignoreCrc] <src> ...]
         [-touchz <path> ...]
         [-truncate [-w] <length> <path> ...]
         [-usage [cmd ...]]

Generic options supported are:
-conf <configuration file>        specify an application configuration file
-D <property=value>               define a value for a given property
-fs <file:///|hdfs://namenode:port> specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.
-jt <local|resourcemanager:port>  specify a ResourceManager
-files <file1,...>                specify a comma-separated list of files to be copied to the map reduce cluster
-libjars <jar1,...>               specify a comma-separated list of jar files to be included in the classpath
-archives <archive1,...>          specify a comma-separated list of archives to be unarchived on the compute machines

The general command line syntax is:
command [genericOptions] [commandOptions]

If you want to view the detailed syntax for any command, you can try the following command:

hadoop fs -help [command]

For example, run command ‘hadoop fs -help copyToLocal’ will generate the following output:

hadoop fs -help copyToLocal
-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst> :
   Identical to the -get command.

info Last modified by Raymond at 2 years ago * This page is subject to Site terms.

More from Kontext

local_offer hadoop local_offer hive local_offer Java

visibility 398
thumb_up 1
access_time 3 months ago

When I was configuring Hive 3.0.0 in Hadoop 3.2.1 environment, I encountered the following error: Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V Ro...

open_in_new Hadoop

local_offer jupyter-notebook local_offer hdfs

visibility 258
thumb_up 0
access_time 4 months ago

Jupyter notebook service can be started in most of operating system. In the system where Hadoop clients are available, you can also easily ingest data into HDFS (Hadoop Distributed File System) using HDFS CLIs.  *Python 3 Kernel is used in the following examples. List files in H...

open_in_new Code snippets

local_offer hdfs local_offer hadoop local_offer windows

visibility 354
thumb_up 0
access_time 4 months ago

Network Attached Storage are commonly used in many enterprises where files are stored remotely on those servers.  They typically provide access to files using network file sharing protocols such as  ...

open_in_new Hadoop

local_offer hive local_offer hdfs

visibility 115
thumb_up 0
access_time 5 months ago

In Hive, there are two types of tables can be created - internal and external table. Internal tables are also called managed tables. Different features are available to different types. This article lists some of the common differences.  Internal table By default, Hive creates ...

open_in_new Hadoop

info About author

comment Comments (0)

comment Add comment

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

No comments yet.

Dark theme mode

Dark theme mode is available on Kontext.

Learn more arrow_forward

Kontext Column

Created for everyone to publish data, programming and cloud related articles. Follow three steps to create your columns.


Learn more arrow_forward