Check HDFS folder size in Shell / Hadoop

access_time 2 years ago visibility292 comment 0

Hadoop provides a number of CLIs that can be used to perform many tasks/activities. This code snippet shows you how to check file/folder size in HDFS.

hdfs dfs command can be used to check disk usage in HDFS.

Syntax:

hdfs dfs -du -h ${hdfs_path}

Code snippet

hdfs dfs -du -h /path/to/your/folder
info Last modified by Raymond 2 years ago copyright This page is subject to Site terms.
Like this article?
Share on

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

Follow Kontext

Get our latest updates on LinkedIn or Twitter.

Want to publish your article on Kontext?

Learn more

More from Kontext

Install Hadoop 3.3.0 on Windows 10 Step by Step Guide
visibility 6605
thumb_up 7
access_time 6 months ago

This detailed step-by-step guide shows you how to install the latest Hadoop v3.3.0 on Windows 10. It leverages Hadoop 3.3.0 winutils tool and WSL is not required. This version was released on July 14 2020. It is the first release of Apache Hadoop 3.3 line. There are significant changes compared with Hadoop 3.2.0, such as Java 11 runtime support, protobuf upgrade to 3.7.1, scheduling of opportunistic containers, non-volatile SCM support in HDFS cache directives, etc.

visibility 55
thumb_up 0
access_time 2 years ago

The following code snippet shows how to list and kill Hadoop jobs including (MapReduce and YARN jobs).

visibility 1636
thumb_up 0
access_time 2 years ago

After finishing installation Hadoop 3.0.0 in my Windows: Install Hadoop 3.0.0 in Windows (Single Node) , I got the following error after I formated the name node several times. The following error is thrown out when I tried to start Hadoop HDFS. 2018-02-19 22:02:06,848 WARN common.Storage ...