Hadoop Daemon Log Files Location

event 2021-01-21 visibility 1,034 comment 0 insights
ios_share
Twitter Facebook LinkedIn
more_vert
insights Stats
Raymond Raymond
Raymond
Raymond
articleArticles 578
imageDiagrams 58
commentComments 303
loyaltyKontext Points 6373
 Hadoop, Hive & HBase

Articles about Apache Hadoop, Hive and HBase installation, performance tuning and general tutorials.

*The yellow elephant logo is a registered trademark of Apache Hadoop.

About Hadoop daemon services

To start a Hadoop service, we use scripts under sbin folder of Hadoop home folder. 

For example, the following two commands start Hadoop HDFS services (namenode and datanode) on Windows and UNIX-alike systems respectively. 

%HADOOP_HOME%\sbin\start-dfs.cmd

%HADOOP_HOME%\sbin\start-dfs.sh
Sometimes, these daemon services may fail to start. To investigate the root cause, we need to look into log folder. 

HADOOP_LOG_DIR

This environment variable is used for Hadoop log directory.

By default the values are:

  • Windows:
    @rem Where log files are stored.  %HADOOP_HOME%/logs by default.
    @rem set HADOOP_LOG_DIR=%HADOOP_LOG_DIR%\%USERNAME%

    The logs are located in folder %HADOOP_HOME%/logs.

  • Linux:
    # Where (primarily) daemon log files are stored.
# ${HADOOP_HOME}/logs by default.
# Java property: hadoop.log.dir
# export HADOOP_LOG_DIR=${HADOOP_HOME}/logs

    The logs are located in folder ${HADOOP_HOME}/logs.

More from Kontext
comment Comments
No comments yet.

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts