Hadoop Daemon Log Files Location

Raymond Raymond event 2021-01-21 visibility 1,068
more_vert

About Hadoop daemon services

To start a Hadoop service, we use scripts under sbin folder of Hadoop home folder. 

For example, the following two commands start Hadoop HDFS services (namenode and datanode) on Windows and UNIX-alike systems respectively. 

%HADOOP_HOME%\sbin\start-dfs.cmd
%HADOOP_HOME%\sbin\start-dfs.sh
Sometimes, these daemon services may fail to start. To investigate the root cause, we need to look into log folder. 

HADOOP_LOG_DIR

This environment variable is used for Hadoop log directory.

By default the values are:

  • Windows:
    @rem Where log files are stored.  %HADOOP_HOME%/logs by default.
    @rem set HADOOP_LOG_DIR=%HADOOP_LOG_DIR%\%USERNAME%

    The logs are located in folder %HADOOP_HOME%/logs.

  • Linux:
    # Where (primarily) daemon log files are stored.
    # ${HADOOP_HOME}/logs by default.
    # Java property: hadoop.log.dir
    # export HADOOP_LOG_DIR=${HADOOP_HOME}/logs

    The logs are located in folder ${HADOOP_HOME}/logs.

More from Kontext
comment Comments
No comments yet.

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts