access_time 10 months ago languageEnglish
more_vert

Apache Hive 3.1.2 Installation on Windows 10

visibility 3,197 comment 21
Hive 3.1.2 was released on 26th Aug 2019. It is still the latest 3.x release and works with Hadoop 3.x.y releases. In this article, I’m going to provide step by step instructions about installing Hive 3.1.2 on Windows 10. * Logos are registered trademarks of Apache Hive and Microsoft Windows.
info Last modified by Raymond 3 months ago
thumb_up 2

comment Comments

7 months ago link more_vert

Hi Raymond,

Thanks for sharing the links for installation of additional Hadoop softwares.

I started with installing Hive.

When I run the command $HIVE_HOME/bin/hive --service metastore &

the command keeps running and I do not get the command prompt back.

So I opened another Cygwin terminal to run the below command.

$HIVE_HOME/bin/hive --service hiveserver2 start &

Then I opened a 3rd Cygwin terminal to run the commands 

$HIVE_HOME/bin/beeline -u jdbc:hive2://localhost:10000

$HIVE_HOME/bin/hive

and got the below message.

Could you please guide me if I need to change it ?


7 months ago link more_vert

Hi Ankit,

If you are using default derby as metastore, you cannot open multiple sessions. When you run the command yes you can still see logs but you can press enter so that you can input commands without starting a new session. 

If you prefer to allow multiple sessions, please follow the link to use a remote metastore like SQL Server. 

format_quote

person Ankit access_time 7 months ago
Re: Apache Hive 3.1.2 Installation on Windows 10

Hi Raymond,

Thanks for sharing the links for installation of additional Hadoop softwares.

I started with installing Hive.

When I run the command $HIVE_HOME/bin/hive --service metastore &

the command keeps running and I do not get the command prompt back.

So I opened another Cygwin terminal to run the below command.

$HIVE_HOME/bin/hive --service hiveserver2 start &

Then I opened a 3rd Cygwin terminal to run the commands 

$HIVE_HOME/bin/beeline -u jdbc:hive2://localhost:10000

$HIVE_HOME/bin/hive

and got the below message.

Could you please guide me if I need to change it ?


7 months ago link more_vert

Hi Raymond,

I faced issues during Hive installation, so did the complete process again and found that Schematool step has some issues.

Please see the below screen-shot. It says that "Found configuration file null" conf.Hiveconf.

I guess that's why my metastore step has issues.

There are a lot of blank lines in the schematool log.

If you know what can be done to fix this, could you please help me?


format_quote

person Raymond access_time 7 months ago
Re: Apache Hive 3.1.2 Installation on Windows 10

Hi Ankit,

If you are using default derby as metastore, you cannot open multiple sessions. When you run the command yes you can still see logs but you can press enter so that you can input commands without starting a new session. 

If you prefer to allow multiple sessions, please follow the link to use a remote metastore like SQL Server. 

7 months ago link more_vert

For some of my previous configurations, the logs were not printed out either.

Can you please check whether there is a folder named metastore_db created? It should be ok if the folder exists and have content.

To make everything easy, I would suggest to install SQL Server express or Developer edition and then configure your Hive metastore to use SQL Server:

https://kontext.tech/column/hadoop/302/configure-a-sql-server-database-as-remote-hive-metastore 

If it still doesn't work, please send an email to enquiry[at]kontext.tech and I can arrange a Teams meeting with you to debug. 

format_quote

person Ankit access_time 7 months ago
Re: Apache Hive 3.1.2 Installation on Windows 10

Hi Raymond,

I faced issues during Hive installation, so did the complete process again and found that Schematool step has some issues.

Please see the below screen-shot. It says that "Found configuration file null" conf.Hiveconf.

I guess that's why my metastore step has issues.

There are a lot of blank lines in the schematool log.

If you know what can be done to fix this, could you please help me?


6 months ago link more_vert

Hi Raymond,

Thank you so much for your help.

I will first try to find out the issue on my own. If it doesn't work, would send email to enquiry mailbox.

I installed Hadoop 3.3.0 on my system and then directly tried to install Hive 3.1.2.

Could you please confirm if that sequence is fine or I need to install something in between Hadoop and hive?

Thanks again :)

Regards,

Ankit

format_quote

person Raymond access_time 7 months ago
Re: Apache Hive 3.1.2 Installation on Windows 10

For some of my previous configurations, the logs were not printed out either.

Can you please check whether there is a folder named metastore_db created? It should be ok if the folder exists and have content.

To make everything easy, I would suggest to install SQL Server express or Developer edition and then configure your Hive metastore to use SQL Server:

https://kontext.tech/column/hadoop/302/configure-a-sql-server-database-as-remote-hive-metastore 

If it still doesn't work, please send an email to enquiry[at]kontext.tech and I can arrange a Teams meeting with you to debug. 

6 months ago link more_vert

The sequence is fine. Hadoop needs to be installed first before you install Hive as Hive utilizes HDFS for data store in a on-premise cluster.

format_quote

person Ankit access_time 6 months ago
Re: Apache Hive 3.1.2 Installation on Windows 10

Hi Raymond,

Thank you so much for your help.

I will first try to find out the issue on my own. If it doesn't work, would send email to enquiry mailbox.

I installed Hadoop 3.3.0 on my system and then directly tried to install Hive 3.1.2.

Could you please confirm if that sequence is fine or I need to install something in between Hadoop and hive?

Thanks again :)

Regards,

Ankit

5 months ago link more_vert

Hi, thanks for the article.

When I run the command "$HIVE_HOME/bin/schematool -dbType derby -initSchema", I am getting the error .. Error: Could not find or load main class ???. (??? - is my user name).

Can you please help how to resolve it.

5 months ago link more_vert

Hi, did you follow all the steps exactly? As we have to use Cygwin to run the commands since the Command Prompt script version is not available for the latest Hive, all the steps I included in the guide is critical. It looks like the script cannot find your derby JAR files due to Java classpath (Hadoop/Hive environment variables are not setup correctly). The folder where you run the schema init script is also important to use derby since it is a file based database.

Considering derby is not good for concurrency for Hive connections, I suggest to use a remote metastore like SQL Server: Configure a SQL Server Database as Remote Hive Metastore

Alternatively, to save all the troubles, I highly recommend you follow my newly publish article Apache Hive 3.1.2 Installation on Linux Guide. That article will configure Hadoop 3.3.0 and Hive 3.1.2 in a WSL environment on Windows 10. It also includes steps to install MySQL as remote metastore for your Hive data warehouse. 

Let me know if you encounter errors in WSL.

-Raymond

format_quote

person Renganathan access_time 5 months ago
Re: Apache Hive 3.1.2 Installation on Windows 10

Hi, thanks for the article.

When I run the command "$HIVE_HOME/bin/schematool -dbType derby -initSchema", I am getting the error .. Error: Could not find or load main class ???. (??? - is my user name).

Can you please help how to resolve it.

5 months ago link more_vert

Hi Raymond,

Thanks for looking into this. I see that I have followed all the steps. I do have derby JAR files under apache-hive-3.1.2-bin\lib. I use Cygwin to setup the environment variables and run the schematool in it.

I started debugging the shell scripts. The class not found occurs in hive.ksh (located in the bin folder) at the last line --> $TORUN "$@"

It seems that TORUN resolves to schemaTool (capital 'T'). 

I am not sure how to proceed further to identify the issue root cause.

Thanks!

format_quote

person Raymond access_time 5 months ago
Re: Apache Hive 3.1.2 Installation on Windows 10

Hi, did you follow all the steps exactly? As we have to use Cygwin to run the commands since the Command Prompt script version is not available for the latest Hive, all the steps I included in the guide is critical. It looks like the script cannot find your derby JAR files due to Java classpath (Hadoop/Hive environment variables are not setup correctly). The folder where you run the schema init script is also important to use derby since it is a file based database.

Considering derby is not good for concurrency for Hive connections, I suggest to use a remote metastore like SQL Server: Configure a SQL Server Database as Remote Hive Metastore

Alternatively, to save all the troubles, I highly recommend you follow my newly publish article Apache Hive 3.1.2 Installation on Linux Guide. That article will configure Hadoop 3.3.0 and Hive 3.1.2 in a WSL environment on Windows 10. It also includes steps to install MySQL as remote metastore for your Hive data warehouse. 

Let me know if you encounter errors in WSL.

-Raymond

5 months ago link more_vert

Hi Renganathan,

For all these environment variables setup, did you add all of them into ~/.bashrc and run command 'source ~/.bashrc'?

export HADOOP_HOME='/cygdrive/f/big-data/hadoop-3.3.0'
export PATH=$PATH:$HADOOP_HOME/bin
export HIVE_HOME='/cygdrive/f/big-data/apache-hive-3.1.2-bin'
export PATH=$PATH:$HIVE_HOME/bin
export HADOOP_CLASSPATH=$(hadoop classpath)
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HIVE_HOME/lib/*.jar

Can you also run the following command line in cygwin and paste the output here?

echo $HADOOP_CLASSPATH

And also can you paste all your detailed error here if it is ok?

The schema tool requires the following JAVA class to be present and its JAR file needs to be in the Java classpath. 

schemaTool() {
  HIVE_OPTS=''
  CLASS=org.apache.hive.beeline.HiveSchemaTool
  execHiveCmd $CLASS "$@"
}

schemaTool_help () {
  HIVE_OPTS=''
  CLASS=org.apache.hive.beeline.HiveSchemaTool
  execHiveCmd $CLASS "--help"
}

format_quote

person Renganathan access_time 5 months ago
Re: Apache Hive 3.1.2 Installation on Windows 10

Hi Raymond,

Thanks for looking into this. I see that I have followed all the steps. I do have derby JAR files under apache-hive-3.1.2-bin\lib. I use Cygwin to setup the environment variables and run the schematool in it.

I started debugging the shell scripts. The class not found occurs in hive.ksh (located in the bin folder) at the last line --> $TORUN "$@"

It seems that TORUN resolves to schemaTool (capital 'T'). 

I am not sure how to proceed further to identify the issue root cause.

Thanks!

Forum discussions for column Hadoop.

Follow Kontext

Get our latest updates on LinkedIn or Twitter.

Want to contribute on Kontext to help others?

Learn more