visibility 10,395 comment 20 access_time 4 years ago languageEnglish
share
Twitter Facebook LinkedIn
more_vert
Raymond Raymond
articleArticles 549
imageDiagrams 49
codeCode 3
chat_bubble_outlineThreads 8
commentComments 265
loyaltyKontext Points 6055
account_circleProfile

Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

Previously, I demonstrated how to configured Apache Hive 3.0.0 on Windows 10. Apache Hive 3.0.0 Installation on Windows 10 Step by Step Guide On this page, I’m going to show you how to install the latest version Apache Hive 3.1.1 on Windows 10 using Windows Subsystem for Linux (WSL) Ubuntu ...
open_in_new View full article
info Last modified by Administrator 7 months ago
thumb_up 3
reply Reply

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts

comment Comments
1-10 of 20
2 years ago link more_vert
G Guy
articleArticles 0
imageDiagrams 0
codeCode 0
chat_bubble_outlineThreads 0
commentComments 15
loyaltyKontext Points 15
#409 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

thank you :)


the walk through helped me alot.

now waiting for impala guide !


format_quote

person Raymond access_time 2 years ago
Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

Thanks for summarizing this, Guy.

For the Hive 3.1.2 installation guide on this site, I've already incorporated it.

Cheers, Raymond

reply Reply
2 years ago link more_vert
Raymond Raymond
articleArticles 549
imageDiagrams 49
codeCode 3
chat_bubble_outlineThreads 8
commentComments 265
loyaltyKontext Points 6055
account_circleProfile
#408 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

I have not got time to create the Impala guide yet but I've created one for HBase just in case you are interested.

Install HBase in WSL - Pseudo-Distributed Mode


format_quote

person Guy access_time 3 years ago
Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

yes please ! to both.

i will add the steps and fixes i did.

and would very much be happy with impala walk through.

Thank you

Guy

reply Reply
2 years ago link more_vert
Raymond Raymond
articleArticles 549
imageDiagrams 49
codeCode 3
chat_bubble_outlineThreads 8
commentComments 265
loyaltyKontext Points 6055
account_circleProfile
#407 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

Thanks for summarizing this, Guy.

For the Hive 3.1.2 installation guide on this site, I've already incorporated it.

Cheers, Raymond

format_quote

person Guy access_time 3 years ago
Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

some missing steps:


just before running the initSchema, need to do 3 things:

1. need to copy the conf file:

cp $HIVE_HOME/conf/hive-default.xml.template $HIVE_HOME/conf/hive-site.xml


2. need to remove a typo from line 3215 of the file: 

nano or vim /home/guy/hadoop/apache-hive-3.1.2-bin/conf/hive-site.xml and remore the letters in the middle of the line ( Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire Exclusive locks for <<&#8;>> transactional tables.  This ensures that inserts (w/o overwrite) running concurrently)

3. also need to remove  guava-19.0.jar from   apache-hive-3.1.2/lib and copy the current one from the folder hadoop-3.3.0/share/hadoop/common/lib/


after that you can run the init schema for the derbyDB

reply Reply
3 years ago link more_vert
G Guy
articleArticles 0
imageDiagrams 0
codeCode 0
chat_bubble_outlineThreads 0
commentComments 15
loyaltyKontext Points 15
#406 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

some missing steps:


just before running the initSchema, need to do 3 things:

1. need to copy the conf file:

cp $HIVE_HOME/conf/hive-default.xml.template $HIVE_HOME/conf/hive-site.xml


2. need to remove a typo from line 3215 of the file: &#8;

nano or vim /home/guy/hadoop/apache-hive-3.1.2-bin/conf/hive-site.xml and remore the letters in the middle of the line ( Ensures commands with OVERWRITE (such as INSERT OVERWRITE) acquire Exclusive locks for <<&#8;>> transactional tables.  This ensures that inserts (w/o overwrite) running concurrently)

3. also need to remove  guava-19.0.jar from   apache-hive-3.1.2/lib and copy the current one from the folder hadoop-3.3.0/share/hadoop/common/lib/


after that you can run the init schema for the derbyDB

reply Reply
3 years ago link more_vert
G Guy
articleArticles 0
imageDiagrams 0
codeCode 0
chat_bubble_outlineThreads 0
commentComments 15
loyaltyKontext Points 15
#405 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

yes please ! to both.

i will add the steps and fixes i did.

and would very much be happy with impala walk through.

Thank you

Guy

format_quote

person Raymond access_time 3 years ago
Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

I'm glad it worked. I will try to publish one installation guide for Impala when I get time. If you are interested, you can also try and publish your steps on Kontext too. 

Cheers,

Raymond

reply Reply
3 years ago link more_vert
Raymond Raymond
articleArticles 549
imageDiagrams 49
codeCode 3
chat_bubble_outlineThreads 8
commentComments 265
loyaltyKontext Points 6055
account_circleProfile
#404 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

I'm glad it worked. I will try to publish one installation guide for Impala when I get time. If you are interested, you can also try and publish your steps on Kontext too. 

Cheers,

Raymond

format_quote

person Guy access_time 3 years ago
Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

hello

yes, after fixing some stuff .. mentioned in my previous message, i am able now to connect.

the problems ( 5 of them ) where consisting on missing information.


thank you


can i also ask if you can add a tutorial of how to implement impala on windows WSL ? 

:)

reply Reply
3 years ago link more_vert
G Guy
articleArticles 0
imageDiagrams 0
codeCode 0
chat_bubble_outlineThreads 0
commentComments 15
loyaltyKontext Points 15
#403 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

hello

yes, after fixing some stuff .. mentioned in my previous message, i am able now to connect.

the problems ( 5 of them ) where consisting on missing information.


thank you


can i also ask if you can add a tutorial of how to implement impala on windows WSL ? 

:)

format_quote

person Raymond access_time 3 years ago
Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

Hi Guy,

Can you confirm whether you can connect to HiveServer2 or not?

It takes a little bit time for the web UI service to get started. 

Can you run the following command to see if two RunJar process are running (One is for metastore and another for HiverServer2)?

jps -mlv

-Raymond 

reply Reply
3 years ago link more_vert
Raymond Raymond
articleArticles 549
imageDiagrams 49
codeCode 3
chat_bubble_outlineThreads 8
commentComments 265
loyaltyKontext Points 6055
account_circleProfile
#402 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

Hi Guy,

Can you confirm whether you can connect to HiveServer2 or not?

It takes a little bit time for the web UI service to get started. 

Can you run the following command to see if two RunJar process are running (One is for metastore and another for HiverServer2)?

jps -mlv

-Raymond 

format_quote

person Guy access_time 3 years ago
Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

one more thing, it appears that there are two bugs in the apache-hive-3.1.2

1. there is a typo in line 3215 of file hive-site.xml ( needed to be deleted )

2. the guava jar should be the same version as the hadoop one.

after fixing those 2 things.  ( and copying the default xml file to hive-site.xml ) i was able to run the :

$HIVE_HOME/bin/schematool -dbType derby -initSchema

script successfully

so now we are back to why i cannot log into the web page:

http://localhost:10002/

as two services are running

$HIVE_HOME/bin/hive --service metastore &
$HIVE_HOME/bin/hive --service hiveserver2 &


and no error in the log files.


thank you

reply Reply
3 years ago link more_vert
G Guy
articleArticles 0
imageDiagrams 0
codeCode 0
chat_bubble_outlineThreads 0
commentComments 15
loyaltyKontext Points 15
#401 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

one more thing, it appears that there are two bugs in the apache-hive-3.1.2

1. there is a typo in line 3215 of file hive-site.xml ( needed to be deleted )

2. the guava jar should be the same version as the hadoop one.

after fixing those 2 things.  ( and copying the default xml file to hive-site.xml ) i was able to run the :

$HIVE_HOME/bin/schematool -dbType derby -initSchema

script successfully

so now we are back to why i cannot log into the web page:

http://localhost:10002/

as two services are running

$HIVE_HOME/bin/hive --service metastore &
$HIVE_HOME/bin/hive --service hiveserver2 &


and no error in the log files.


thank you

format_quote

person Raymond access_time 3 years ago
Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

Hi Guy,

Most likely it is because that your Hive Metastore database schema is not correct, for example, init schema operation was not done properly.

Can you conduct the following two actions?

  1. Double check your Hive metastore configurations are correct. for example, database name, type, and username and password, etc.
  2. Check whether your metastore database is initialized successfully. For example, the following screenshot shows all the tables created by the init step in this guide. 


These tables are used by Hive to store metadata of your objects in Hive database. 

If you cannot start metastore service successfully, HiveServer2 thrift service will not be able to function properly.

reply Reply
3 years ago link more_vert
G Guy
articleArticles 0
imageDiagrams 0
codeCode 0
chat_bubble_outlineThreads 0
commentComments 15
loyaltyKontext Points 15
#400 Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

hello


thank you sir for answer.

i found some stuff ... while doing the step-by-step i went alone with derby DB .. but since it didn't work. i decided to try mssql DB.. following this link:

https://kontext.tech/column/hadoop/302/configure-a-sql-server-database-as-remote-hive-metastore


found out there are steps in this page that are not in the original page.

namely this command:

cp $HIVE_HOME/conf/hive-default.xml.template $HIVE_HOME/conf/hive-site.xml

there is though a code you said to add the hive-site.xml file .. but maybe there are lines missing there ? ( this is a huge file and millions of parameters in it )


could you recheck please if there is not omitted part in the original page :

https://kontext.tech/column/hadoop/309/apache-hive-311-installation-on-windows-10-using-windows-subsystem-for-linux


thank you again for help !


Guy

format_quote

person Raymond access_time 3 years ago
Re: Apache Hive 3.1.1 Installation on Windows 10 using Windows Subsystem for Linux

Hi Guy,

Most likely it is because that your Hive Metastore database schema is not correct, for example, init schema operation was not done properly.

Can you conduct the following two actions?

  1. Double check your Hive metastore configurations are correct. for example, database name, type, and username and password, etc.
  2. Check whether your metastore database is initialized successfully. For example, the following screenshot shows all the tables created by the init step in this guide. 


These tables are used by Hive to store metadata of your objects in Hive database. 

If you cannot start metastore service successfully, HiveServer2 thrift service will not be able to function properly.

reply Reply
1-10 of 20
recommendMore from Kontext