Raymond Raymond / Hadoop, Hive & HBase

Install HBase in WSL - Standalone Mode

event 2021-02-02 visibility 3,072 comment 12 insights toc
more_vert
insights Stats
toc Table of contents

HBase is the short name for Hadoop database. HBase is a distributed non-SQL database like Google Bigtable, which can utilizes distributed file system like HDFS. HBase can run in two modes - standalone and distributed. In this tutorial, I will show you how to install a standalone HBase on WSL (Windows Subsystem for Linux) without using HDFS. 

Prerequisites

WSL

Please ensure you have WSL enabled on your Windows 10 system.  Follow Install Windows Subsystem for Linux on a Non-System Drive to install WSL on a non-C drive.  This tutorial utilizes Debian distro. You could also replicate these steps on Debian Linux system directly.

JDK

Follow article Install Open JDK on WSL to install JDK if it is not available.

Ensure JAVA_HOME environment variable is setup properly.

Step by step guide

1) Download HBase from a mirror site: Apache Download Mirrors.

For example, I am using the following command to download the released binary into my WSL user home folder:

wget https://www.strategylions.com.au/mirror/hbase/2.4.1/hbase-2.4.1-bin.tar.gz

The version downloaded is 2.4.1.

2) Extract the downloaded file using the following command:

tar xzvf hbase-2.4.1-bin.tar.gz

3) Change directory to the extracted folder:

cd hbase-2.4.1

The folder includes these files/subfolders:

~/hbase-2.4.1$ ls
bin  CHANGES.md  conf  docs  hbase-webapps  LEGAL  lib  LICENSE.txt  NOTICE.txt  README.txt  RELEASENOTES.md

4) Start HBase daemon service:

~/hbase-2.4.1$ bin/start-hbase.sh

The service may not start successfully due to the following error:

2021-02-02 23:13:40,683 INFO  [main] server.NIOServerCnxnFactory: binding to port localhost/127.0.0.1:2181
2021-02-02 23:13:40,684 ERROR [main] master.HMasterCommandLine: Master exiting
java.net.SocketException: Permission denied
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:461)
        at sun.nio.ch.Net.bind(Net.java:453)
        at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:687)
        at org.apache.zookeeper.server.ServerCnxnFactory.configure(ServerCnxnFactory.java:76)
        at org.apache.hadoop.hbase.zookeeper.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:241)
        at org.apache.hadoop.hbase.zookeeper.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:189)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:227)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:149)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2863)

I resolved this issue via the following two steps:

  • Update conf/hbase-site.xml file to use a different client port:
    As shown in line 56 of the following screenshot, the client port is changed to 10231.
    20210202122906-image.png
  • And then rerun the command to start the daemons.  Make sure to allow java communications:20210202122651-image.png

5) Run jps command to ensure HMaster Java process exists.

~/hbase-2.4.1$ jps
1608 HMaster
2508 Jps

6) Check the Web UI: http://localhost:16010.

The UI looks like the following:

20210202123302-image.png

7) Connect to HBase using Shell:

~/hbase-2.4.1$ bin/hbase shell
2021-02-02 23:34:14,903 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.4.1, rb4d9639f66fccdb45fea0244202ffbd755341260, Fri Jan 15 10:58:57 PST 2021
Took 0.0017 seconds
hbase:001:0>

8) Practice the following commands in the HBase shell:

  • create 'test_table', 'cf'
  • list 'test_table'
  • describe 'test_table'
  • put 'test_table', 'row1', 'cf:a', 'value1'
  • put 'test_table', 'row2', 'cf:b', 'value B'
  • put 'test_table', 'row3', 'cf:c', 'value 3'
  • scan 'test_table'
  • get 'test_table', 'row1'
  • drop 'test_table'
  • disable 'test_table'
  • drop 'test_table'

The output looks like the following screenshot:

20210202124014-image.png

9) Quit the shell by running this command:

quit
# or
exit

10) Stop HBase daemon services if you don't want to use HBase in WSL:

~/hbase-2.4.1$ bin/stop-hbase.sh
stopping hbase............

More from Kontext
comment Comments
Raymond Raymond #1636 access_time 3 years ago more_vert

I am glad it works for you now too. 

format_quote

person Guy access_time 3 years ago

thank you sir !

working for me too now without "sudo"


guy@x360:~/code$ python3 hbase_read.py

b'value1'



hbase:006:0> scan "table1"
ROW                  COLUMN+CELL
row1                   column=cf:a, timestamp=2022-03-22T15:31:11.676, value=value1

1 row(s) Took 0.0298 seconds



G Guy A #1635 access_time 3 years ago more_vert

thank you sir !

working for me too now without "sudo"


guy@x360:~/code$ python3 hbase_read.py

b'value1'



hbase:006:0> scan "table1"
ROW                  COLUMN+CELL
row1                   column=cf:a, timestamp=2022-03-22T15:31:11.676, value=value1

1 row(s) Took 0.0298 seconds



format_quote

person Raymond access_time 3 years ago

I just tried to run it in my previous Debian WSL distro but it didn't work anymore and then I used sudo command and it works:

sudo bin/start-hbase.sh

I installed latest happybase package:

pip3 install happybase

Then I started the thirft server:

sudo bin/hbase-daemon.sh start thrift

And then I run the following python code:

import happybase
connection = happybase.Connection('127.0.0.1',9090)
table = connection.table('test_table')
row = table.row(b'row1')
print(row[b'cf:a'])

The output looks like the following:


So it does work for me. I've documented the details here: Connect to HBase in Python via HappyBase

Raymond Raymond #1634 access_time 3 years ago more_vert

I just tried to run it in my previous Debian WSL distro but it didn't work anymore and then I used sudo command and it works:

sudo bin/start-hbase.sh

I installed latest happybase package:

pip3 install happybase

Then I started the thirft server:

sudo bin/hbase-daemon.sh start thrift

And then I run the following python code:

import happybase
connection = happybase.Connection('127.0.0.1',9090)
table = connection.table('test_table')
row = table.row(b'row1')
print(row[b'cf:a'])

The output looks like the following:


So it does work for me. I've documented the details here: Connect to HBase in Python via HappyBase

format_quote

person Guy access_time 3 years ago

Hey 


so in second vm, only hbase was working fine, not even the errors you talked about in the tutorial, tried to use python with it but failed.

so i installed thrift, successfully .. but since this moment hbase died. 


so, i will let it go now.


thank you for the tutorial anyhow.

:)

Guy

G Guy A #1633 access_time 3 years ago more_vert

Hey 


so in second vm, only hbase was working fine, not even the errors you talked about in the tutorial, tried to use python with it but failed.

so i installed thrift, successfully .. but since this moment hbase died. 


so, i will let it go now.


thank you for the tutorial anyhow.

:)

Guy

G Guy A #1632 access_time 3 years ago more_vert

also, do you have a manual how to integrate thrift api into the standalone hbase ? 


thanks

format_quote

person Raymond access_time 3 years ago

This article I published is for a standalone mode installation.

Are you trying to configure HBase in distributed mode? 

G Guy A #1631 access_time 3 years ago more_vert

Hey


so i installed a new fresh copy of ubuntu inside the windows WSL.

only hbase and it is working perfectly.


thank you

Guy 

format_quote

person Raymond access_time 3 years ago

This article I published is for a standalone mode installation.

Are you trying to configure HBase in distributed mode? 

Raymond Raymond #1630 access_time 3 years ago more_vert

Good luck! I have not installed a distributed mode for HBase thus am not able to advise right now. I will publish a new article if I do that. 

format_quote

person Guy access_time 3 years ago

Well . I do have hive hdfs and spark from you other tutorials .. so since it didn't work i tried twicking and trying .. i use previously installed zk. And changed root.dir to hdfs ( i found i need to change something from false to true .. can't remember now what. I will check tomorrow.


I installed new copy of Ubuntu today into wsl and will try tomorrow only hbase on it. 

G Guy A #1629 access_time 3 years ago more_vert

Well . I do have hive hdfs and spark from you other tutorials .. so since it didn't work i tried twicking and trying .. i use previously installed zk. And changed root.dir to hdfs ( i found i need to change something from false to true .. can't remember now what. I will check tomorrow.


I installed new copy of Ubuntu today into wsl and will try tomorrow only hbase on it. 

format_quote

person Raymond access_time 3 years ago

This article I published is for a standalone mode installation.

Are you trying to configure HBase in distributed mode? 

Raymond Raymond #1628 access_time 3 years ago more_vert

This article I published is for a standalone mode installation.

Are you trying to configure HBase in distributed mode? 

format_quote

person Guy access_time 3 years ago

Hey

thank you for answering, yes i did get several error messages, i was googling deeply and had some advancement .. yet the shell is not connecting to the hbase server and i cannot create a table or get status.


[2022-03-17 16:10:56,313] WARN Empty contextPath (org.eclipse.jetty.server.handler.ContextHandler) [2022-03-17 16:10:56,328] INFO jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 (org.eclipse.jetty.server.Server) [2022-03-17 16:10:56,355] INFO DefaultSessionIdManager workerName=node0 (org.eclipse.jetty.server.session) [2022-03-17 16:10:56,355] INFO No SessionScavenger set, using defaults (org.eclipse.jetty.server.session) [2022-03-17 16:10:56,356] INFO node0 Scavenging every 600000ms (org.eclipse.jetty.server.session) [2022-03-17 16:10:56,360] WARN ServletContext@o.e.j.s.ServletContextHandler@6950e31{/,null,STARTING} has uncovered http methods for path: /* (org.eclipse.jetty.security.SecurityHandler) [2022-03-17 16:10:56,371] INFO Started o.e.j.s.ServletContextHandler@6950e31{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler) [2022-03-17 16:10:56,387] INFO Started ServerConnector@2d8f65a4{HTTP/1.1, (http/1.1)}{0.0.0.0:8081} (org.eclipse.jetty.server.AbstractConnector) [2022-03-17 16:10:56,388] INFO Started @682ms (org.eclipse.jetty.server.Server) [2022-03-17 16:10:56,388] INFO Started AdminServer on address 0.0.0.0, port 8081 and command URL /commands (org.apache.zookeeper.server.admin.JettyAdminServer) [2022-03-17 16:10:56,394] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory) [2022-03-17 16:10:56,395] WARN maxCnxns is not configured, using default value 0. (org.apache.zookeeper.server.ServerCnxnFactory) [2022-03-17 16:10:56,396] INFO Configuring NIO connection handler with 10s sessionless connection timeout, 2 selector thread(s), 16 worker threads, and 64 kB direct buffers. (org.apache.zookeeper.server.NIOServerCnxnFactory) [2022-03-17 16:10:56,397] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory) [2022-03-17 16:10:56,413] INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [2022-03-17 16:10:56,413] INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [2022-03-17 16:10:56,413] INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase) [2022-03-17 16:10:56,413] INFO zookeeper.commitLogCount=500 (org.apache.zookeeper.server.ZKDatabase) [2022-03-17 16:10:56,419] INFO zookeeper.snapshot.compression.method = CHECKED (org.apache.zookeeper.server.persistence.SnapStream) [2022-03-17 16:10:56,425] INFO Reading snapshot /tmp/zookeeper/version-2/snapshot.143 (org.apache.zookeeper.server.persistence.FileSnap)


hbase:010:0> create "tbl1", "fc1"
ERROR: KeeperErrorCode = NoNode
for /hbase/master For usage try 'help "create"'
Took 0.0058 seconds
hbase:011:0>




guy@x360:~$ hadoop/hbase-2.4.9/bin/start-hbase.sh
guy@x360:~$ jps
15539 HRegionServer
1285 NodeManager
918 SecondaryNameNode
15334 HMaster
520 NameNode
1144 ResourceManager
1656 QuorumPeerMain
666 DataNode
3819 JarBootstrapMain
15885 Jps
guy@x360:~$


G Guy A #1627 access_time 3 years ago more_vert

Hey

thank you for answering, yes i did get several error messages, i was googling deeply and had some advancement .. yet the shell is not connecting to the hbase server and i cannot create a table or get status.


[2022-03-17 16:10:56,313] WARN Empty contextPath (org.eclipse.jetty.server.handler.ContextHandler) [2022-03-17 16:10:56,328] INFO jetty-9.4.43.v20210629; built: 2021-06-30T11:07:22.254Z; git: 526006ecfa3af7f1a27ef3a288e2bef7ea9dd7e8; jvm 1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 (org.eclipse.jetty.server.Server) [2022-03-17 16:10:56,355] INFO DefaultSessionIdManager workerName=node0 (org.eclipse.jetty.server.session) [2022-03-17 16:10:56,355] INFO No SessionScavenger set, using defaults (org.eclipse.jetty.server.session) [2022-03-17 16:10:56,356] INFO node0 Scavenging every 600000ms (org.eclipse.jetty.server.session) [2022-03-17 16:10:56,360] WARN ServletContext@o.e.j.s.ServletContextHandler@6950e31{/,null,STARTING} has uncovered http methods for path: /* (org.eclipse.jetty.security.SecurityHandler) [2022-03-17 16:10:56,371] INFO Started o.e.j.s.ServletContextHandler@6950e31{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler) [2022-03-17 16:10:56,387] INFO Started ServerConnector@2d8f65a4{HTTP/1.1, (http/1.1)}{0.0.0.0:8081} (org.eclipse.jetty.server.AbstractConnector) [2022-03-17 16:10:56,388] INFO Started @682ms (org.eclipse.jetty.server.Server) [2022-03-17 16:10:56,388] INFO Started AdminServer on address 0.0.0.0, port 8081 and command URL /commands (org.apache.zookeeper.server.admin.JettyAdminServer) [2022-03-17 16:10:56,394] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory) [2022-03-17 16:10:56,395] WARN maxCnxns is not configured, using default value 0. (org.apache.zookeeper.server.ServerCnxnFactory) [2022-03-17 16:10:56,396] INFO Configuring NIO connection handler with 10s sessionless connection timeout, 2 selector thread(s), 16 worker threads, and 64 kB direct buffers. (org.apache.zookeeper.server.NIOServerCnxnFactory) [2022-03-17 16:10:56,397] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory) [2022-03-17 16:10:56,413] INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [2022-03-17 16:10:56,413] INFO Using org.apache.zookeeper.server.watch.WatchManager as watch manager (org.apache.zookeeper.server.watch.WatchManagerFactory) [2022-03-17 16:10:56,413] INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase) [2022-03-17 16:10:56,413] INFO zookeeper.commitLogCount=500 (org.apache.zookeeper.server.ZKDatabase) [2022-03-17 16:10:56,419] INFO zookeeper.snapshot.compression.method = CHECKED (org.apache.zookeeper.server.persistence.SnapStream) [2022-03-17 16:10:56,425] INFO Reading snapshot /tmp/zookeeper/version-2/snapshot.143 (org.apache.zookeeper.server.persistence.FileSnap)


hbase:010:0> create "tbl1", "fc1"
ERROR: KeeperErrorCode = NoNode
for /hbase/master For usage try 'help "create"'
Took 0.0058 seconds
hbase:011:0>




guy@x360:~$ hadoop/hbase-2.4.9/bin/start-hbase.sh
guy@x360:~$ jps
15539 HRegionServer
1285 NodeManager
918 SecondaryNameNode
15334 HMaster
520 NameNode
1144 ResourceManager
1656 QuorumPeerMain
666 DataNode
3819 JarBootstrapMain
15885 Jps
guy@x360:~$


format_quote

person Raymond access_time 3 years ago

Did you get any error message when starting the service?

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts