Install and Run Kafka 3.2.0 On WSL

This article provides step-by-step guidance about installing Kafka 3.2.0 on Windows 10 or 11 via WSL for test and learn purposes. My WSL distro is Ubuntu 20.04 and this can work on other distros with no or small changes.  

Install Open JDK

Java Open JDK is required to run Kafka. If you have not installed Java Open JDK, please install it.

sduo apt update
sudo apt install default-jre
sudo apt install openjdk-11-jre-headless
sudo apt install openjdk-8-jre-headless

Verify Java version:

$ java -version
openjdk version "1.8.0_312"
OpenJDK Runtime Environment (build 1.8.0_312-8u312-b07-0ubuntu1~20.04-b07)
OpenJDK 64-Bit Server VM (build 25.312-b07, mixed mode)

Setup JAVA_HOME variable 

Setup environment variables by editing file ~/.bashrc.

 vi ~/.bashrc

Add the following environment variables:

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64

Save the file and source the changes.

source ~/.bashrc

Download Kafka binary package

1) Go to Kafka download portal and select a version. For this tutorial, version Scala 2.13  - kafka_2.13-3.2.0.tgz is downloaded.

wget https://dlcdn.apache.org/kafka/3.2.0/kafka_2.13-3.2.0.tgz

2) Unzip the binary package to a installation folder.

And then run the following command to unzip:

tar -xvzf  kafka_2.13-3.2.0.tgz

Most of the scripts that we need to run in the following steps are located in bin folder as the following screenshot shows:

3) Setup Kafka environment variable.

Let's add a environment variable KAFKA_HOME so that we can easily reference it in the following steps.

Setup environment variables by editing file ~/.bashrc.

 vi ~/.bashrc

Add the following environment variables:

export KAFKA_HOME=~/kafka_2.13-3.2.0/

Save the file and source the changes.

source ~/.bashrc

Remember to change variable value accordingly based on your environment setup.

Start Kafka environment

1) Start ZooKeeper services by running this command:

$KAFKA_HOME/bin/zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties

In this version, ZooKeeper is still required.

2) Start Kafka server 

Open another WSL terminal and run the following command:

$KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/server.properties

Once all the services are launched, you will have a Kafka environment ready to use.

You can verify by running jps command in a third WSL terminal (if you have Hadoop installed in your environment): 

$ jps -ml
979 kafka.Kafka /home/kontext/kafka_2.13-3.2.0//config/server.properties
1556 sun.tools.jps.Jps -ml
507 org.apache.zookeeper.server.quorum.QuorumPeerMain /home/kontext/kafka_2.13-3.2.0//config/zookeeper.properties

Let's run some tests about Kafka environment. 

Till now, there are three WSL terminals opened for the same distro as the following screenshot shows:

Create a Kafka topic

In the third terminal window, run the following command:

~$ $KAFKA_HOME/bin/kafka-topics.sh --create --topic kontext-events --bootstrap-server localhost:9092
Created topic kontext-events.

The command will create a topic named kontext-events as the above screenshot shows.

Describe Kafka topic

Run the following command to describe the created topic.

$KAFKA_HOME/bin/kafka-topics.sh --describe --topic kontext-events --bootstrap-server localhost:9092

The output looks like the following:

Topic: kontext-events   TopicId: yhyrvqWgRHWYIkoTiXZp9w PartitionCount: 1       ReplicationFactor: 1    Configs: segment.bytes=1073741824
        Topic: kontext-events   Partition: 0    Leader: 0       Replicas: 0     Isr: 0

Write some events into the topic

Let's start to write some events into the topic by running the following command:

$KAFKA_HOME/bin/kafka-console-producer.sh --topic kontext-events --bootstrap-server localhost:9092

Each line represents an event. Let's type into some events:

$ $KAFKA_HOME/bin/kafka-console-producer.sh --topic kontext-events --bootstrap-server localhost:9092
>This is the first event.
>This is the second event.
>This is the third event.

Press Ctrl + C to terminate this Console producer client. 

Read the events in the topic

Let's read the events by running the following command:

$KAFKA_HOME/bin/kafka-console-consumer.sh --topic kontext-events --from-beginning --bootstrap-server localhost:9092

The Console consumer client prints out the following events:

Press Ctrl + C to terminate the consumer client. 

Shutdown Kafka services

After finish practices, you can turn off the series by running the following commands:

$KAFKA_HOME/bin/kafka-server-stop.sh $KAFKA_HOME/config/server.properties
$KAFKA_HOME/bin/zookeeper-server-stop.sh $KAFKA_HOME/config/zookeeper.properties

References

• Apache Kafka Quick Start

As you can see, it is very easy to configure and run Kafka on WSL.