Streaming Analytics

Streaming analytics related tutorials and ideas.

rss_feed Subscribe RSS

local_offer kafka local_offer shell

visibility 210
thumb_up 0
access_time 3 months ago

This page summarizes commonly used Apache Kafka Windows commands.  Setup an environment variable named KAFKA_HOME that points to where Kafka is located.  Example: SET KAFKA_HOME=F:\big-data\kafka_2.13-2.6.0 warning  Remember to change the server address, port number and Kafka ...

Spark Structured Streaming - Read from and Write into Kafka Topics

local_offer spark local_offer kafka

visibility 895
thumb_up 0
access_time 3 months ago

Spark structured streaming provides rich APIs to read from and write to Kafka topics. When reading from Kafka, Kafka sources can be created for both streaming and batch queries. When writing into Kafka, Kafka sinks can be created as destination for both streaming and batch queries too.  ...

Kafka Topic Partitions Walkthrough via Python

local_offer kafka local_offer python

visibility 363
thumb_up 0
access_time 3 months ago

Partition is the parallelism unit in a Kafka cluster. Partitions are replicated in Kafka cluster (cluster of brokers) for fault tolerant and throughput. This articles show you how to work with Kafka partitions using Python as programming language. Package kafka-python will be used in the ...

local_offer kafka local_offer python

visibility 368
thumb_up 1
access_time 3 months ago

Apache Kafka is written with Scala. Thus, the most natural way is to use Scala (or Java) to call Kafka APIs, for example, Consumer APIs and Producer APIs. For Python developers, there are open source packages available that function similar as official Java clients.  This article shows you ...

local_offer kafka

visibility 413
thumb_up 0
access_time 3 months ago

Kafka is a distributed event streaming platform that can be used for high-performance streaming analytics, asynchronous event processing and reliable applications. This article provides step-by-step guidance about installing Kafka on Windows 10 for test and learn purposes.  Download Git Bash ...