Streaming Analytics

Streaming analytics related tutorials and ideas.

rss_feed Subscribe RSS

local_offer kafka local_offer shell

visibility 10
thumb_up 0
access_time 17 days ago

This page summarizes commonly used Apache Kafka Windows commands.  Setup an environment variable named KAFKA_HOME that points to where Kafka is located.  Example: SET KAFKA_HOME=F:\big-data\kafka_2.13-2.6.0 warning  Remember to change the server address, port number and Kafka ...

Spark Structured Streaming - Read from and Write into Kafka Topics

local_offer spark local_offer kafka

visibility 95
thumb_up 0
access_time 18 days ago

Spark structured streaming provides rich APIs to read from and write to Kafka topics. When reading from Kafka, Kafka sources can be created for both streaming and batch queries. When writing into Kafka, Kafka sinks can be created as destination for both streaming and batch queries too.  ...

Kafka Topic Partitions Walkthrough via Python

local_offer kafka local_offer python

visibility 29
thumb_up 0
access_time 18 days ago

Partition is the parallelism unit in a Kafka cluster. Partitions are replicated in Kafka cluster (cluster of brokers) for fault tolerant and throughput. This articles show you how to work with Kafka partitions using Python as programming language. Package kafka-python will be used in the ...

local_offer kafka local_offer python

visibility 14
thumb_up 0
access_time 19 days ago

Apache Kafka is written with Scala. Thus, the most natural way is to use Scala (or Java) to call Kafka APIs, for example, Consumer APIs and Producer APIs. For Python developers, there are open source packages available that function similar as official Java clients.  This article shows you ...

local_offer kafka

visibility 58
thumb_up 0
access_time 25 days ago

Kafka is a distributed event streaming platform that can be used for high-performance streaming analytics, asynchronous event processing and reliable applications. This article provides step-by-step guidance about installing Kafka on Windows 10 for test and learn purposes.  Download Git Bash ...