Posts

Showing posts with the label Batch data

Kafka Flowchart Useful for Dummies

Image
Here're the prime points on Kafka stream-processing. In Mainframe, the data you receive/process in two methods (Batch and online). In Kafka, it receives data and sends it to consumers. Here're the details with Architecture, Logs, and applications that use Kafka. The streaming data is different (YouTube Live). When the data comes into Queue the data will then be processed. In the batch process, you need to wait till you get the Batch completes. In the case of stream processing, it is on the fly. How Kafka Works 1. Architecture 2. Process Kafka is a publish/subscribe system , but it would be more precise to say that Kafka acts as a message broker. A broker is an intermediary that brings together two parties that don’t necessarily know each other for a mutually beneficial exchange or deal. Kafka stores messages in topics and retrieves messages from topics. There’s no direct connection between the producers and the consumers of the messages. Additionally, Kafka doesn’t keep any st