Posts

Showing posts with the label Kafka Performance

Featured Post

SQL Interview Success: Unlocking the Top 5 Frequently Asked Queries

Image
 Here are the five top commonly asked SQL queries in the interviews. These you can expect in Data Analyst, or, Data Engineer interviews. Top SQL Queries for Interviews 01. Joins The commonly asked question pertains to providing two tables, determining the number of rows that will return on various join types, and the resultant. Table1 -------- id ---- 1 1 2 3 Table2 -------- id ---- 1 3 1 NULL Output ------- Inner join --------------- 5 rows will return The result will be: =============== 1  1 1   1 1   1 1    1 3    3 02. Substring and Concat Here, we need to write an SQL query to make the upper case of the first letter and the small case of the remaining letter. Table1 ------ ename ===== raJu venKat kRIshna Solution: ========== SELECT CONCAT(UPPER(SUBSTRING(name, 1, 1)), LOWER(SUBSTRING(name, 2))) AS capitalized_name FROM Table1; 03. Case statement SQL Query ========= SELECT Code1, Code2,      CASE         WHEN Code1 = 'A' AND Code2 = 'AA' THEN "A" | "A

How to Monitor Kafka-stream's Performance

Image
Kafka Streams API is a part of Kafka, it goes without saying that monitoring your application will require some monitoring of Kafka as well. Performance The consumer and producer performance is one of the fundamental performance concerns for a producer and consumer.   The Kafka data flow diagram What is lag For producers, we care mostly about how fast the producer is sending messages to the broker. Obviously, the higher the throughput, the better. For consumers, we’re also concerned with performance, or how fast we can read messages from a broker. we care about how much and how fast our producers can publish to a broker, and we simultaneously care about how quickly our consumers can read those messages from the broker. The difference between how fast the producers place records on the broker and when consumers read those messages is called consumer lag How to check consumer lag To check for consumer lag, Kafka provides a convenient command-line tool, kafka-consumer-groups.sh, found in