Featured Post

Scraping Website: How to Write a Script in Python

Image
Here's a python script that you can use as a model to scrape a website. Python script The below logic uses BeautifulSoup Package for web scraping. import requests from bs4 import BeautifulSoup url = 'https://www.example.com' response = requests.get(url) soup = BeautifulSoup(response.text, 'html.parser') # Print the title of the webpage print(soup.title.text) # Print all the links in the webpage for link in soup.find_all('a'):     print(link.get('href')) In this script, we first import the Requests and Beautiful Soup libraries. We then define the URL we want to scrape and use the Requests library to send a GET request to that URL. We then pass the response text to Beautiful Soup to parse the HTML contents of the webpage. We then use Beautiful Soup to extract the title of the webpage and print it to the console. We also use a for loop to find all the links in the webpage and print their href attributes to the console. This is just a basic example, but

Messages in Kafka the Types and Details

A message, also called a record, is the basic piece of data flowing through Kafka. Messages are how Kafka represents your data.


types of messages

Kafka producer Vs. consumer messages

Kafka is an intermediate server that receives a message from a producer and sends them to the consumer. Here is a set of 10 Kafka Interview Questions.

Kafka message format

Each message has a timestamp, a value, and an optional key. Custom headers can be used if desired as well. 

A simple example of a message could be something like the following: the machine with host ID “1234567” (a message key) failed with the message “Alert: Machine Failed” (a message value) at “2020-10-02T10:34:11.654Z” (a message timestamp). Here is Kafka's flowchart for dummies.


Kafka message format
Kafka record

The above image shows probably the most important and common parts of a message that users deal with directly.

Each key and value can interact in its own specific ways to serialize or deserialize its data.

Now that we have a record, how do we let Kafka know about it? You will deliver this message to Kafka by sending it to what is known as brokers.

References

Comments

Popular posts from this blog

7 AWS Interview Questions asked in Infosys, TCS

How to Decode TLV Quickly