Posts

Featured Post

SQL Interview Success: Unlocking the Top 5 Frequently Asked Queries

Image
 Here are the five top commonly asked SQL queries in the interviews. These you can expect in Data Analyst, or, Data Engineer interviews. Top SQL Queries for Interviews 01. Joins The commonly asked question pertains to providing two tables, determining the number of rows that will return on various join types, and the resultant. Table1 -------- id ---- 1 1 2 3 Table2 -------- id ---- 1 3 1 NULL Output ------- Inner join --------------- 5 rows will return The result will be: =============== 1  1 1   1 1   1 1    1 3    3 02. Substring and Concat Here, we need to write an SQL query to make the upper case of the first letter and the small case of the remaining letter. Table1 ------ ename ===== raJu venKat kRIshna Solution: ========== SELECT CONCAT(UPPER(SUBSTRING(name, 1, 1)), LOWER(SUBSTRING(name, 2))) AS capitalized_name FROM Table1; 03. Case statement SQL Query ========= SELECT Code1, Code2,      CASE         WHEN Code1 = 'A' AND Code2 = 'AA' THEN "A" | "A

How to Monitor Kafka-stream's Performance

Image
Kafka Streams API is a part of Kafka, it goes without saying that monitoring your application will require some monitoring of Kafka as well. Performance The consumer and producer performance is one of the fundamental performance concerns for a producer and consumer.   The Kafka data flow diagram What is lag For producers, we care mostly about how fast the producer is sending messages to the broker. Obviously, the higher the throughput, the better. For consumers, we’re also concerned with performance, or how fast we can read messages from a broker. we care about how much and how fast our producers can publish to a broker, and we simultaneously care about how quickly our consumers can read those messages from the broker. The difference between how fast the producers place records on the broker and when consumers read those messages is called consumer lag How to check consumer lag To check for consumer lag, Kafka provides a convenient command-line tool, kafka-consumer-groups.sh, found in

How to Read CSV file Data in Python

Image
Here is a way to read  CSV files  in Python pandas. The packages you need to import are numpy and pandas. On the flip side, f or Text files, you don't need to import these special libraries since python by default support it. Python pandas read_csv >>> import numpy as np >>> import pandas as pd To see how pandas handle this kind of data, we'll create a small CSV file in the working directory as ch05_01.csv. white, red, blue, green, animal 1,5,2,3,cat  2,7,8,5,dog  3,3,6,7,horse  2,2,8,3,duck  4,4,2,1,mouse Since this file is comma-delimited , you can use the read_csv() function to read its content and convert it to a dataframe object. >>> csvframe = pd.read_csv('ch05_01.csv') >>> csvframe white red blue green animal 0 1 5 2 3 cat 1 2 7 8 5 dog 2 3 3 6 7 horse 3 2 2 8 3 duck 4 4 4 2 1 mouse Python reading text files Since python supp

How to Fix a Byte-like Object is Required in Python

Image
Here is a way to fix TypeError: a bytes-like object is required, not 'str.' The error occurs while writing a file due to incorrect usage of the file mode. Below, you'll see how to solve this error. Python Open File in Write Mode Here's a way to open files in write mode. The "wb" mode's purpose is to write input to a file. If the file does not exist, it creates one.   After executing it, it throws an error highlighted in red. The file mode causes to error. Below, you'll see how to fix it and the details. a=open("file1.txt", 'wb') b=a.write('I am writing data to a file!') print(b) Traceback (most recent call last):  File "main.py", line 2, in <module>   b=a.write('I am writing data to a file!') TypeError:  a bytes-like object is required, not 'str' ** Process exited - Return Code: 1 ** Press Enter to exit terminal How to Fix the Error You can write data to a file in two ways a string or bytes. Th

Python Split String: Techniques and Best Practices

Image
Here is an example for  splitting a string in Python . The functions you need are strip and split to split string, or text.  Here's the string taken for split My task is I want to split address string. Historically, the address is lengthy. It contains the door number, floor number, street number, and street name. Rama Krishna 20/3 ABC street EFG Nagar US 234567 How to Split a string The python split string works in two steps. The first step is to use  strip function , which removes both leading and trailing spaces. Next, use the the split function, which splits the input string, or text into a list. Python program user_address = input("Enter an address") if user_address. strip() : #check that string is not empty (after removing leading #and trailing spaces)     print("Your address is " + user_address) split_address = user_address. split() print(split_address) Output Interactively, it asks the user to enter the address. After the code runs, the address splits i

How to Understand Pickling and Unpickling in Python

Image
Here are the Python pickling and unpickling best examples and the differences between these two. These you can use to serialize and deserialize the python data structures. The concept of writing the total state of an object to the file is called  pickling,  and to read a Total Object from the file is called  unpickling. Pickle and Unpickle The process of writing the state of an object to the file (converting a class object into a byte stream) and storing it in the file is called pickling. It is also called object serialization . The process of reading the state of an object from the file ( converting a byte stream back into a class object) is called unpickling.  It is an inverse operation of pickling. It is also called object deserialization .  The pickling and unpickling can implement by using a pickling module since binary files support byte streams. Pickling and unpickling should be possible using binary files. Data types you can pickle Integers Booleans Complex numbers Floats Nor

How to Configure Firewall For an Application

Image
A firewall is a set of rules. When a data packet moves into or out of protected network space, its contents (in particular, information about its origin, its target, and the protocol it plans to use) are tested against the firewall rules to see if it should be allowed through. How a Firewall concept works Let's say that the web server has to be open to incoming web traffic from anywhere on earth using either the insecure HTTP or secure HTTPS protocol. Because your developers and admins will need to get into the backend from time to time to do their work, you’ll also want to allow SSH traffic, but only for those people who’ll need it. Requests for any other services should be automatically refused. A Linux machine can be configured to apply firewall rules at the kernel level through a program called iptables. Creating table rules isn't all that difficult; the syntax can be learned without too much fuss. But, in the interest of simplifying your life, many Linux distributions ha

Python DateTime Objects: Manipulating and Calculating Dates

Image
Date and Time how to get and how to use is well needed in a software project. So in Python, you can find these features. Date and Time Features in Python Date feature you need it to get Date in the coding. Here the point is the Date and how to get it, and the various formats of it. Generally, you will find three different Date and Time formats. Before going into detail, here are those formats. Date and Time formats Epoch: It is a point where the time starts and is dependent on the platform. It is taken as January 1st of the current year, 00:00:00. But for UNIX systems, it is 1st January, 1970, 00:00:00 (UTC). UTC: UTC means Coordinated Universal Time (also known as Greenwich Mean Time) and is a compromise between English and French. DST: It is Daylight Saving Time which is an adjustment of the time zone by one hour during part of the year. In other words, the clock is forward by one hour in the spring and backward by one hour in the autumn to return to standard time. 1. How to get the