Featured Post

SQL Interview Success: Unlocking the Top 5 Frequently Asked Queries

Image
 Here are the five top commonly asked SQL queries in the interviews. These you can expect in Data Analyst, or, Data Engineer interviews. Top SQL Queries for Interviews 01. Joins The commonly asked question pertains to providing two tables, determining the number of rows that will return on various join types, and the resultant. Table1 -------- id ---- 1 1 2 3 Table2 -------- id ---- 1 3 1 NULL Output ------- Inner join --------------- 5 rows will return The result will be: =============== 1  1 1   1 1   1 1    1 3    3 02. Substring and Concat Here, we need to write an SQL query to make the upper case of the first letter and the small case of the remaining letter. Table1 ------ ename ===== raJu venKat kRIshna Solution: ========== SELECT CONCAT(UPPER(SUBSTRING(name, 1, 1)), LOWER(SUBSTRING(name, 2))) AS capitalized_name FROM Table1; 03. Case statement SQL Query ========= SELECT Code1, Code2,      CASE         WHEN Code1 = 'A' AND Code2 = 'AA' THEN "A" | "A

5 Python Pandas Tricky Examples for Data Analysis

Here are five tricky Python Pandas examples. These provide detailed insights to work with Pandas in Python,


Pandas examples

#1 Dealing with datetime data (parse_dates pandas example)


import pandas as pd

# Convert a column to datetime format

data['date_column'] = pd.to_datetime(data['date_column'])


# Extract components from datetime (e.g., year, month, day)

data['year'] = data['date_column'].dt.year

data['month'] = data['date_column'].dt.month


# Calculate the time difference between two datetime columns

data['time_diff'] = data['end_time'] - data['start_time']


#2 Working with text data

 

# Convert text to lowercase

data['text_column'] = data['text_column'].str.lower()


# Count the occurrences of specific words in a text column

data['word_count'] = data['text_column'].str.count('word')


# Extract information using regular expressions

data['extracted_info'] = data['text_column'].str.extract(r'(\d+)')


#3 Handling large datasets efficiently


# Read a large dataset in chunks

chunk_size = 100000

data_chunks = pd.read_csv('large_data.csv', chunksize=chunk_size)

# Process data in chunks

for chunk in data_chunks:

    # Perform calculations or manipulations on each chunk


# Append data from multiple files

file_list = ['file1.csv', 'file2.csv', 'file3.csv']

combined_data = pd.concat([pd.read_csv(file) for file in file_list])


#4 Pivot tables and reshaping data


# Create a pivot table

pivot_table = data.pivot_table(values='column2', index='column1', columns='column3', aggfunc='mean')


# Unstack a multi-index DataFrame

unstacked_data = pivot_table.unstack().reset_index()


# Melt a DataFrame from wide to long format

melted_data = pd.melt(data, id_vars=['id'], value_vars=['var1', 'var2'], var_name='variable', value_name='value')


#5 Efficient memory usage


# Optimize memory usage of DataFrame columns

data['numeric_column'] = pd.to_numeric(data['numeric_column'], downcast='integer')

data['category_column'] = data['category_column'].astype('category')


# Load a subset of columns from a large dataset

selected_columns = ['column1', 'column2', 'column3']

data_subset = pd.read_csv('large_data.csv', usecols=selected_columns)


These examples demonstrate more advanced techniques for handling datetime data, text data, large datasets, reshaping data, and optimizing memory usage. They highlight some of the powerful features that pandas provide for complex data analysis tasks.


Related

Comments

Popular posts from this blog

How to Fix datetime Import Error in Python Quickly

Explained Ideal Structure of Python Class

How to Check Kafka Available Brokers