Posts

Showing posts with the label Python

Featured Post

SQL Query: 3 Methods for Calculating Cumulative SUM

Image
SQL provides various constructs for calculating cumulative sums, offering flexibility and efficiency in data analysis. In this article, we explore three distinct SQL queries that facilitate the computation of cumulative sums. Each query leverages different SQL constructs to achieve the desired outcome, catering to diverse analytical needs and preferences. Using Window Functions (e.g., PostgreSQL, SQL Server, Oracle) SELECT id, value, SUM(value) OVER (ORDER BY id) AS cumulative_sum  FROM your_table; This query uses the SUM() window function with the OVER clause to calculate the cumulative sum of the value column ordered by the id column. Using Subqueries (e.g., MySQL, SQLite): SELECT t1.id, t1.value, SUM(t2.value) AS cumulative_sum FROM your_table t1 JOIN your_table t2 ON t1.id >= t2.id GROUP BY t1.id, t1.value ORDER BY t1.id; This query uses a self-join to calculate the cumulative sum. It joins the table with itself, matching rows where the id in the first table is greater than or

15 Top Data Analyst Interview Questions: Read Now

Image
We will explore the world of data analysis using Python, covering topics such as data manipulation, visualization, machine learning, and more. Whether you are a beginner or an experienced data professional, join us on this journey as we dive into the exciting realm of Python analytics and unlock the power of data-driven insights. Let's harness Python's versatility and explore the endless possibilities it offers for extracting valuable information from datasets. Get ready to level up your data analysis skills and stay tuned for informative and practical content! Python Data Analyst Interview Questions 01: How do you import the pandas library in Python?  A: To import the pandas library in Python, you can use the following statement: import pandas as pd. Q2: What is the difference between a Series and a DataFrame in pandas?  A: A Series in pandas is a one-dimensional labeled array, while a DataFrame is a two-dimensional labeled data structure with columns of potentially different

How to Effectively Parse and Read Different Files in Python

Image
Here is Python logic that shows Parse and Read Different Files in Python. The formats are XML, JSON, CSV, Excel, Text, PDF, Zip files, Images, SQLlite, and Yaml. Python Reading Files import pandas as pd import json import xml.etree.ElementTree as ET from PIL import Image import pytesseract import PyPDF2 from zipfile import ZipFile import sqlite3 import yaml Reading Text Files # Read text file (.txt) def read_text_file(file_path):     with open(file_path, 'r') as file:         text = file.read()     return text Reading CSV Files # Read CSV file (.csv) def read_csv_file(file_path):     df = pd.read_csv(file_path)     return df Reading JSON Files # Read JSON file (.json) def read_json_file(file_path):     with open(file_path, 'r') as file:         json_data = json.load(file)     return json_data Reading Excel Files # Read Excel file (.xlsx, .xls) def read_excel_file(file_path):     df = pd.read_excel(file_path)     return df Reading PDF files # Read PDF file (.pdf) def rea

How to Write Complex Python Script: Explained Each Step

Image
 Creating a complex Python script is challenging, but I can provide you with a simplified example of a script that simulates a basic bank account system. In a real-world application, this would be much more elaborate, but here's a concise version. Python Complex Script Here is an example of a Python script that explains each step: class BankAccount:     def __init__(self, account_holder, initial_balance=0):         self.account_holder = account_holder         self.balance = initial_balance     def deposit(self, amount):         if amount > 0:             self.balance += amount             print(f"Deposited ${amount}. New balance: ${self.balance}")         else:             print("Invalid deposit amount.")     def withdraw(self, amount):         if 0 < amount <= self.balance:             self.balance -= amount             print(f"Withdrew ${amount}. New balance: ${self.balance}")         else:             print("Invalid withdrawal amount o

Python Regex: The 5 Exclusive Examples

Image
 Regular expressions (regex) are powerful tools for pattern matching and text manipulation in Python. Here are five Python regex examples with explanations: 01 Matching a Simple Pattern import re text = "Hello, World!" pattern = r"Hello" result = re.search(pattern, text) if result:     print("Pattern found:", result.group()) Output: Output: Pattern found: Hello This example searches for the pattern "Hello" in the text and prints it when found. 02 Matching Multiple Patterns import re text = "The quick brown fox jumps over the lazy dog." patterns = [r"fox", r"dog"] for pattern in patterns:     if re.search(pattern, text):         print(f"Pattern '{pattern}' found.") Output: Pattern 'fox' found. Pattern 'dog' found. It searches for both "fox" and "dog" patterns in the text and prints when they are found. 03 Matching Any Digit   import re text = "The price of the

Best Practices for Handling Duplicate Elements in Python Lists

Image
Here are three awesome ways that you can use to remove duplicates in a list. These are helpful in resolving your data analytics solutions.  01. Using a Set Convert the list into a set , which automatically removes duplicates due to its unique element nature, and then convert the set back to a list. Solution: original_list = [2, 4, 6, 2, 8, 6, 10] unique_list = list(set(original_list)) 02. Using a Loop Iterate through the original list and append elements to a new list only if they haven't been added before. Solution: original_list = [2, 4, 6, 2, 8, 6, 10] unique_list = [] for item in original_list:     if item not in unique_list:         unique_list.append(item) 03. Using List Comprehension Create a new list using a list comprehension that includes only the elements not already present in the new list. Solution: original_list = [2, 4, 6, 2, 8, 6, 10] unique_list = [] [unique_list.append(item) for item in original_list if item not in unique_list] All three methods will result in uni

How to Handle Spaces in PySpark Dataframe Column

Image
In PySpark, you can employ SQL queries by importing your CSV file data to a DataFrame. However, you might face problems when dealing with spaces in column names of the DataFrame. Fortunately, there is a solution available to resolve this issue. Reading CSV file to Dataframe Here is the PySpark code for reading CSV files and writing to a DataFrame. #initiate session spark = SparkSession.builder \ .appName("PySpark Tutorial") \ .getOrCreate() #Read CSV file to df dataframe data_path = '/content/Test1.csv' df = spark.read.csv(data_path, header=True, inferSchema=True) #Create a Temporary view for the DataFrame df2.createOrReplaceTempView("temp_table") #Read data from the temporary view spark.sql("select * from temp_table").show() Output --------+-----+---------------+---+ |Student| Year|Semester1|Semester2| | ID | | Marks | Marks | +----------+-----+---------------+ | si1 |year1|62.08| 62.4| | si1 |year2|75.94| 76.75| | si

How to Convert Dictionary to Dataframe: Pandas from_dict

Image
 Pandas is a data analysis Python library.  The example shows you to convert a dictionary to a data frame. The point to note here is DataFrame will take only 2D data. So you need to supply 2D data.  Pandas Dictionary to Dataframe import pandas as pd import numpy as np data_dict = {'item1' : np.random.randn(4), 'item2' : np.random.randn(4)} df3=pd.DataFrame. from_dict (data_dict, orient='index') print(df3) Output 0 1 2 3 item1 -0.109300 -0.483624 0.375838 1.248651 item2 -0.274944 -0.857318 -1.203718 -0.061941 Explanation Using the NumPy package, created a dictionary with random values. There are two items - item 1 and item 2. The data_dict is input to the data frame. The from_dict method needs two parameters. These are data_dict and index. Here's the syntax you can refer to quickly. Related Hands-on Data Analysis Using Pandas How to create 3D data frame in Pandas

The Easy Way to Split String Python Partition Method

Image
Here's a way without the Split function you can split (or extract) a substring. In Python the method is Partition. You'll find here how to use this method with an example.  How to Split the string using Partition method   Returns Left side part Example-1 my_string='ABCDEFGH||10||123456.25|' my_partition=my_string.partition('|')[0] print(my_partition) Output |10||123456.25| ** Process exited - Return Code: 0 ** Press Enter to exit terminal Example-2 Returns from the separator to last of the string. my_string='ABCDEFGH||10||123456.25|' my_partition=my_string.partition('|')[-1] print(my_partition) Output |10||123456.25| ** Process exited - Return Code: 0 ** Press Enter to exit terminal The use of Rpartition to split a string in Python Example-1 Returns except right side last separator. my_string='ABCDEFGH||10||123456.25|' my_partition=my_string.rpartition('|')[0] print(my_partition) Output ABCDEFGH||10||123456.25 ** Process exited -

5 Python Pandas Tricky Examples for Data Analysis

Image
Here are five tricky Python Pandas examples. These provide detailed insights to work with Pandas in Python, #1 Dealing with datetime data ( parse_dates pandas example) import pandas as pd # Convert a column to datetime format data['date_column'] = pd.to_datetime(data['date_column']) # Extract components from datetime (e.g., year, month, day) data['year'] = data['date_column'].dt.year data['month'] = data['date_column'].dt.month # Calculate the time difference between two datetime columns data['time_diff'] = data['end_time'] - data['start_time'] #2 Working with text data   # Convert text to lowercase data['text_column'] = data['text_column'].str.lower() # Count the occurrences of specific words in a text column data['word_count'] = data['text_column'].str.count('word') # Extract information using regular expressions data['extracted_info'] = data['text_column'].

2 User Input Python Sample Programs

Image
Here are the Python programs that work on taking user input and giving responses to the user. These are also called interactive programs.  Python enables you to read user input from the command line via the input() function or the raw_input() function. Typically, you assign user input to a variable containing all characters that users enter from the keyboard. User input terminates when users press the <return> key (included with the input characters). #1 User input sample program The following program takes input and replies if the given input value is a string or number. my_input = input("Enter something: ")  try:       x = 0 + eval(my_input)       print('You entered the number:', my_input)  except:       print(userInput,'is a string') Output Enter something:  100 You entered the number: 100 ** Process exited - Return Code: 0 ** Press Enter to exit terminal.  #2 User input sample program The following program takes two inputs from the user and calcula

The Quick and Easy Way to Analyze Numpy Arrays

Image
The quickest and easiest way to analyze NumPy arrays is by using the numpy.array() method. This method allows you to quickly and easily analyze the values contained in a numpy array. This method can also be used to find the sum, mean, standard deviation, max, min, and other useful analysis of the value contained within a numpy array. Sum You can find the sum of Numpy arrays using the np.sum() function.  For example:  import numpy as np  a = np.array([1,2,3,4,5])  b = np.array([6,7,8,9,10])  result = np.sum([a,b])  print(result)  # Output will be 55 Mean You can find the mean of a Numpy array using the np.mean() function. This function takes in an array as an argument and returns the mean of all the values in the array.  For example, the mean of a Numpy array of [1,2,3,4,5] would be  result = np.mean([1,2,3,4,5])  print(result)  #Output: 3.0 Standard Deviation To find the standard deviation of a Numpy array, you can use the NumPy std() function. This function takes in an array as a par

These 10 Skills You Need to Become Data Analyst

Image
To become a data analyst with Python, there are several technical skills you need to learn. Here are the key ones: #1 Python Programming Python is widely used in data analysis due to its simplicity, versatility, and the availability of powerful libraries. You should have a strong understanding of Python fundamentals, including data types, variables, loops, conditional statements, functions, and file handling. #2 Data Manipulation Libraries Familiarize yourself with libraries like NumPy and Pandas, which are essential for data manipulation and analysis. NumPy provides support for efficient numerical operations, while Pandas offers data structures (e.g., DataFrames) for easy data manipulation, cleaning, and transformation. #3 Data Visualization Gain proficiency in data visualization libraries like Matplotlib and Seaborn. These libraries enable you to create insightful visual representations of data, such as line plots, scatter plots, bar charts, histograms, and heatmaps. #4 SQL (Structu

How to Find Non-word Character: Python Regex Example

Image
In Python, the regular expression pattern \W matches any non-word character. Here's an example of usage. The valid word characters are [a-zA-Z0-9_]. \W (upper case W) matches any non-word character. Regex examples to find non-word char #1 Example import re text = "Hello, world! How are you today?" non_words = re.findall(r'\W', text) print(non_words) In the above example, the re.findall() function is used to find all non-word characters in the text string using the regular expression pattern \W. The output will be a list of non-word characters found in the string: Output [',', '!', ' ', ' ', '?'] This includes punctuation marks and spaces but excludes letters, digits, and underscores, which are considered word characters in regular expressions. #2 Example import re text = "Hello, world! How are non-word-char:! you today?" non_words = re.findall(r'non-word-char:\W', text) print(non_words) Output ['non-wo

How to Write ETL Logic in Python: Sample Code to Practice

Image
Here's an example Python code that uses the mysql-connector library to connect to a MySQL database, extract data from a table, transform it, and load it as a JSON file. Here's an example: Python ETL Sample Code import mysql.connector import json # Connect to the MySQL database cnx = mysql.connector.connect(user='username', password='password',                               host='localhost',                               database='database_name') # Define a cursor to execute SQL queries cursor = cnx.cursor() # Define the SQL query to extract data query = ("SELECT column1, column2, column3 FROM table_name") # Execute the SQL query cursor.execute(query) # Fetch all rows from the result set rows = cursor.fetchall() # Transform the rows into a list of dictionaries result = [] for row in rows:     result.append({'column1': row[0], 'column2': row[1], 'column3': row[2]}) # Save the result as a JSON file with open('ou

How to Write Lambda Function Quickly in Python: 5 Examples

Image
Here are the top python lambda function examples for your project and interviews. "Python's lambda functions are a powerful way to create small, anonymous functions on the fly. In this post, we'll explore some examples of how to use lambda functions in Python. 5 Best Python Lambda Function Examples #1 Sorting a List of Tuples by the Second Element This lambda function sorts a list of tuples based on the second element of each tuple. python code my_list = [(1, 2), (4, 1), (9, 10), (13, 6), (5, 7)] sorted_list = sorted(my_list, key=lambda x: x[1]) print(sorted_list) Output: [(4, 1), (1, 2), (13, 6), (5, 7), (9, 10)] ** Process exited - Return Code: 0 ** Press Enter to exit terminal #2 Finding the Maximum Value in a List of Dictionaries This lambda function finds the maximum value in a list of dictionaries based on a specific key. python code my_list = [{'name': 'Alice', 'age': 25}, {'name': 'Bob', 'age': 30}, {'name': &

The Quick and Easy Way to Fix Python UnboundLocalError

Image
Here is the easy way to fix the issue of the Python UnboundLocalError, allowing users to resolve any problems quickly. Python UnboundLocalError While the variable in the function has already been defined, during execution, the result prints with an error of UnboundLocalError. Below, you will find an example that explains the issue and resolution. Error: file 'example.txt' not found Traceback (most recent call last): File "main.py", line 16, in <module> open_file("example.txt") File "main.py", line 11, in open_file if f: UnboundLocalError: local variable 'f' referenced before assignment ** Process exited - Return Code: 1 ** Press Enter to exit terminal Python program using try, except and finally Below program explains how to use try, except and finally blocks in python. But during the execution, it prints UnboundLocalError. Program using try, except and finally # Define a function that may raise an exception def open_file(filename):  

Scraping Website: How to Write a Script in Python

Image
Here's a python model script to scrape a website using the BeautifulSoup. Python script The logic below uses BeautifulSoup Package for web scraping. import requests from bs4 import BeautifulSoup url = "https://www.example.com" response = requests.get(url) soup = BeautifulSoup(response.text, "html.parser") # Print the title of the webpage print(soup.title.text) # Print all the links in the webpage for link in soup.find_all("a"):     print(link.get("href")) In this script, we first import the Requests and Beautiful Soup libraries. We then define the URL we want to scrape and use the Requests library to send a GET request to that URL. We then pass the response text to Beautiful Soup to parse the HTML contents of the webpage. We then use Beautiful Soup to extract the title of the webpage and print it to the console. We also use a for loop to find all the links in the webpage and print their href attributes to the console. This is just a basic exa

How to Delete an Item from a Set in Python: Best Example

Image
Set is a built-in data type in Python. Furthermore, it is an unordered collection without duplicate items. Here are the two methods that explain to delete an item from a Set. Methods to delete an item from a Set discard remove Discrd Vs. Remove discard() will not raise an error if the item to remove does not exist. The remove() will raise an error if the item does not exist. Explanation to discard and remove methods Python program: #Prints all the Set items food = {"pasta", "burger", "hot dog", "pizza"} print(food) # Prints the Set items without pasta food.discard("pasta") print(food) # Prints the Set items without burger and pasta food.remove("burger") print(food) # The next two lines try to remove an item that isn't in the set! food.discard("pasta")  # this will not report an error food.remove("pasta")   # this will report an error The output: {'pasta', 'burger', 'pizza', '

How to Access Dictionary Key-Value Data in Python

Image
Use for-loop to read dictionary data in python. Here's an example of reading dictionary data. It's helpful to use in real projects. Python program to read dictionary data yearly_revenue = {    2017 : 1000000,    2018 : 1200000,    2019 : 1250000,    2020 : 1100000,    2021 : 1300000,  } total_income = 0 for year_id in yearly_revenue.keys() :   total_income+=yearly_revenue[year_id]   print(year_id, yearly_revenue[year_id]) print(total_income) print(total_income/len(yearly_revenue)) Output 2017 1000000 2018 1200000 2019 1250000 2020 1100000 2021 1300000 5850000 1170000.0 ** Process exited - Return Code: 0 ** Press Enter to exit the terminal Explanation The input is dictionary data. The total revenue sums up for each year. Notably, the critical point is using the dictionary keys method. References Python in-depth and sample programs

How to Decode Python Exception Messages Like a Pro

Image
While developing python programs, you might see exception messages from python. Here's an explanation to understand each part of the message. Here're tips on how to understand python exceptions. You can find two kinds of exceptions. These are StandardError and StopIteration errors. Here is a chart that shows the types of python errors. Python exceptions class Python exceptions are basically three parts. Reading an error message produced by  Python is not very difficult . The error type, the error description, and the traceback. Understand the python exception message The Error Type There are so many in-built exception types in python. Here is the command to get all the exception types: [x for x in dir(__builtins__) if 'Error' in x] The Error description The text message right after the error type gives us a description of what exactly the problem was. These descriptions are sometimes very accurate, sometimes not. Sample error Traceback (most recent call last):       Fil