Posts

Showing posts with the label Scraping Website

Featured Post

Mastering flat_map in Python with List Comprehension

Image
Introduction In Python, when working with nested lists or iterables, one common challenge is flattening them into a single list while applying transformations. Many programming languages provide a built-in flatMap function, but Python does not have an explicit flat_map method. However, Python’s powerful list comprehensions offer an elegant way to achieve the same functionality. This article examines implementation behavior using Python’s list comprehensions and other methods. What is flat_map ? Functional programming  flatMap is a combination of map and flatten . It transforms the collection's element and flattens the resulting nested structure into a single sequence. For example, given a list of lists, flat_map applies a function to each sublist and returns a single flattened list. Example in a Functional Programming Language: List(List(1, 2), List(3, 4)).flatMap(x => x.map(_ * 2)) // Output: List(2, 4, 6, 8) Implementing flat_map in Python Using List Comprehension Python’...

Scraping Website: How to Write a Script in Python

Image
Here's a python model script to scrape a website using the BeautifulSoup. Python script The logic below uses BeautifulSoup Package for web scraping. import requests from bs4 import BeautifulSoup url = "https://www.example.com" response = requests.get(url) soup = BeautifulSoup(response.text, "html.parser") # Print the title of the webpage print(soup.title.text) # Print all the links in the webpage for link in soup.find_all("a"):     print(link.get("href")) In this script, we first import the Requests and Beautiful Soup libraries. We then define the URL we want to scrape and use the Requests library to send a GET request to that URL. We then pass the response text to Beautiful Soup to parse the HTML contents of the webpage. We then use Beautiful Soup to extract the title of the webpage and print it to the console. We also use a for loop to find all the links in the webpage and print their href attributes to the console. This is just a basic exa...