Featured Post

Python Regex: The 5 Exclusive Examples

Image
 Regular expressions (regex) are powerful tools for pattern matching and text manipulation in Python. Here are five Python regex examples with explanations: 01 Matching a Simple Pattern import re text = "Hello, World!" pattern = r"Hello" result = re.search(pattern, text) if result:     print("Pattern found:", result.group()) Output: Output: Pattern found: Hello This example searches for the pattern "Hello" in the text and prints it when found. 02 Matching Multiple Patterns import re text = "The quick brown fox jumps over the lazy dog." patterns = [r"fox", r"dog"] for pattern in patterns:     if re.search(pattern, text):         print(f"Pattern '{pattern}' found.") Output: Pattern 'fox' found. Pattern 'dog' found. It searches for both "fox" and "dog" patterns in the text and prints when they are found. 03 Matching Any Digit   import re text = "The price of the

Efficient Data Storage: Exploring 5 Patterns to Handle a Variety of Data

Data is now a variety of patterns. Data is now more than just plain text, it can exist in various persistence-storage mechanisms, with Hadoop distributed file system (HDFS) being one of them.

The way data is ingested or the sources from which data is ingested affects the way data is stored. On the other hand, how the data is pushed further into the downstream systems or accessed by the data access layer decides how the data is to be stored.

Role of RDBMS

The need to store huge volumes of data has forced databases to follow new rules of data relationships and integrity that are different from those of relational database management systems (RDBMS). RDBMS follow the ACID rules of atomicity, consistency, isolation, and durability.

These rules make the database reliable to any user of the database. However, searching huge volumes of big data and retrieving data from them would take large amounts of time if all the ACID rules were enforced.
A typical scenario is when we search for a certain topic using Google. The search returns innumerable pages of data; however, only one page is visible or basically available (BA). 
The rest of the data is in a soft state (S) and is still being assembled by Google, though the user is not aware of it. By the time the user looks at the data on the first page, the rest of the data becomes eventually consistent (E). This phenomenon—basically available soft state and eventually consistent—is the rule followed by the big data databases, which are generally NoSQL databases following BASE properties.

Database theory suggests that any distributed NoSQL big database can satisfy only two properties predominantly and will have to relax standards on the third. 

The three properties are consistency, availability, and partition tolerance (CAP). This is the CAP theorem. 

Polyglot pattern: Multiple types of storage mechanisms—like RDBMS, file storage, CMS, OODBMS, NoSQL, and HDFS—co-exist in an enterprise to solve the big data problem.

The aforementioned paradigms of ACID, BASE, and CAP give rise to new big data storage patterns like below:
  • Façade pattern: HDFS serves as the intermittent Façade for the traditional DW systems. 
  • Lean pattern: HBase is indexed using only one column-family and only one column and unique row-key. 
  • NoSQL pattern: Traditional RDBMS systems are replaced by NoSQL alternatives to facilitate faster access and querying of big data.

Comments

Popular posts from this blog

Explained Ideal Structure of Python Class

6 Python file Methods Real Usage

How to Decode TLV Quickly