Featured Post

Mastering flat_map in Python with List Comprehension

Image
Introduction In Python, when working with nested lists or iterables, one common challenge is flattening them into a single list while applying transformations. Many programming languages provide a built-in flatMap function, but Python does not have an explicit flat_map method. However, Python’s powerful list comprehensions offer an elegant way to achieve the same functionality. This article examines implementation behavior using Python’s list comprehensions and other methods. What is flat_map ? Functional programming  flatMap is a combination of map and flatten . It transforms the collection's element and flattens the resulting nested structure into a single sequence. For example, given a list of lists, flat_map applies a function to each sublist and returns a single flattened list. Example in a Functional Programming Language: List(List(1, 2), List(3, 4)).flatMap(x => x.map(_ * 2)) // Output: List(2, 4, 6, 8) Implementing flat_map in Python Using List Comprehension Python’...

Analytics on Fly - Read It

The basis for real-time analytics is to have all resources at disposal in the moment they are called for . So far, special materialized data structures, called cubes, have been created to efficiently serve analytical reports. Such cubes are based on a fixed number of dimensions along which analytical reports can define their result sets. Consequently, only a particular set of reports can be served by one cube. If other dimensions are needed, a new cube has to be created or existing ones have to be extended. In the worst case, a linear increase in the number of dimensions of a cube can result in an exponential growth of its storage requirements. Extending a cube can result in a deteriorating performance of those reports already using it. The decision to extend a cube or build a new one has to be considered carefully. 

In any case, a wide variety of cubes may be built during the lifetime of a system to serve reporting, thus increasing storage requirements and also maintenance efforts.

Instead of working with a predefined set of reports, business users should be able to formulate ad-hoc reports. Their playground should be the entire set of data the company owns, possibly including further data from external sources. Assuming a fast in-memory database, no more pre-computed materialized data structures are needed. As soon as changes to data are committed to the database, they will be visible for reporting. 

The preparation and conversion steps of data if still needed for reports are done during query execution and computations take place on the fly. Computation on the fly during reporting on the basis of cubes that do not store data, but only provide the interface for reporting, solves a problem that has existed up to now and allows for performance optimization of all analytical reports likewise

Comments

Popular posts from this blog

SQL Query: 3 Methods for Calculating Cumulative SUM

Big Data: Top Cloud Computing Interview Questions (1 of 4)

5 SQL Queries That Popularly Used in Data Analysis