Featured Post

8 Ways to Optimize AWS Glue Jobs in a Nutshell

  Improving the performance of AWS Glue jobs involves several strategies that target different aspects of the ETL (Extract, Transform, Load) process. Here are some key practices. 1. Optimize Job Scripts Partitioning : Ensure your data is properly partitioned. Partitioning divides your data into manageable chunks, allowing parallel processing and reducing the amount of data scanned. Filtering : Apply pushdown predicates to filter data early in the ETL process, reducing the amount of data processed downstream. Compression : Use compressed file formats (e.g., Parquet, ORC) for your data sources and sinks. These formats not only reduce storage costs but also improve I/O performance. Optimize Transformations : Minimize the number of transformations and actions in your script. Combine transformations where possible and use DataFrame APIs which are optimized for performance. 2. Use Appropriate Data Formats Parquet and ORC : These columnar formats are efficient for storage and querying, signif

Explained Ideal Structure of Python Class

When you are designing a class, you need to ensure that the classification of its critical parts is outlined at the beginning. The clearer the initial design, the more performant and scalable the class is. Some of the components in the order in which they should be defined in the class are mentioned as follows.

Python class structure

Ideal structure of a class

Class variables

Constants or default variables are usually defined at the top of the class. For someone who is reading the code, it comes as an easy-to-view consolidated list, and for the interpreter it ensures that all such variables are processed before diving into the main logic of the class, including any other Instance method or constructor.

The __init__ method

The __init__ method provides information about inputs needed and how to instantiate the class. It is also the constructor of the class, which the very first method called while initializing the class.

Special Python methods

These methods change the functionality of the class or provide additional capabilities to the class. For example, descriptors and other meta classes like __call__ are special methods. 

Keeping these at the top of the class before defining the core functionality helps to communicate to the readers and users what the custom behaviors are, that have been added to the class over and above the normal class behavior in Python.

Class method

Class methods are callable even without instantiating the class. Therefore, they can be treated on an equivalent level as constructors. 

Class methods
Image Courtesy: pynative.com

Static method

Static methods are same as the class methods in terms of functionality. 

Keeping them next in line is the natural and more useful convention here.

Private method

Python does not have a concept of privacy when it comes to methods. There are no access-specifiers like Java or C++. 

However, as we've read earlier, conventionally, the methods beginning with an underscore (_) are treated to be for internal use only, and not exposed via bulk import. 

The best option is to keep the private method section closer and just before the Instance methods, so that your code is more readable with the required elements being present close to each other. 

Note that the order of the Private methods relative to the Instance methods is not fixed.

Instance method 

The Instance methods are what you use to add functionality or the business logic to the class and can access the data of the instances of the class. 

In succession to the preceding methods, these define the core functionality of the class, rather than the meta-information, and hence should be placed after the preceding methods.



Popular posts from this blog

How to Fix datetime Import Error in Python Quickly

How to Check Kafka Available Brokers

SQL Query: 3 Methods for Calculating Cumulative SUM