Posts

Showing posts with the label python dictionary

Featured Post

8 Ways to Optimize AWS Glue Jobs in a Nutshell

-
Image
  Improving the performance of AWS Glue jobs involves several strategies that target different aspects of the ETL (Extract, Transform, Load) process. Here are some key practices. 1. Optimize Job Scripts Partitioning : Ensure your data is properly partitioned. Partitioning divides your data into manageable chunks, allowing parallel processing and reducing the amount of data scanned. Filtering : Apply pushdown predicates to filter data early in the ETL process, reducing the amount of data processed downstream. Compression : Use compressed file formats (e.g., Parquet, ORC) for your data sources and sinks. These formats not only reduce storage costs but also improve I/O performance. Optimize Transformations : Minimize the number of transformations and actions in your script. Combine transformations where possible and use DataFrame APIs which are optimized for performance. 2. Use Appropriate Data Formats Parquet and ORC : These columnar formats are efficient for storage and querying, signif
Post a Comment
Read more

Python Dictionary Vs List With Examples

-
Image
Dictionary and List we use interchangeably in Python to store values. For beginners, both look the same. In reality, they both differ. Here are the differences. Dictionary Vs Lists Values in lists are accessed by means of integers called indices, which indicate where in the list a given value is found. Dictionaries access values by means of integers, strings, or other Python objects called keys , which indicate where in the dictionary a given value is found.  In other words, both lists and dictionaries provide indexed access to arbitrary values, but the set of items that can be used as dictionary indices is much larger than, and contains, the set of items that can be used as list indices.   Also, the mechanism that dictionaries use to provide indexed access is quite different from that used by lists. Both lists and dictionaries can store objects of any type. Values stored in a list are implicitly ordered by their positions in the list because the indices that access these values are c
Post a Comment
Read more