Posts

Showing posts with the label native

Featured Post

8 Ways to Optimize AWS Glue Jobs in a Nutshell

-
Image
  Improving the performance of AWS Glue jobs involves several strategies that target different aspects of the ETL (Extract, Transform, Load) process. Here are some key practices. 1. Optimize Job Scripts Partitioning : Ensure your data is properly partitioned. Partitioning divides your data into manageable chunks, allowing parallel processing and reducing the amount of data scanned. Filtering : Apply pushdown predicates to filter data early in the ETL process, reducing the amount of data processed downstream. Compression : Use compressed file formats (e.g., Parquet, ORC) for your data sources and sinks. These formats not only reduce storage costs but also improve I/O performance. Optimize Transformations : Minimize the number of transformations and actions in your script. Combine transformations where possible and use DataFrame APIs which are optimized for performance. 2. Use Appropriate Data Formats Parquet and ORC : These columnar formats are efficient for storage and querying, signif
Post a Comment
Read more

Python Supports These 5 Native Data types

-
Image
Python supports five native data types. The Data types are such that a Programmer can use to write the logic and get the output. Many beginners may not aware of native data types. So I am adding a short note on that here. Native means as is Python supported data types. Python Five Key Native Data Types Python Native Data Types In Python, you can find five types of native data types. Here is a quick list for you. Those are Number, String, List, Tuple, Set, and Dictionary. 1. Number For all the numeric values,  you can use this data type. 2. String It handles all Characters, Special-symbols, and Alphanumeric values. 3. List It is something like sequential data. A program can do Sort, Merge, etc. on this data. 4. Tuple Data is a little different from the List. 5. Set This kind of Data-type helps you to do set operations. Those are like Intersection, Difference, etc. 6. Dictionary Here, the Dictionary something like a group of List kinds of data. But, each value has a key associated with i
Post a Comment
Read more