Posts

Showing posts with the label CSV

Featured Post

8 Ways to Optimize AWS Glue Jobs in a Nutshell

-
Image
  Improving the performance of AWS Glue jobs involves several strategies that target different aspects of the ETL (Extract, Transform, Load) process. Here are some key practices. 1. Optimize Job Scripts Partitioning : Ensure your data is properly partitioned. Partitioning divides your data into manageable chunks, allowing parallel processing and reducing the amount of data scanned. Filtering : Apply pushdown predicates to filter data early in the ETL process, reducing the amount of data processed downstream. Compression : Use compressed file formats (e.g., Parquet, ORC) for your data sources and sinks. These formats not only reduce storage costs but also improve I/O performance. Optimize Transformations : Minimize the number of transformations and actions in your script. Combine transformations where possible and use DataFrame APIs which are optimized for performance. 2. Use Appropriate Data Formats Parquet and ORC : These columnar formats are efficient for storage and querying, signif
Post a Comment
Read more

Python Program: JSON to CSV Conversion

-
Image
JavaScript object notion is also called JSON file, it's data you can write to a CSV file. Here's a sample python logic for your ready reference.  You can write a simple python program by importing the JSON, and CSV packages. This is your first step. It is helpful to use all the JSON methods in your python logic. That means the required package is JSON. So far, so good. In the next step, I'll show you how to write a Python program. You'll also find each term explained. What is JSON File JSON is key value pair file. The popular use of JSON file is to transmit data between heterogeneous applications. Python supports JSON file. What is CSV File The CSV is comma separated file. It is popularly used to send and receive data. How to Write JSON file data to a CSV file Here the JSON data that has written to CSV file. It's simple method and you can use for CSV file conversion use. import csv, json json_string = '[{"value1": 1, "value2": 2,"value3
Post a Comment
Read more