Showing posts with the label Bash

Featured Post

8 Ways to Optimize AWS Glue Jobs in a Nutshell

  Improving the performance of AWS Glue jobs involves several strategies that target different aspects of the ETL (Extract, Transform, Load) process. Here are some key practices. 1. Optimize Job Scripts Partitioning : Ensure your data is properly partitioned. Partitioning divides your data into manageable chunks, allowing parallel processing and reducing the amount of data scanned. Filtering : Apply pushdown predicates to filter data early in the ETL process, reducing the amount of data processed downstream. Compression : Use compressed file formats (e.g., Parquet, ORC) for your data sources and sinks. These formats not only reduce storage costs but also improve I/O performance. Optimize Transformations : Minimize the number of transformations and actions in your script. Combine transformations where possible and use DataFrame APIs which are optimized for performance. 2. Use Appropriate Data Formats Parquet and ORC : These columnar formats are efficient for storage and querying, signif

How to Write Recursive Shell Script in Bash Terminal

Here's a simple bash recursive function. To write it, you can use JavaScript and other scripting languages. Below is the example that shows you how to write a recursive function in the bash shell. Recursive function Precisely, calling the same function within it is called the Recursive function. We call itself and its contents.  Moreover, the recursive functions go in the loop due to self-calling. While writing the code, ensure it has a condition that breaks the loop. Recursive logic #!/bin/bash for ((i = 1; i < 65; i++)) { ((arr[i - 1] = i)) } i = 1; key = 8 function linear_search { echo "Element value: ${arr[$i-1]}" if ((arr[i - 1] == key)) then echo "Linear search found $key on attempt $i" return 0 => it breaks the loop else ((i++)) linear_search = > Calling th same script fi } linear_search = > Calling the same script Output Here is the dissection of the output. Displayed the array's elements until match-condition occurs. In the end, a displa

Here is Sample Logic to get Random numbers in Bash

Here's a bash script to generate a random number. You can use this logic to generate a random number, and it is useful for AWS engineers. Random number Script - Here's sample logic to get a random number RANDOM=$$ # Set the seed to the PID of the script UPPER_LIMIT=$1 RANDOM_NUMBER=$(($RANDOM % $UPPER_LIMIT + 1)) echo "$RANDOM_NUMBER" If you select UPPER_LIMIT as 100, then the result would be a pseudo-random number between 1 and 100. Her is the output after executing the script Related posts Structured Vs. Un-structured data