Featured Post

8 Ways to Optimize AWS Glue Jobs in a Nutshell

  Improving the performance of AWS Glue jobs involves several strategies that target different aspects of the ETL (Extract, Transform, Load) process. Here are some key practices. 1. Optimize Job Scripts Partitioning : Ensure your data is properly partitioned. Partitioning divides your data into manageable chunks, allowing parallel processing and reducing the amount of data scanned. Filtering : Apply pushdown predicates to filter data early in the ETL process, reducing the amount of data processed downstream. Compression : Use compressed file formats (e.g., Parquet, ORC) for your data sources and sinks. These formats not only reduce storage costs but also improve I/O performance. Optimize Transformations : Minimize the number of transformations and actions in your script. Combine transformations where possible and use DataFrame APIs which are optimized for performance. 2. Use Appropriate Data Formats Parquet and ORC : These columnar formats are efficient for storage and querying, signif

Tail Command in Linux: A Comprehensive Overview

The tail in Linux is handy command. You can check the last lines of a file in Linux/Unix operating systems.

You can use it to display last lines from single file, display last lines from multiple files, display the last entries of log files.

Tail Command in Linux

During production support the usage of Tail command is helpful since you can check latest logs quickly. Here are the top Tail command examples.

#1 Display last lines in a file (Tail file Linux)

Here's the tail command that shows last three lines of a file.

cat sample.txt | tail -3

It displays last 3 lines of a file. The same command you can use as

tail -3 sample.txt

#2 Display last lines of multiple files

There are three files. sample2.txt, sample3.txt, sample4.txt. The command displays the last 3 lines from all the three files.

tail -3 sample[2-4].txt

Tail command

#3 Tail -f option (Tail f Linux)

The –f option is to check status of long-running process that is redirecting output to a file. For example, if you invoke the below command, the status it writes to the output.

find . -print |xargs grep -i abc </tmp/abc &

Using the -f option you can see contents of the file /tmp/abc whenever it is updated: 

tail -f /tmp/abc




Popular posts from this blog

How to Fix datetime Import Error in Python Quickly

Explained Ideal Structure of Python Class

How to Check Kafka Available Brokers