Featured Post

8 Ways to Optimize AWS Glue Jobs in a Nutshell

  Improving the performance of AWS Glue jobs involves several strategies that target different aspects of the ETL (Extract, Transform, Load) process. Here are some key practices. 1. Optimize Job Scripts Partitioning : Ensure your data is properly partitioned. Partitioning divides your data into manageable chunks, allowing parallel processing and reducing the amount of data scanned. Filtering : Apply pushdown predicates to filter data early in the ETL process, reducing the amount of data processed downstream. Compression : Use compressed file formats (e.g., Parquet, ORC) for your data sources and sinks. These formats not only reduce storage costs but also improve I/O performance. Optimize Transformations : Minimize the number of transformations and actions in your script. Combine transformations where possible and use DataFrame APIs which are optimized for performance. 2. Use Appropriate Data Formats Parquet and ORC : These columnar formats are efficient for storage and querying, signif

The Linux Commands List That Give Deep Insights on Usage

Here is the list of Linux commands, which explains all the daily-use commands. And each command has showed with an example so you can get deep insights quickly.

List of Linux commands

1. What is a command shell?

A program that interprets commands Allows a user to execute commands by typing them manually at a terminal, or automatically in programs called shell scripts. A shell is not an operating system. It is a way to interface with the operating system and run commands.

2. What is BASH?

BASH = Bourne Again Shell

Bash is a shell written as a free replacement to the standard Bourne Shell (/bin/sh) originally written by Steve Bourne for UNIX systems. It has all of the features of the original Bourne Shell, plus additions that make it easier to program with and use from the command line. Since it is Free Software, it has been adopted as the default shell on most Linux systems.

3. How is BASH different from the DOS command prompt?

Case Sensitivity: In Linux/UNIX, commands and filenames are case sensitive, meaning that typing “EXIT” instead of the proper “exit” is a mistake.

\” vs. “/”: In DOS, the forward-slash “/” is the command argument delimiter, while the backslash “\” is a directory separator. In Linux/UNIX, the “/” is the directory separator, and the “\” is an escape character. More about these special characters in a minute!

Character Description \ Escape character. If you want to reference a special character, you must “escape” it with a backslash first.


touch /tmp/filename\* 

/ Directory separator used to separate a string of directory names.



. Current directory. Can also “hide” files when it is the first character in a filename is '.' 

Parent directory ~ 
User's home directory * 
Represents 0 or more characters in a filename, or by itself, all files in a directory. 


pic*2002 can represent the files pic2002, picJanuary2002, picFeb292002, etc. 

? Represents a single character in a filename. 


hello?.txt can represent hello1.txt, helloz.txt, but not hello22.txt [] Can be used to represent a range of values, e.g. [0-9], [A-Z], etc. 


hello[0-2].txt represents the names hello0.txt, hello1.txt, and hello2.txt 

The | “Pipe”. Redirect the output of one command into another command.


ls | more > Redirect output of a command into a new file. If the file already exists, overwrite it. 


ls>myfiles.txt>>Redirect the output of command onto the end of an existing file. 


echo “Mary 555-1234”>>phonenumbers.txt < Redirect a file as input to a program. 


more < phonenumbers.txt; 

Command separator. Allows you to execute multiple commands on a single line. 

Example: cd /var/log; 

less messages 

& & Command separator as above, but only runs the second command if the first one finished without errors. 


cd /var/logs & & less messages 

& Execute a command in the background, and immediately get your shell back. 

Example: find / -name core>/tmp/corefiles.txt & Filenames:

The DOS world uses the “eight dot three” filename convention, meaning that all files followed a format that allowed up to 8 characters in the filename, followed by a period (“dot”), followed by an option extension, up to 3 characters long (e.g. FILENAME.TXT). In UNIX/Linux, there is no such thing as a file extension.

Periods can be placed at any part of the filename, and “extensions” may be interpreted differently by all programs, or not at all.

4. What are Special Characters?

Before we continue to learn about Linux shell commands, it is important to know that there are
many symbols and characters that the shell interprets in special ways. This means that certain
typed characters:

  • cannot be used in certain situations,
  • maybe used to perform special operations, or,
  • must be “escaped” if you want to use them in a normal way.

5. What are Executing Commands?

The Command PATH:

Most common commands are located in your shell's “PATH”, meaning that you can just type the name of the program to execute it.


Typing “ ls” will execute the “ ls” command.

Your shell's “PATH” variable includes the most common program locations, such as
/bin, /usr/bin, /usr/X11R6/bin, and others.

To execute commands that are not in your current PATH, you have to give the complete location of the command.



./program (Execute a program in the current directory)

~/bin/program (Execute program from a personal bin directory)

Command Syntax

Commands can be run by themselves, or you can pass in additional arguments to make them do
different things. 

Typical command syntax can look something like this:

command [-argument] [-argument] [--argument] [file]


ls List files in the current directory
  • ls -l Lists files in “long” format
  • ls -l --color As above, with colorized output
  • cat filename Show contents of a file
  • cat -n filename Show contents of a file, with line numbers

6. How to get help?

When you're stuck and need help with a Linux command, help is usually only a few keystrokes
away! Help on most Linux commands is typically built right into the commands themselves,
available through online help programs (“man pages” and “info pages”), and of course online.

Using a Command's Built-In Help. Many commands have simple “help” screens that can be invoked with special command flags. These flags usually look like “-h” or “--help”.


grep --help

Online Manuals: “Man Pages”

The best source of information for most commands can be found in the online manual pages, known as “man pages” for short. To read a command's man page, type “man command”.


man ls ==> Get help on the “ls” command.

man A manual about how to use the manual!

To search for a particular word within a man page, type “/word”. To quit from a man page, just
type the “Q” key.

Sometimes, you might not remember the name of Linux command and you need to search for it.
For example, if you want to know how to change a file's permissions, you can search the man page
descriptions for the word “permission” like this:
man -k permission

If you look at the output of this command, you will find a line that looks something like:
chmod (1) - change file access permissions.

Now you know that “chmod” is the command you were looking for. Typing “man chmod” will
show you the chmod command manual page!

Info Pages

Some programs, particularly those released by the Free Software Foundation, use info pages as
their main source of online documentation.

Info pages are similar to man page, but instead of being displayed on one long scrolling screen, they are presented in shorter segments with links to other pieces of information. Info pages are accessed with the “info” command, or on some Linux distributions, “pinfo” (a nicer info browser).

For example: 

info df  ==> Loads the “df” info page.

7. How to Navigating the Linux Filesystem?

The Linux filesystem is a tree-like hierarchy of directories and files. At the base of the filesystem is the “/” directory, otherwise known as the “root” (not to be confused with the root user). Unlike DOS or Windows filesystems that have multiple “roots”, one for each disk drive, the Linux filesystem mounts all disks somewhere underneath the / filesystem. The following table describes many of the most common Linux directories.

The Linux Directory Layout

Directory Description

The nameless base of the filesystem. All other directories, files, drives, and devices are attached to this root. Commonly (but incorrectly) referred to as the “slash” or “/” directory. The “/” is just a directory separator, not a directory itself.

/bin Essential command binaries (programs) are stored here (bash, ls, mount, tar, etc.)

/boot Static files of the boot loader.

/dev Device files. In Linux, hardware devices are accessed just like other files, and
they are kept under this directory.

/etc Host-specific system configuration files.

/home Location of users' personal home directories (e.g. /home/susan).

/lib Essential shared libraries and kernel modules.

/proc Process information pseudo-filesystem. An interface to kernel data structures.

/root The root (superuser) home directory.

/sbin Essential system binaries (fdisk, fsck, init, etc).

/tmp Temporary files. All users have permission to place temporary files here.

/usr The base directory for most shareable, read-only data (programs, libraries,
documentation, and much more).

/usr/bin Most user programs are kept here (cc, find, du, etc.).

/usr/include Header files for compiling C programs.

/usr/lib Libraries for most binary programs.

/usr/local “Locally” installed files. This directory only really matters in environments
where files are stored on the network. Locally-installed files go in

/usr/local/bin, /usr/local/lib, etc.). Also often used for
software packages installed from source, or software not officially shipped
with the distribution.

/usr/sbin Non-vital system binaries (lpd, useradd, etc.)

/usr/share Architecture-independent data (icons, backgrounds, documentation, terminfo,
man pages, etc.).

/usr/src Program source code. E.g. The Linux Kernel, source RPMs, etc.

/usr/X11R6 The X Window System.

/var Variable data: mail and printer spools, log files, lock files, etc.

Commands for Navigating the Linux Filesystems

The first thing you usually want to do when learning about the Linux filesystem is take some time to look around and see what's there! These next few commands will:
  • Tell you where you are,
  • take you somewhere else,
  • show you what's there.
The following table describes the basic operation of the pwd, cd, and ls commands, and compares them to certain DOS commands that you might already be familiar with.

Linux Command DOS Command Description

pwd cd “Print Working Directory”. Shows the current location in the directory tree.
cd cd, chdir “Change Directory”. When typed all by itself, it returns you to your home directory.
cd directory Change into the specified directory name.


cd /usr/src/linux

  • cd ~ “~” is an alias for your home directory. It can be used as a shortcut to your “home”, or other
  • directories relative to your home.
  • cd .. cd.. Move up one directory. For example, if you are in /home/vic and you type “cd ..”, you will end up in /home.
  • cd - Return to the previous directory. An easy way to get back to your previous location!
  • ls dir /w List all files in the current directory, in column format.
  • ls directory dir directory List the files in the specified directory.


ls /var/log

ls -l dir List files in “long” format, one file per line. This also shows you additional info about the file, such as ownership, permissions, date, and size.

ls -a dir /a List all files, including “hidden” files. Hidden files are those files that begin with a “.”, e.g.

The .bash_history file in your home directory.

ls -ld directory

A “long” list of “directory”, but instead of showing the directory contents, show the directory's detailed information. For example, compare the output of the following two commands:
  • ls -l /usr/bin
  • ls -ld /usr/bin
  • ls /usr/bin/d* dir d*.* List all files whose names begin with the letter “d” in the /usr/bin directory.


Popular posts from this blog

How to Fix datetime Import Error in Python Quickly

How to Check Kafka Available Brokers

SQL Query: 3 Methods for Calculating Cumulative SUM