Linux/Unix CLI commands

APT

Remove a PPA repository

sudo add-apt-repository -r $repo_name

or

sudo rm -i /etc/apt/sources.list.d/$repo_name.list

Find out which package provides the given file

apt-file search <file_absolute_path>

List files in a package

apt-file list $package

List installed packages

apt list --installed

List a specific package

apt list -a $package_name

Search packages by name using wildcard

dpkg -l $name_pattern
# e.g. dpkg -l "php8*"

ag

• By default, only searches text files

• GitHub - ggreer/the_silver_searcher (opens in a new tab)

ag - Search for a pattern in files with the specified file name pattern

ag $pattern -G $file_name_pattern

ag - List files matching a name pattern

ag -g $file_name_pattern $start_directory

ag - List all file types

ag --list-file-types

ag - List files with matches without printing the matching lines

ag -l $pattern

at

Get an overview of the pending jobs for the current user

atq

or

at -l

alternatives

• All alternatives symbolic links are under /etc/alternatives.

Display details about an alternative

alternatives --display $name
# e.g. alternatives --display java

Interactively select an alternative

alternatives --config $name

List all alternatives

alternatives --list

Set an alternative

sudo alternatives --set $name <command>

Choose an alternative from list

sudo update-alternatives --config java

awk

• Red Hat - Enable Sysadmin - A beginner's guide to gawk (opens in a new tab)

Print all users name and UID

awk -F: '{ printf("%5s %s\n", $3, $1) }' /etc/passwd | sort -n

Change text to lowercase (awk)

echo 'text' | awk '{print toupper($0)}'

base64

Encode text to base64 format

echo -n 'hello' | base64

or

echo -n 'hello' | openssl enc -a

Encode a text file to base64 format

base64 $input_file > $output_file

Encode without line breaks

This can be used to avoid cross-platform issues

echo -n 'hello' | base64 -w0

Decode text from base64 format

• % at the end of decoded text means the text doesn't end with a new line

• Example

  echo -n 'YWRtaW4=' | base64 -d

  or

  openssl base64 -d <<< 'YWRtaW4='

basename

Get the file name from full path

basename -- $file_path

# e.g.
basename -- /home/user_vf/Downloads/abc.txt
> abc.txt

Get the file extension

# Works in shell script
local filename=README.md
extension="${filename##*.}"
echo $extension
> md

Get the file name without extension

basename $file | cut -f 1 -d '.'

or

# Works in shell script
local filename=README.md
filename_without_ext="${filename%.*}"
echo $filename_without_ext

Get the name of the current directory

basename $PWD

bg

Move a running job to background

1. Ctrl + Z: Suspends the running job

2. bg %<JOB_ID>: Resume the suspended job in background

   Job ID is the sequence number, not the PID

3. Use jobs -l to check job status

busybox

• sh (ash)

  • Range not supported, must use enums or seq

    sh -c "for i in 1 2 3 4 5 6 7 8 9 10; do echo \"Welcome \$i times\"; done"

    sh -c "for i in \$(seq 1 10); do echo \"Welcome \$i times\"; done"

cat

Show non-printing symbols

cat -e package.json

Input multi-line string in shell

# Input 'EOF' to end
❯ cat <<EOF > build-image.sh
heredoc> docker build -t playground-messaging-service:0.1 .
heredoc> EOF

• Resources

  • Bash Heredoc (opens in a new tab)

chage

Show account aging info

sudo chage -l $username

chattr

Make a file or directory immutable to changes and deletion, even by superuser

chattr +i $path_to_file_or_directory

Make a file or directory mutable to changes and deletion

chattr -i $path_to_file_or_directory

chmod

• Resources

  • Chmod Calculator (opens in a new tab)

• SUID, SGID, and Sticky bit

  • Sticky bit: a file or folder created inside a sticky bit-enabled folder could only be deleted by the creator itself
  • SUID: the executable which has the SUID set runs with the ownership of the program owner. That is, if you own an executable, and another person issues the executable, then it runs with your permission and not his. The default is that a program runs with the ownership of the person executing the binary.
  • SGID: The SGID bit is the same as of SUID, only the case is that it runs with the permission of the group. Another use is it can be set on folders, making any files or folders created inside the SGID set folder to have a common group ownership.
  • SUID, SGID and Sticky bit use the first byte of the permission, i.e. X644
  • SUID only work on files, while SGID work on both files and directories.
  • You can only apply the sticky bit to directories.
  • If the “s“, “g“, or “t” indicators appear in uppercase, the executable bit (x) hasn’t been set.

Update permissions of all target directories, excluding files

• find <directory> -type d -exec chmod <mode/octal-mode> -- {} +

  + sign is expanded by find to the list of the file paths (or part of it, if too long).

  -- indicates the end of command line options. This prevents a file starting with a hyphen from being interpreted as a command line option as it would come after the --.

Display verbose log to confirm the permission changes

chmod -v <mode/octal-mode> <file...>

Add SUID

sudo chmod u+s $file_or_directory

Find files with SUID set

find $directory -perm /1000

Add SGID

sudo chmod g+s $directory

Find files with SGID set

find $directory -perm /2000

chsh

Change login shell for the current user

chsh -s $full_path_of_shell

cmp

Compare 2 text files

No output means 2 files are identical

cmp $file1 $file2

Compare 2 binary files

# '-l' flag shows the byte offset and byte value of the differing bytes
# '-b' flag shows the ASCII representation of those bytes
cmp -l -b $file1 $file2

conntrack

• Cloudflare Blog - Conntrack tales - one thousand and one flows (opens in a new tab)
• Fedora Magazine - Network address translation part 2 – the conntrack tool (opens in a new tab)
• ArthurChiao's Blog - Connection Tracking (conntrack): Design and Implementation Inside Linux Kernel (opens in a new tab)

List all connections

sudo conntrack -L

cp

Copy file(s) preserving attributes

# Default attribute list: `mode,ownership,timestamps`
cp -p / --preserve ...

# Specific attributes
cp -p=mode ...

csplit

curl

• Notes

  • http:// can be omitted
  • GET is the default HTTP method, so it can be omitted
  • 80 is the default port for HTTP, so it can be omitted

• README - Everything curl (opens in a new tab)

  Enhance your understanding of HTTP protocol with detailed explanations on HTTP headers

Show HTTP response headers only

curl -I $URL

Specify HTTP method for the request

curl -X $HTTP_METHOD $URL

Download HTTP response as a file with specified name

curl -o $FILE_NAME $URL

Follow URL and redirection if any, and download HTTP response as a file keeping original name

curl -OL $URL

Send HTTPS requests without verification

This will skip verifying the remote certificate.

curl -k $URL

Send HTTPS requests with verification by the specified CA certificate

curl --cacert $URL

Send raw text via Telnet

curl telnet://$host:$port <<< "GET / HTTP/1.1"

Upload a file using POST method

curl -F "file=@${file_path}" $URL

Use a HTTP proxy server

# HTTP proxy
export http_proxy=http://$proxy_ip:$proxy_port
# HTTPS proxy
export https_proxy=https://$proxy_ip:$proxy_port

# e.g.
# avsc.tar.gz is under the current directory
# POST method is inferred, so it can be omitted
curl -F "file=@avsc.tar.gz" localhost:8080/upload

date

Convert Epoch seconds to a date

# e.g. `date -d @1682924719`
date -d @$SECONDS_SINCE_THE_EPOCH

Convert a date to Epoch seconds

# e.g. `date -d "2021-09-28 10:00:00" +%s`
date -d $date +%s

Convert a date to Epoch milliseconds

# e.g. `date -d "2021-09-28 10:00:00" +%s%3N`
date -d $date +%s%3N

Get current date and time in UTC without space or special characters

date -u +"%Y%m%dT%H%M%SZ"

Get current date and time in ISO-8601 format with the specified precision

date -Ihours
> 2024-08-15T22+10:00

date -Iminutes
> 2024-08-15T22:11+10:00

date -Iseconds
> 2024-08-15T22:09:58+10:00

date -Ins
> 2024-08-15T22:10:33,245135536+10:00

delv

• Eleven, twelve; dig and delv: BIND 9.10 (opens in a new tab)

Show entire DNSSEC validation chain

delv @$nameserver $domain +vtrace

df

• Alternative

  • GitHub - muesli/duf (opens in a new tab)

diff

Compare 2 text files side by side

diff -y $file1 $file2

diff - Compare 2 text files and report even when the files are identical

diff -s $file1 $file2

diff - Compare 2 binary files and report even when the files are identical

diff -s $file1 $file2

dig

• Resources

  • How to find a website’s DNS (name server) address (opens in a new tab)
  • Rackspace Customer Support Documentation - Use dig to query nameservers (opens in a new tab)

Show detailed answer of a domain for all record types

dig $domain +noall +answer ANY

Reverse DNS lookup by IP address

dig +noall +answer -x $IP

or

nslookup -type=ptr $IP

Show name servers of a domain

dig NS/ns $domain

Set default options for dig command

vim ~/.digrc

+noall
+answer

dircolors

Print default color codes

dircolors -p

• Resources

  • Unix & Linux Stack Exchange - What causes this green background in ls output? (opens in a new tab)

dirname

Get the parent directory of the given path

dirname $path

dnf / yum

Display details about a package

dnf info $package_name

List installed packages (RPM)

dnf list installed

or

yum list installed

List dependencies of a package

# with yum-utils
repoquery --requires --resolve $package_name
 
# with DNF
dnf repoquery --requires --resolve $package_name

Find which packages containing files matching the given pattern

dnf provides "*/${file_name}"

# e.g. Look for missing header file:
dnf provides "*/sd-daemon.h"
 
# e.g. Look for executable:
dnf provides "*/bin/pgbench"

or

yum provides "*/${file_name}"

Install local RPM package

dnf install "${package_name.rpm}"

or

yum localinstall "${package_name.rpm}"

Install a package from a specific repo

dnf install ${package_name} --repo ${repo_id}

List files in a package (RPM)

dnf repoquery -l ${package_name}

or

sudo yum install yum-utils
 
repoquery -l $package_name

or

rpm -ql $package_name

Add a repo

1. Create a text file with .repo extension, with the format specified in Setting [repository] Options (opens in a new tab)
2. Copy the .repo file to /etc/yum.repos.d
3. sudo dnf config-manager --add-repo /etc/yum.repos.d/"$filename.repo"

• Resources

  • Adding or removing software repositories in Fedora (opens in a new tab)

Remove a repo (RPM)

sudo rm /etc/yum.repos.d/"$file_name.repo"

List enabled repos (RPM)

dnf repolist --enabled

List disabled repos (RPM)

dnf repolist --disabled

Disable a repo (RPM)

sudo dnf config-manager --set-disabled $repo_id

Enable a repo (RPM)

sudo dnf config-manager --set-enabled $repo_id

Show which repo is defined in which .repo file (RPM)

grep -E "^\[.*]" /etc/yum.repos.d/*

Using modules in Fedora

• Resources

  • Using modules in Fedora (opens in a new tab)

dpkg

• Unmet dependencies cannot be installed due to conflicting installed package, and apt cannot install or remove anything

  1. Use dpkg to uninstall referencing packages: dpkg -r <PACKAGE>
  2. Now apt can install again
  3. Remove the conflicting installed package
  4. Install the desired package

List files of the specified deb file

• dpkg --contents <deb_file>

List files of the installed package

• dpkg -L <package_name>

Find out which package provides the given file (dpkg)

• dpkg-query -S <file_absolute_path>

  or

• dpkg -S <file_absolute_path>

  e.g. dpkg -S /usr/bin/pip3

du

By default, current directory is specified implicitly.

• Alternative

  • GitHub - bootandy/dust (opens in a new tab)

Disk usage of all directories directly under the current directory, excluding files

du -d1

or

du -d 1

Disk usage of all files recursively, excluding directories (can be used to find out the biggest files)

find . -type f -exec du -ha {} + | sort -h | less

ethtool

Show network interface info

sudo ethtool $interface

fd

Note: use -u to include all files

fd - Search files by extension

fd -u -e $extension

fd - Search files by size

# Search file with the specified size
fd -u -t f -S $size

# Search file with the size larger than the specified size
fd -u -t f -S +$size

# Search file with the size smaller than the specified size
fd -u -t f -S -$size

fd - Search files with the given pattern

fd -u $pattern

fd - Execute command for each match

# e.g.
fd -e $extension -x cp {} .

• Placeholder syntax

  The -x and -X options take a command template as a series of arguments (instead of a single string). If you want to add additional options to fd after the command template, you can terminate it with a \;.

  The syntax for generating commands is similar to that of GNU Parallel:

  • {}: A placeholder token that will be replaced with the path of the search result (documents/images/party.jpg).
  • {.}: Like {}, but without the file extension (documents/images/party).
  • {/}: A placeholder that will be replaced by the basename of the search result (party.jpg).
  • {//}: The parent of the discovered path (documents/images).
  • {/.}: The basename, with the extension removed (party).

  If you do not include a placeholder, fd automatically adds a {} at the end.

fdisk

Identify USB drive name

• sudo fdisk -l

fg

Bring a specific job to the foreground

• fg <job-command>

file

Check file type as well as line separator

$ file vividfire-wiki.iml
vividfire-wiki.iml: XML 1.0 document, ASCII text, with CRLF line terminators

find

{} is called a placeholder. This placeholder will hold the result found by find.

Follow symbolic links: -L flag

Example:

find -L $directory -type f -iname "*.log"

Sensitive/insensitive name: -name/-iname flag

Example:

find $directory -type f -name "*.log"

List all direct children under the specified directory

List all directories under the specified directory

find $directory -maxdepth 1 -mindepth 1 -type d

List all files under the specified directory

find $directory -maxdepth 1 -mindepth 1 -type f

Get files with the given user

> find $directory -user $username
# or
> find $directory -uid $uid

Get files with the given group

find $directory -group $groupname

or

find $directory -gid $gid

Get files modified in the last n min

find $directory -mmin -$n -type f

# e.g.
find . -mmin -5 -type f

Execute a command for each match

• find <file-pattern> -exec <command> {} \;

  e.g. find . -type f -exec file {} \;

Execute a command with all matches

find $file_pattern -exec $command {} +

Example:

find . -type f -exec file {} +

# Output
./PostgreSQL.md:                                                            UTF-8 Unicode text
./Prawn-leek-spaghetti/prawn-leek-spaghetti-83115-1.jpeg:                   JPEG image data, progressive, precision 8, 720x480, components 3
./Accounting.md:                                                            UTF-8 Unicode text
./Apache-Kafka.md:                                                          UTF-8 Unicode text, with very long lines
./Kubernetes.md:                                                            UTF-8 Unicode text, with very long lines
./Microsoft-Azure.md:                                                       UTF-8 Unicode text

find - Search files by size

• Search files with the specified size

  find . -type f -size $size

• Search files with the size larger than the specified size

  find . -type f -size +$size

• Search files with the size smaller than the specified size

  find . -type f -size -$size

find,sed - Replace text in files

find . -type f -name '$file_pattern' -readable -writable -exec sed -n 's/$from_pattern/$to_pattern/gp' {} \;

Dry run search and replace

find . -type f -name '$file_pattern' -readable -writable -exec sed -i 's/$from_pattern/$to_pattern/gp' {} \;

Use -i to replace text in place

findmnt

List mount point of a device

findmnt $device

# Example:
findmnt /dev/sdb1

firewalld

• Frontend for managing Netfilter rules

• Zones

  • A zone defines the trust level for a interface.

  • A interface can be assigned to only one zone at a time.

  • A zone can be used by many interfaces.

  • If an interface is not assigned to any zone, it will be assigned to the default zone.

  • Predefined zones, from the least trusted to the most trusted

    • drop

      The most untrusted zone. All incoming connections are dropped without reply and only outgoing connections are possible.

    • block

      Similar to the above, but instead of simply dropping connections, incoming requests are rejected with an icmp-host-prohibited or icmp6-adm-prohibited message.

    • public

      Represents public, untrusted networks. You don’t trust other computers but may allow selected incoming connections on a case-by-case basis.

    • external

      External networks in the event that you are using the firewall as your gateway. It is configured for NAT masquerading so that your internal network remains private but reachable.

    • dmz

      Used for computers located in a DMZ (isolated computers that will not have access to the rest of your network). Only certain incoming connections are allowed.

    • work

      Used for work machines. Trust most of the computers in the network. A few more services might be allowed.

    • home

      A home environment. It generally implies that you trust most of the other computers and that a few more services will be accepted.

    • internal

      The other side of the external zone, used for the internal portion of a gateway. The computers are fairly trustworthy and some additional services are available.

    • trusted

      Trust all of the machines in the network. The most open of the available options and should be used sparingly.

• Services

  • A service is a predefined list of local ports, destinations, or firewall helper modules.

  • Built-in services are defined in /usr/lib/firewalld/services

  • Custom services are defined in /etc/firewalld/services

• ICMP types

  • firewalld can use the predefined ICMP types to limit how much diagnostic information a system provides to potentially hostile systems.

• Resources

  • man - firewall-cmd (opens in a new tab)
  • Red Hat Product Documentation - Chapter 1. Using and configuring firewalld (opens in a new tab)

firewalld - List all zones

firewall-cmd --get-zones | tr " " "\n" | sort

firewalld - List all active zones

Output shows the active zones and assigned interfaces

$ firewall-cmd --get-active-zones
FedoraServer (default)
  interfaces: enp2s0
docker
  interfaces: br-cf3c17e668e9 br-f40f08285ab2 docker0

firewalld - Get information of a single zone

$ sudo firewall-cmd --zone=$zone --list-all
FedoraServer (active)
  target: default
  icmp-block-inversion: no
  interfaces: enp2s0
  sources:
  services: actualbudget cockpit dhcpv6-client jellyfin ssh
  ports:
  protocols:
  forward: yes
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

firewalld - Get all services

firewall-cmd --get-services | tr " " "\n" | sort

Note:

• Output includes both predefined and custom services

• Custom service definition files are under /usr/lib/firewalld/services

firewalld - Get information of a single service

sudo firewall-cmd --info-service=$service

# e.g.
> sudo firewall-cmd --info-service=Jellyfin--8096

Jellyfin--8096
  ports: 8096/tcp
  protocols:
  source-ports:
  modules:
  destination:
  includes:
  helpers:

firewalld - Add a new custom service

sudo firewall-cmd --permanent --new-service-from-file=${service.xml} --name=$service_name

Example

1. Create a file called actualbudget.xml in /etc/firewalld/services folder with the following content.

   <?xml version="1.0" encoding="utf-8"?>
   <service>
     <description>actualbudget</description>
     <port port="5006" protocol="tcp"/>
   </service>

2. Add new service rules

   # File will be copied to /etc/firewalld/services with correct permissions
   sudo firewall-cmd --permanent --new-service-from-file=actualbudget.xml --name=actualbudget

3. Enable traffic for the service permanently

   # Add the service to a public zone to enable traffic
   sudo firewall-cmd --permanent --zone=FedoraServer --add-service=actualbudget
    
   sudo firewall-cmd --reload

firewalld - Remove a service from a zone

sudo firewall-cmd --zone=$zone --remove-service=$service

firewalld - Allow HTTP traffic temporarily (revert after service restart)

sudo firewall-cmd --zone=$zone --add-service=$service

firewalld - Allow HTTP traffic permanently (stay effective after service restart)

Make changes without --permanent flag for experimental purposes

sudo firewall-cmd --zone=$zone --add-service=$service --permanent

firewalld - Configure an existing service

fw --permanent --service=myservice --set-description=description
fw --permanent --service=myservice --set-short=description
fw --permanent --service=myservice --add-port=portid[-portid]/protocol
fw --permanent --service=myservice --add-protocol=protocol
fw --permanent --service=myservice --add-source-port=portid[-portid]/protocol
fw --permanent --service=myservice --add-module=module
fw --permanent --service=myservice --set-destination=ipv:address[/mask]

or

Edit the service file directly

# user-defined service
sudo vim /etc/firewalld/services/$service.xml
# predefined service
sudo vim /usr/lib/firewalld/services/$service.xml

firewalld - Save runtime changes to permanent

sudo firewall-cmd --runtime-to-permanent

firewalld - Reload firewall configuration

sudo firewall-cmd --reload

firewalld - Check if a service is allowed in a zone

sudo firewall-cmd --zone=$zone --query-service=$service

# e.g.
sudo firewall-cmd --zone=FedoraServer --query-service=ssh

firewalld - Check if a port is allowed in a zone

sudo firewall-cmd --zone=$zone --query-port=$port

# e.g.
sudo firewall-cmd --zone=FedoraServer --query-port=5006/tcp

firewalld - Firewall configuration with Docker containers

• Docker Docs - Packet filtering and firewalls (opens in a new tab)

fzf

• Setup

  • There is an install script under the fzf installation directory, and use install --help for help.

• Type anything on the command line followed by ** and press TAB to trigger fzf.

  ** is the default trigger sequence, configurable by FZF_COMPLETION_TRIGGER environment variable.

  e.g.

  • vim ~/path/** + <TAB>
  • kill ** + <TAB>
  • ssh ** + <TAB>

• Key bindings

  • CTRL + j - Move cursor up

  • CTRL + k - Move cursor down

  • ALT + c - Triggers fzf, select and change to one directory under current directory

  • CTRL + t - Triggers fzf, select one file under current directory

  • CTRL + r - Triggers fzf, select one command from history

• Resources

  • GitHub - ohmyzsh - plugins - fzf (opens in a new tab)
  • GitHub - fzf - Vim integration (opens in a new tab)

getent

List all users on the system

getent passwd | cut -d: -f1 | sort

List all groups on the system

getent group | tr ":" "\t" | column -t

or

less /etc/group

List all members of a group on the system

getent group $groupname | tr ":" "\t" | column -t

getfacl

Get ACL of the specified file

getfacl $file

gpasswd

Add a user to a group

sudo gpasswd -a $username $groupname

grep

• Always use quotes to enclose regular expression, otherwise the expression has to be properly escaped.

• Target file

  grep [OPTION]... $search_pattern $file

• Target directory

  grep -r [OPTION]... $search_pattern $directory

Enable Perl regular expressions, such as \d, \w and \s

grep -P

Display matches without filtering lines without matches / Highlight matches

# Replace ${regex} with the actual regex
grep -E '${regex}|$'

Example

grep -Erw --include='*.java' 'log.info|$' .

Note: must be quoted

Filter out empty lines in file

grep -E -v '^\s*$' $file

Filter out lines starting with # in file

grep -v '^#' $file...

Search specified files only

Example

grep -r --include='*.java' 'search_pattern' $directory

grep variants

egrep

• extended grep, equivalent to grep -E

fgrep

• fixed-string search, not recognizing any regular expression meta-characters as being special, resulting in better performance, equivalent to grep -F

pgrep

List the processes whose names match a given regex

pgrep -l $regex

Use -l to display program name

List the processes run by a given user

pgrep -u $username

List the processes whose names do not match a given regex

pgrep -lv $regex

rgrep

• can recursively descend directories without using find, resulting in much better performance.

• Resources

  • What’s Difference Between Grep, Egrep and Fgrep in Linux? (opens in a new tab)
  • Regular expressions in grep ( regex ) with examples (opens in a new tab)

groupdel

Delete a group

sudo groupdel $groupname

head / tail

Specify the number of lines, by default 10

• head -10 / tail -10

hexdump

Display the content of a file in both hexadecimal and ASCII format

• hexdump -C <file>

Display the first 8 bytes of a file in hexadecimal format

• hexdump -n 8 <file>

  Can be used to detect file type

host

Lookup A, AAAA, and MX records of a domain

• host <domain>

Reverse lookup an IP

• host <IP>

Specify an alternate DNS server to query

• host <domain> <DNS-server-IP>

hostname / hostnamectl

Show hostname

# Need to install `hostname` first, dnf install hostname
hostname

or

cat /etc/hostname

Show system hostname and related info

hostnamectl

Permanently change host name

hostnamectl hostname $new_host_name

List all FQDNs of the machine

• hostname -A | tr " " "\n" | sort

hping3

Use traceroute mode in TCP mode against the target host

hping3 -T -V $target_host

id

List all groups of the specified user

groups $username

List all groups of the current user

groups

or

id -nG

Show the primary group of the current user

id -ng

ipcalc

Calculate subnets from CIDR

ipcalc --all-info $CIDR

Get network info from IP and netmask

ipcalc --all-info $IP $Netmask

e.g. ipcalc --all-info 192.168.50.221 255.255.255.0

Can also get CIDR from this command

Output in JSON format

ipcalc -j | jq

kill

Send SIGKILL to terminate a process forcibly

kill -9 $PID

Send SIGHUP to a daemon process to reload configuration

If a daemon process has a configuration file which is modified after the process has been started, there should be a way to tell that process to reload its configuration file without stopping the process. Many daemons provide this mechanism using a SIGHUP signal handler. When you want to tell the daemon to reload the config, simply send it the SIGHUP signal.

kill -1 $PID

ldapsearch

Authentication with specified bind account

ldapsearch -D $bind_account -w $password

ldd

List dynamic linked libraries of an executable

ldd $executable

less

less - Display color output

# Interpret color escape sequences
less -R $file

ls

customize files and directories colors

• For linux terminal running on Windows, directories color could be unfriendly to users

  • export LS_COLORS='ow=01;36;40'

    bold (01) cyan (36) dir names with black (40) background

• LSCOLORS Generator (opens in a new tab)

Only list direct directories under the specified directory

• ls -d */

  Caveat: hidden directories are not included

• find . -maxdepth 1 -mindepth 1 -type d

  To include hidden directories

lsblk

Display info about filesystems

sudo lsblk -f

lslogins

Display all accounts in the system

lslogins

Display system accounts in the system

lslogins --system-accs

Display user accounts in the system

lslogins --user-accs

Display details about a single account

lslogins $user_name

lsmod

To check if the KVM modules are enabled

If this command lists kvm_intel or kvm_amd, KVM is properly configured.

lsmod | grep kvm

lsof

• Note

  • Use sudo if the process is started by root

  • By default when you use more than one list option in lsof, they will be ORed.

    Flag	Description
    -P	Do not translate port numbers
    -n	Do not translate host names
    +c0	Display maximum number of characters in COMMAND column

• Resources

  • Linode Docs - How to List Open Files with lsof (opens in a new tab)

Make options to be ANDed

Use -a flag, and all specified options will be effective together

lsof -U -c java -a

List LISTEN TCP connections

sudo lsof -iTCP -sTCP:LISTEN

List ESTABLISHED TCP connections

sudo lsof -iTCP -sTCP:ESTABLISHED

List processes using the specified UDP port

sudo lsof -P -iUDP:<port>
# e.g. `sudo lsof -P -iUDP:8080`

List processes using the specified TCP port

• How to check if port is in use on Linux or Unix (opens in a new tab)

sudo lsof -P -iTCP:<port>
# e.g. `sudo lsof -P -iTCP:8080`

List all TCP ports currently opened by the given processes

lsof -a -iTCP -p <PID1,PID2...>

List all TCP ports currently opened by the given command

# $command argument is a `regex`.
lsof -a -iTCP -c $command

List all files opened by a process

lsof -p $PID

List files for processes executing the specified command

# $command argument could be a regex.
lsof -c $command

List processes using the specified file

lsof $file

List processes that are using any file under the specified directory

lsof +D $directory

List files opened by a user

lsof +u $username

List UNIX domain sockets

lsof -U

ltrace

Display dynamic library calls of a program

ltrace $program

LXC/LXD

• Installation

  • Fedora

    • Linux Containers - LXD - Getting started (opens in a new tab)
    • Getting Started with LXD on Fedora (opens in a new tab)

• Profiles

  • Profiles are instance specific configurations, like AWS EC2 Launch Template.

    Profiles - LXD documentation (opens in a new tab)

    e.g. lxc profile list

man

Search the short manual page descriptions for keywords and display any matches

m -k <keyword>

Open another man page within a man page

• Switch to command mode by typing !

• Input command to open man page

  man <section> <command>

mkdir

Create hierarchical directories

mkdir -p

mount

• How To Mount and Unmount Drives on Linux (opens in a new tab)

  Include most recipes about mounting and unmounting drives

Lists all the file systems mounted yet

mount -l | column -t | sort

List unmounted disks or partitions

The ones without mount point are unmounted.

lsblk -f

Mount a partition

• sudo mount -t <file-system> <partition> <folder>

  e.g. Mount a exfat device

  sudo mount -t exfat /dev/sdb1 /media/T7

netcat

Client/Server socket communications

• Server

  netcat -l <port>

• Client

  • netcat <host> <port>

    Send data interactively

  • echo "text" | netcat <host> <port>

    Send data via piping

Test if the specified host and TCP port is open

• netcat -zv <host> <port>

Test if the specified host and TCP ports are open

• netcat -zv <host> <port-range>

  e.g. netcat -zv <host> 8000-8080

nmap

• TCP SYN scan

  nmap -sS <HOST_NAME/IP_ADDRESS> -p <PORT>

• TCP CONNECT scan

  nmap -sT <HOST_NAME/IP_ADDRESS> -p <PORT>

• UDP scan

  nmap -sU <HOST_NAME/IP_ADDRESS> -p <PORT>

Scan most common ports of a range of hosts

sudo nmap -sS --top-ports=${number-of-ports-to-scan} ${CIDR-notation}

pacman

Search a package

pacman -Ss $package

List files in a package (Pacman)

pacman -Ql $package

perf

pidof

• Functions similar to pgrep

Find the PID(s) of one or multiple running processes

• pidof <program>

ps

• Monitor your Linux system in your terminal with procps-ng (opens in a new tab)

List all processes in full format

ps -ef | less

List process(es) with specified PID(s)

ps -f $(pidof $program) | cat

Display full path of command with cat without truncating

Pretty print command line of a running process

• With column header

  ps -ocmd -p $PID | tr " " "\n"

• Add an equal sign to suppress header

  ps -ocmd= -p $PID | tr " " "\n"

Display environment variables along with command line

• Use output modifier: e

  ps e -ww -ocmd -p $PID | tr " " "\n" > ps.log

Check if a process is attached to a terminal

ps -o tty= -p <PID>
# or just check the output of ps -ef, if tty column is empty, then it's not attached to a terminal

pv

• How to Monitor Progress of (Copy/Backup/Compress) Data using ‘pv’ Command (opens in a new tab)

pv - Copy a file with progress bar

pv $source_file > $destination_file

pv - Compress a file with progress bar

pv $source_file | zstd > $destination_file

readelf

• Resources

  • Wikipedia - Executable and Linkable Format (opens in a new tab)

Display headers of ELF files

readelf -h <ELF-file>

readlink

Get the absolute path of the file the specified file/symlink

• readlink -f <file/symlink>

  If file is a symlink, output will be the path of the file the symlink points to， otherwise it would just be the absolute path of the file.

rename

Rename files with a regex

rename 's/<regex>/<replacement>/' <file-pattern>

repoquery

resolvectl

Check the DNS currently in use by systemd-resolved

resolvectl status

Display DNS per network interface

resolvectl dns

ripgrep

Search pattern in files based on file extension

rga "$pattern" --glob "*.$file_ext"

Example:

rga "java" --glob "*.epub"

rpm

• Red Hat - Enable Sysadmin - RPM and GPG: How to verify Linux packages before installing them (opens in a new tab)

List imported PGP public keys

rpm -qa gpg-pubkey

Find which installed packages containing local files matching the given pattern

rpm -qf $local_file

Display information of a PGP public key

rpm -qi $key_id

e.g. rpm -qi gpg-pubkey-98ab5139-4bf2d0b0

Remove an imported PGP public key

sudo rpm -e $key_id

e.g. sudo rpm -e gpg-pubkey-98ab5139-4bf2d0b0

rpm2cpio

Extract files from a RPM package

rpm2cpio $rpm_file | cpio -idmv

scp

Copy a Local File to a Remote System with the scp Command

scp $local_file $remote_user@$host_ip:$remote_path

• Resources

  • How to Use SCP Command to Securely Transfer Files (opens in a new tab)

Copy a file from remote

scp $remote_user@$host_ip:$remote_path $local_path

sed

• 5.9. sed Examples (opens in a new tab)

Filter out empty lines in file (sed)

sed '/^\s*$/d' $file

• Red Hat - Enable Sysadmin - Manipulating text at the command line with sed (opens in a new tab)

Use Extended Regular Expression (ERE) in sed

sed -E 's/<from-pattern>/<to-pattern>/g' $file

Replace text in place in a file

sed -i 's/<from-pattern>/<to-pattern>/g' $file

Only print lines that match the pattern

-n supporess the default output, and p prints the matched lines.

sed -n 's/<from_pattern>/<to_pattern>/p' $file

e.g.

$ sed -n 's@3.8.2@3.10.4@p' pom.xml
        <gatling-charts-highcharts.version>3.10.4</gatling-charts-highcharts.version>

Debug sed command

sed --debug 's/<from-pattern>/<to-pattern>/g' $file | grep -B 3 -A 4 'MATCHED'

e.g.

$ sed --debug 's@1.35@1.37@g' pom.xml | grep -B 3 -A 4 'MATCHED'
INPUT:   'pom.xml' line 17
PATTERN:         <jmh.version>1.35</jmh.version>
COMMAND: s/1.35/1.37/g
MATCHED REGEX REGISTERS
  regex[0] = 21-25 '1.35'
PATTERN:         <jmh.version>1.37</jmh.version>
END-OF-CYCLE:
        <jmh.version>1.37</jmh.version>

Compare changes

$ sed --debug 's@1.35@1.37@g' pom.xml | grep -B 3 -A 4 'MATCHED' | grep PATTERN
PATTERN:         <jmh.version>1.35</jmh.version>
PATTERN:         <jmh.version>1.37</jmh.version>

Apply changes to the file

$ sed -i --debug 's@1.35@1.37@g' pom.xml | grep -B 3 -A 4 'MATCHED' | grep PATTERN
PATTERN:         <jmh.version>1.35</jmh.version>
PATTERN:         <jmh.version>1.37</jmh.version>

Run sed against multiple files

sed -n 's@<from-pattern>@<to-pattern>@p' $file1 $file2

More files

find $start_dir -type f -iname $file_pattern -print -exec sed -n 's@<from_pattern>@<to_pattern>@p' {} \;

e.g.

$ find . -type f -iname 'pom.xml' -print -exec sed -n 's@1.35@1.37@p' {} \;
./playground.java-concurrency/vertx-concurrency/pom.xml
./spring-microservice/pom.xml
./cq-devops.admin-client/pom.xml
./playground-cryptography/pom.xml
./playground.jvm-performance/pom.xml
        <jmh.version>1.37</jmh.version>
./spring-boot-starter/pom.xml
./playground.vertx-web/pom.xml
./playground.spring-boot-cli/pom.xml
./cq-devops.aws-client/pom.xml
./project-crystal-lover.admin/pom.xml
./devops.maven/pom.xml
./project-crystal-lover.image/pom.xml

setsid

Run a command as a daemon

Closing a session will terminate all related processes for that TTY, which is not desired for system wide processes.

You must disassociate your daemon process from the terminal to avoid being sent signals related to terminal's operation (like SIGHUP when the terminal session ends as well as potentially SIGTTIN and SIGTTOU).

setsid $command > /dev/null 2>&1 < /dev/null &
# To run it as a full daemon from a shell, you'll need to use setsid and redirect its output. You can redirect the output to a logfile, or to /dev/null to discard it.
 
# This will completely detach the process from your current shell (stdin, stdout and stderr). If you want to keep the output in a logfile, replace the first /dev/null with your /path/to/logfile.
 
# You have to redirect the output, otherwise it will not run as a true daemon (it will depend on your shell to read and write output).

shar

Compress a binary file with gzip and create a shar archive

shar -z <binary-file> > <archive.sh>

shc

Compile a shell script into a binary executable

shc -vrf <shell-script> -o <binary-file>

sleep

Run a command after a specified time

sleep $time && $command

ss

Display TCP ports opened by a process

ss -tapn | grep <PID>

Show all UNIX domain sockets (LISTENING and ESTABLISHED)

ss -xan

stat

Display octal file permissions

• stat <file/directory>

Check what init system Linux is using

• sudo stat /proc/1/exe | grep File

strace

Trace system calls and signals of a program

• strace <program>

Trace system calls and signals of a running process

• strace -p <PID>

strings

Display environment variables of a running process

• Retrieve PID of a process

  pidof <program>

  or if it's a Systemd service

  systemctl status <name>.service

• Display environment variables

  sudo strings /proc/<PID>/environ | sort

• Resources

  • Show the environment variables of a running process in Linux (opens in a new tab)

SysV

List all services

service --status-all

Check status of a service

service $service_name status

Start a service

sudo service $service_name start

tar

Package files into a tar archive

tar cvf $TAR_FILE $file1, $file2, $file3 ...

Compress and package files into a tar archive

tar --zstd -cvf $tar_file $dir | $files_to_be_included...

tar -I"zstd -19 -T0" -cvf ${file.tar.zst} ${dir} | ${files_to_be_included}...

Extract files from a tar archive into the specified directory (creating the directory if it does not exist)

tar xvf $tar_file --one-top-level=$dir

Extract files from a tar archive into the specified directory (without creating the directory)

tar xvf $tar_file -C $dir

$dir must exist already.

List files in a tar archive

tar tvf $tar_file

• When working with a tar file, -f must be used to specify the file.

tcpdump

• Resources

  • BPF and tcpdump - Andreas Karis Blog (opens in a new tab)

Print the list of the network interfaces available on the system and on which tcpdump can capture packets

tcpdump -D | grep Connected

BPF filter for potentially HTTP traffic

tcpdump -i ${interface} tcp and (dst port 80 or dst port 8080 or dst port 443)

Capture the packets and write into a pcap file

tcpdump -w ${pcap_file}

Display the contents of a pcap file

tcpdump -r ${pcap_file}

Capture packets with IP address, avoiding DNS lookp

tcpdump -n

Capture packets flows on a particular port

tcpdump -i ${interface} port ${port}

Capture packets for a particular destination IP and port

tcpdump -i ${interface} dst ${ip} and port ${port}

Display MAC addresses on each line

tcpdump -e

telnet

Test if a remote port is open

• telnet <host> <port>

  Use this as a netcat alternative

timeout

Run a command with a time limit

• timeout <time-limit> <command>

  The command will be stopped when the timeout is reached, useful for scheduling stopping a continuously running command.

timedatectl

Display the current system clock time

timedatectl

top

• non-interactive mode

  top -b -n <iteration-count>

• Display command line

  top -c

• Interactive commands

  • h: help

  • z: toggle color mode

  • c: toggle display of command-line

  • i: toggle display of any idle or zombie processes

  • o: filter criteria input

  • u: user processes

  • Shift + P: CPU utilization percentage from high to low

  • d: change refresh interval

    processes that have not used any CPU since the last update will not be displayed.

  • k: kill a process (SIGTERM), prompting a PID

  • Interactive commands can be input at start like top -i.

  • Configuration is saved in .toprc in home directory.

• Resources

  • Boolean World - A Guide to the Linux “Top” Command (opens in a new tab)

List processes with command line arguments

top -bcn1 -w512 | less

tr

• Only works with ASCII characters

Translate characters

• e.g. jcmd 13668 VM.command_line | tr ";" "\n" | tr " " "\n"

  

Delete characters

• tr -d STRING

  e.g. tr -d "[:space:]"

Remove non-printable characters

tr -dc '[[:print:]]'

Change text to lowercase (tr)

• echo "movie" | tr '[:lower:]' '[:upper:]'

  Lower to upper

traceroute

• Resources

  • How to Use the traceroute Command on Linux (opens in a new tab)

umask

• File default permission 666, directory default permission 777, and actual permission will be calculated by umask. If umask = 022, the actual permission would be 666 - 022 = 644

Set umask temporarily for the current session

# e.g. umask 022
umask $permission_sets

Show the umask of a running process

grep '^Umask:' "/proc/$(pidof <program>)/status"

uniq

Compute file checksums and locate duplicates in a directory

find $target_directory -maxdepth 1 -type f -exec md5sum {} \; | uniq -w32 -D

useradd

View the current default options type

useradd -D

userdel

Delete a user and its home directory

sudo userdel -r $username

usermod

Add a user to the given group(s)

usermod -aG <$group1,$group2...> $username

wc

Count characters in a string

echo $STRING | wc -c

wget

Download and print to stdout

• If - is used as file, documents will be printed to standard output, disabling link conversion.

  wget -O - <URL>

• Resources

  • What is the meaning of wget -O - (opens in a new tab)
  • All the Wget Commands You Should Know (opens in a new tab)

xdg-open

Open a file with associated default program from shell

xdg-open $file

xmllint

Pretty print XML to stdout

xmllint --format <file> | less

Pretty print XML to file

xmllint --format <file> > <output-file>

Pretty print XML from pipe

cat $file | xmllint --format - | less

xsel

Copy content to clipboard from shell

echo -n 'can you see me?' | xsel -b

zip

Archive a directory

zip -r $ZIP_FILE $DIR

Show the contents of a zip file

unzip -l $ZIP_FILE

zstd

Archive a directory with zstd compression

tar --zstd -cvf <filename>.tar.zst <dir> / <files-to-be-included...>

Extract a zstd archive

tar --zstd -xvf <filename>.tar.zst

zstd - Specify compression level

• -#

  Selects # compression level 1-19 (default: 3)

zstd - Use multiple threads

• -T#, --threads=#

  Compress using # working threads (default: 1)

  If # is 0, attempt to detect and use the number of physical CPU cores.

  zstd -T0 $file

WSL

Access Windows executable

• Reference executables with file extension, such as clip.exe

  e.g. echo "Hello" | clip.exe

snap not working

sudo apt-get update && sudo apt-get install -yqq daemonize dbus-user-session fontconfig
sudo daemonize /usr/bin/unshare --fork --pid --mount-proc /lib/systemd/systemd --system-unit=basic.target
exec sudo nsenter -t $(pidof systemd) -a su - $LOGNAME
snap version

List distributions available to install

wsl -l -o

List local installed distributions and status

wsl -l -v

Turn on capability to change Windows file permissions from WSL

# In a WSL distribution, create if missing:
# /etc/wsl.conf
[automount]
options = "metadata"

Open file with associated Windows program

• wslview <file>

Tools and utilities

• GitHub - wslutilities/wslu (opens in a new tab)

Environment variables

• The env will only display a list of environment variables that have been exported excluding all shell variables.

• Use printenv command to for the list of all shell variables.

• Child processes cannot export variables back to the parent processes that spawned them.

• set command (opens in a new tab)

  • Options can be specified with either a leading - (enable) or + (disable).

  • set -x

    Print a trace of simple commands and their arguments after they are expanded and before they are executed, useful for debugging shell script.

  • set -e

    Exit immediately if a simple command exits with a non-zero status. Note some exceptions apply.

  • set -a

    Each variable or function that is created or modified is given the export attribute and marked for export to the environment of subsequent commands.

Network

UNIX Domain Sockets

• Resources

  • CloudSavvy IT - What Are Unix Sockets and How Do They Work? (opens in a new tab)
  • Digital Ocean - Understanding Sockets (opens in a new tab)

ip

Colorize output

ip -color

Show IP addresses assigned to all network interfaces

ip -brief a[ddr]