1. To show commit history in reverse chronological order, run:

    git log
   

   

   💡

   If the logs are longer than your screen, it always goes into interactive mode. To escape and come back to the shell, quit git log with q.

2. To display each commit as a single line with its short SHA-1 hash and commit message, run:

    git log --oneline
   

   

3. To show more details about all previous commits, run:

    git log -p
   

   

   💡

   To limit output to the last two commits, you can add the -2 option: git log -p -2

4. To show how many files changed and the number of insertions (+) and deletions (-), run:

    git log --stat
   

   

5. To group commits by author and display commit messages under each author's name, run:

    git shortlog
   

   

   This command can be used to quickly see who contributed the most to a repository, to get an overview of contributions instead of a detailed history, and to prepare contributor statistics for reports or release notes.

6. To visualize some work with branches, run the command:

    git log --graph --oneline
   

   • --graph → Displays a commit graph showing branch relationships.

   • --oneline → Shows each commit in a single line.

We can see that second-branch was created and merged with the master branch.

7. To completely change the log output to display a structured and visually enhanced commit history, use the --pretty option:

    git log --pretty="%C(yellow) Commit: %h %C(magenta)Date: %ad %C(red) Message: %s %C(cyan) Author: %an" --date=human
   

   • --pretty="..." → Customizes the format of the log output.

     • %C(yellow) → Changes Hash: text color to yellow.

     • %h → Displays the short commit hash.

     • %ad → Displays a date of commit.

     • %s → Shows the commit message.

     • %an → Shows who committed the changes.

   • --date=human → Formats the date in a human-readable way (e.g., "2 days ago" instead of a timestamp).

     

     💡

     If --pretty="" contains a custom format string (using %h, %s, %C for colors, etc.), Git treats it as format: automatically. Therefore, you don’t need to explicitly write format: when using a custom format.

8. To show commit stats with the pretty option, run:

    git log --pretty=format:"%Cred %h - %Cgreen %an: %C(yellow) %s" --stat
   

   

9. To create an alias called lg , we need to change the configuration file (config), on global table (--global) by adding the following entry:

    git config --global alias.lg 'log --color --graph --pretty="%C(red) %h%Creset - %C(yellow) %s %C(green) (%cr) %C(blue) <%an>"'
   

   

10. To check the alias definition, run the command:

    git config --global --get alias.lg
    

    

Querying through git log

So far we used ALL information from git log. It is possible to narrow the search by querying though the git log. We can query through git log using various filters and options to find specific commits based on author, date, message, file changes, branches, and more.

1. To show all commits done by 2 people, we can use regex:

    git log --author="John Doe\|Alice"
   

   

2. To search for commit by date, run:

    git log --since="2024-01-01" --until="2024-02-01"
   

   This will show commits made between Jan 1, 2024, and Feb 1, 2024

3. To search for commits by message (keyword search) and make the search case-insensitive, run:

    git log --grep="Feature" --grep="bug fix" --regexp-ignore-case
   

   This will find commits where the message contains both "bug fix", ‘'“BUG FIX“, “Bug fix“, “FEATURE“, “Feature“, "feature".

   

4. To search for commits that modified several files, run:

    git log -- testfile-01 branchfile-01
   

   This will show commits affecting testfile-01 and branchfile-01.

   

   💡

   Using the -- we inform git that we have files in mind, not branches.

5. To show the last 5 commits, run:

    git log -5
   

   

6. To list only merge commits, run:

    git log --merges
    git log --no-merges
   

   • --no-merges - excludes merge commits from the output, run:

7. To search for commits from a specific branch, run:

    git log main
    git log master..second-branch
    git log second-branch..master
   

   

   git log master..second-branch compares the commit history of both branches and shows commits that exist in second-branch but are missing in master. Once the branch second-branch is merged into master, all its commits become part of master’s history. Therefore, Git finds nothing unique in second-branch and the output returns nothing

   git log second-branch..master shows commits in master that second-branch doesn’t have.

8. To get a precise result, we can combine multiple filters:

    git log --author="John Doe" --grep="bug-fix" --since="10 weeks ago"
   

References

1. git log documentation

This article will teach us how to:

• Install Ubuntu Server 24.04 on a laptop from scratch

• Install Ubuntu server by using Vagrant tool with ISO image

Prerequisites:

1. A laptop without any OS installed;

2. A virtualization software Virtualbox that Vagrant uses to create virtual machines.

3. Vagrant which automates the setup and provisioning of virtual machines.

Installing Ubuntu Server on Laptop

Installing Ubuntu Server on a laptop is a straightforward process, though it involves several steps:

1. Go to the Ubuntu Server Download Page and click the download link to get the ISO file. In this hands-on, Ubuntu Server 24.04.1 LTS version was downloaded and used as a base image.

2. Download a balenaEtcher, a USB creation tool that allows you to create bootable media.

   

3. Now, when balenaEtcher is downloaded as a .zip file, extract it with the unzip command:

   

   After extraction, you will find an executable file with .AppImage extension.

4. Insert your USB drive with at least 4 GB of space into your computer.

5. Run the balenaEtcher, select the downloaded Ubuntu Server ISO, choose the USB drive, and start the flashing process:

   

   💡

   This process will erase all data on the USB drive, so make sure to back up any important files.

6. Insert the bootable USB drive into your laptop, restart your laptop, and enter the boot menu (usually by pressing F12 on the Dell laptop).

   💡

   If you have any important data on your laptop, back it up as the installation process may erase the entire disk.

7. Select the USB drive from the boot options and press Enter.

   

8. Choose the language you want to use for the installation process.

9. Select your preferred keyboard layout.

10. Choose the type of installation.

11. Configure Network:

    

    If your network is not found among other networks, you can continue without a network and later configure the network at your own pace.

12. If your system requires a proxy to connect to the Internet, configure your Proxy.

13. If you use an alternative mirror for Ubuntu, enter the Mirror address details.

14. Set Up the Storage:

    

    💡

    In this hands-on, an entire disk was chosen for the Ubuntu Server OS, however choosing the entire disk is not recommended since it will erase all data on the disk, making it unsuitable if you need to preserve existing data or partitions.

15. Create a user account by entering your name, the server name, and a username and password for the system.

    

16. If you plan to access your server remotely, choose to install the OpenSSH server.

    

17. Confirm the installation details and start the process. The installer will copy the files and configure the system. This may take some time.

18. Once the installation is complete, remove the installation media and press Enter to reboot.

    

19. Log in with the username and password you created during the installation.

    Your Ubuntu Server installation is now complete. You can start using it as a server or install additional services as needed.

Install Ubuntu Server via Vagrant

To install Ubuntu Server manually using a Vagrant machine with a downloaded ISO file, follow these steps:

1. Edit the Vagrantfile to configure the virtual machine and specify that it should boot from the ISO file:

     Vagrant.configure("2") do |config|
      config.vm.box = "bento/ubuntu-20.04"
      config.vm.provider "virtualbox" do |vb|  
          vb.gui = true 
          # Ensure the boot order prioritizes booting from the virtual DVD drive
          vb.customize ["modifyvm", :id, "--boot1", "dvd", "--boot2", "disk"]
          # Attach the ISO file to the VM as a virtual DVD drive to the existing SATA controller
          vb.customize ["storageattach", :id, "--storagectl", "SATA Controller", "--port", "0", "--device", "0", "--type", "dvddrive", "--medium", "path/to/your/ubuntu-24.04.1-live-server-amd64.iso"]
   
          # Create and attach a virtual hard disk to the SATA controller for installation
         vb.customize ["createhd", "--filename", "ubuntu-server-disk.vdi", "--size", 20000]  # creates a 20 GB virtual hard disk that the Ubuntu Server can be installed on.
         vb.customize ["storageattach", :id, "--storagectl", "SATA Controller", "--port", "1", "--device", "0", "--type", "hdd", "--medium", "ubuntu-server-disk.vdi"] #the new virtual hard disk is attached to the SATA controller
   
          # Optionally increase VM resources (like memory or CPU)
          vb.memory = "4096"
          vb.cpus = 2
      end
      # Configure a bridged network
      config.vm.network "public_network", bridge: "wlo1: Wi-Fi (Home)"
      #This configuration forwards requests made to port 2222 on your host machine to port 22 on the Vagrant guest machine
      config.vm.network "forwarded_port", guest: 22, host: 2222, auto_correct: true
    end
   

   Replace "path/to/your/ubuntu-server.iso" with the actual path to the Ubuntu Server ISO file on your system.

2. Once your Vagrantfile is ready, you can boot the machine:

    vagrant up
   

   This will start the virtual machine and boot from the ISO file to install Ubuntu Server on the virtual hard disk.

3. Since the ISO is loaded, the Ubuntu Server installation screen will appear. You should now be able to install Ubuntu Server as if you were on physical hardware. Proceed with the standard Ubuntu Server installation steps within the VM.

4. After installation is complete, you may see an error “Failed unmounting crdom.mount /cdrom“, which means that the installation medium (ISO image) is still mounted in the virtual machine's virtual CD-ROM drive. Power off the VM to make sure that the ISO file is detached from the virtual machine so that the system can reboot from the newly installed Ubuntu Server operating system on the virtual hard disk.

5. Now edit the Vagrantfile to change the boot order to boot directly from the virtual hard disk (where the OS was already installed).

    # Ensure the boot order prioritizes booting from the virtual DVD drive
    vb.customize ["modifyvm", :id, "--boot1", "dvd", "--boot2", "disk"]
   

   💡

   The default boot order typically prioritizes the hard disk once an operating system is installed.

6. To access your Ubuntu Server installed via VirtualBox with vagrant ssh, you need to ensure that the VM is configured to allow SSH access. Power on the server through VirtualBox again and install SSH to be able to interact with the Ubuntu Server directly via the terminal:

    sudo apt install openssh-server 
    sudo systemctl enable ssh #to start the service at boot
    sudo systemctl start ssh
    sudo systemctl status ssh
   

   

7. Now that SSH is running on your Ubuntu Server, we need to update the Vagrantfile to handle SSH connections correctly:

    Vagrant.configure("2") do |config|
      # Use the existing box or manually installed Ubuntu Server
      config.vm.box = "bento/ubuntu-20.04"
      config.vm.provider "virtualbox" do |vb|
        # Ensure the boot order prioritizes the hard disk
        vb.customize ["modifyvm", :id, "--boot1", "disk", "--boot2", "dvd"]
        # Optionally increase VM resources
        vb.memory = 4096
        vb.cpus = 2
      end
      # SSH Configurations
      #config.vm.network "public_network", bridge: "wlo1: Wi-Fi (Home)"  # Adjust to your network interface
      config.vm.network "forwarded_port", guest: 22, host: 2222, auto_correct: true  # Forward SSH port
      # Use vagrant's default insecure key (no need for password)
      config.ssh.username = "vagrant"  # Ensure this matches the username created during Ubuntu installation
      config.ssh.private_key_path = "~/.vagrant.d/insecure_private_key" # Ensure key matches
    end
   

8. If Ubuntu Server was installed manually and a vagrant user was not created, create one now. Vagrant expects to use a user vagrant with passwordless SSH access.

    sudo adduser vagrant
    sudo usermod -aG sudo vagrant
   

   

9. Add Vagrant’s Public Key: The default Vagrant SSH key needs to be added to the vagrant user’s authorized_keys. Copy Vagrant's default insecure public key to the vagrant user’s SSH directory to allow passwordless SSH access.

    sudo mkdir /home/vagrant/.ssh
    sudo curl -Lo /home/vagrant/.ssh/authorized_keys https://raw.githubusercontent.com/hashicorp/vagrant/master/keys/vagrant.pub
    sudo chown -R vagrant:vagrant /home/vagrant/.ssh
    sudo chmod 700 /home/vagrant/.ssh
    sudo chmod 600 /home/vagrant/.ssh/authorized_keys
   

   

   • Adding the Vagrant public key ensures Vagrant can authenticate using the default key and configuring authorized_keys and permissions resolves any SSH authentication issues.

10. Now that the Vagrantfile was updated and the vagrant user was set up, reload the Vagrant configuration:

    vagrant reload
    

    💡

    If you encounter “Warning. Authentication failure. Retrying…“ you can ignore it by pressing Ctrl+C

11. At this point, the VM should be up and running. On the host machine, run the following:

    ssh -i ~/.vagrant.d/insecure_private_key -p 2222 vagrant@localhost
    

    If you can successfully SSH into the server, it confirms that the SSH configuration is correct. Enter the password of your Ubuntu Server if needed.

12. You can also run the following:

    vagrant ssh
    

References

1. How to Deploy Ubuntu Server 24.04 LTS: Step-by-Step Installation Overview

2. Create An Ubuntu 20.04 Server Using Vagrant

3. How to Install, Deploy, and Uninstall Vagrant on Ubuntu 22.04

/etc/passwd/ explained

The /etc/passwd file stores basic information about user and system accounts. It is a text file that resides in the /etc directory. /etc/passwd has general read permission on all systems because it does not include hashed passwords and many command utilities use it to map user IDs to user names.

Each line within this file corresponds to the user account and each entry is split into columns, separated by a colon (:).

Image credits: Cyberciti.biz

1. Type the following command to look at the content of the passwd file:

    cat /etc/passwd | tail -5
   

   

Explanation of Fields

1. username: The user’s login name.

2. x: A placeholder indicating that the password is stored in /etc/shadow.

3. UID: The user's unique numerical ID. To the Linux system, each time when we reference user jdoe, we are actually referencing UID 1002. When a user is created, the system by default automatically assigns the next available UID.

4. GID: The unique numerical ID of the primary group that the user belongs to. Creating groups works in a similar way to creating users, in the sense that the group is assigned the next available GID.

5. gecos-field: The comment field. It allows you to add extra information about the users such as full name, phone number etc.

6. home_directory: Path to the user’s home directory.

7. shell: Default shell for the user (e.g., /bin/bash).

/etc/shadow/ explained

The /etc/shadow file stores password-related information securely. Unlike the /etc/passwd file, /etc/shadow does not have general read permission.

Since the /etc/shadow file contains sensitive hashed password information only the root user has full control over the file. Members of the shadow group (e.g., system processes or utilities that require access) can read it but not modify it while others (regular users) cannot access the file at all.

Type the following command to look at the content of the shadow file:

sudo cat /etc/shadow | tail -4

Each line within this file corresponds to the user and contains password hashes and other security information.

Explanation of Fields

1. username: The login name.

2. password: The hashed password. In practice placing !/* signs is one way to lock out an account. The restriction is that we cannot directly log in as root account from the shell or over the network. We have to log in to the system as a normal user account first. Usually, the password format is set to $id$salt$hashed, The $id is the algorithm prefix used On GNU/Linux as follows:

   1. $1$ is MD5

   2. $2a$ is Blowfish

   3. $2y$ is Blowfish

   4. $5$ is SHA-256

   5. $6$ is SHA-512

   6. $y$ is yescrypt

3. lastchanged: Days since Unix Epoch (January 1, 1970) when the password was last changed. 0 means the password must be changed immediately upon the user's next login. Empty field means the system treats it as no password change tracking, potentially indicating a misconfigured or unused account.

4. min: Minimum days between password changes. 0 means the password can be changed anytime.

5. max: Maximum days a password is valid. After that, the user is forced to change her password again.

6. warn: Days before expiration to warn the user.

7. inactive: Days after expiration before the account is disabled.

8. expire: Days since Unix Epoch when the account will be disabled.

Unfortunately, this file is neither readable nor writable to regular Linux users. However, a small set of commands allows modifying /etc/shadow files for a standard user account. For example, the passwd command enables users to change their passwords.

References

1. Understanding /etc/passwd File Format

2. Understanding /etc/shadow file format on Linux

What is Link?

Before starting to work with hard and soft links, let's first understand what a link is in Linux.

Every storage device contains files and directories in a collection of blocks.

Information about a file or directory is stored in a data structure called inode. Inodes contain metadata such as who is the owner of the file, when it was created and when it was last accessed, its size, permissions, ownership, and pointers to the data blocks where the file's contents are actually stored etc.

Every storage medium, such as hard disks, flash drives, etc., has its own inodes. These inodes are specific to the filesystem, storage device, and partition. So if we move one text file from the local storage device to the flash drive, it will get a different inode.

Every inode has an inode number that is unique and used by the filesystem to locate the inode.

A directory entry is essentially a table maintained by the filesystem that contains a name for a file or directory and a pointer (reference, often an address) to the inode number.

A link is simply an additional directory entry for a file or directory. It is between the filename and the actual data stored on the filesystem. By creating links we can have multiple directory entries (names) for the same file or directory, allowing two or more names for the same thing.

Hard Links

A hard link is a directory entry that points directly to the inode of a file. It acts as a separate file and refers to the exact spot on a hard drive. It is a mirror copy of the original file. Let's look at how to create hard links and find out how they work. To create the hard link, use ln command.

Creating a hard link:

1. Create an original file:

    echo "Hello World!" > original.txt
   

   

2. Create a hard link:

    ln original.txt hardlink.txt
   

   

3. Verify the inode numbers of the original and hard link files:

    ls -i original.txt hardlink.txt
   

   

As we can see, both original.txt and hardlink.txt share the same inode number, indicating they point to the same file content.

Editing the hard link

4. Make changes to the hard link hardlink.txt:

    echo "Nice to see you World" >> hardlink.txt
   

5. Check the content of the original file original.txt

    cat original.txt
   

   

As we can see, changes made to hardlink.txt are reflected in original.txt because they refer to the same inode.

Deleting the original file

6. Delete the original file:

    rm original.txt
   

7. Check the content of the hard link hardlink.txt:

    cat hardlink.txt
   

   

As we can see, the file content is still accessible through hardlink.txt since the inode and data blocks are preserved until all hard links are removed.

Creating multiple hard links

8. Create another hard link:

    ln hardlink.txt another-hardlink.txt
   

9. Verify Inode numbers:

    ls -li hardlink.txt another-hardlink.txt
   

   

Again, both hardlink.txt and another-hardlink.txt share the same inode number, indicating they point to the same file content.

Hard Link Advantages and Disadvantages

Hard links have the following advantages:

1. Efficient Storage: Hard links share the same inode and data blocks, which means no additional disk space is used for multiple hard links to the same file.

2. File Availability: As long as at least one hard link exists, the file's data remains accessible. Deleting one hard link does not delete the file.

3. Performance: Hard links are typically faster than symbolic links because they point directly to the inode without the need for path resolution.

4. Transparency: Hard links appear and behave like regular files. There is no distinction between the original file and its hard links.

5. Consistency: All hard links to a file are always up-to-date. Any changes made to the file through any hard link are immediately reflected in all other hard links.

While useful, there are some limitations to what hard links can do.

1. Restrictions on Directories: Hard links can only be created for regular files (not directories or special files).

2. Single Filesystem Limitation: Hard links cannot span multiple filesystems. They only work when the new hard link exists on the same filesystem as the original.

Soft Links

A soft link (symbolic link) is a separate file that contains the path to another file or directory. Unlike hard links, soft links do not reference the inode of the target file directly but instead have their own inodes and contain the path to the target file or directory.

Creating a Soft Link

1. Create an original file:

    echo "Hello World" > original.txt
   

2. Create a soft link using the command ln -s:

    ln -s original.txt softlink.txt
   

   💡

   An absolute path should be specified when creating symbolic links since relative paths will not work.

3. Verify the soft link by typing:

    ls -l original.txt softlink.txt
   

   

4. Read the content of the softlink.txt file:

    cat softlink.txt
   

   

Editing the Soft Link

5. Edit the soft link softlink.txt:

    echo "Nice to see you World!" >> softlink.txt
   

6. Check the content of the original.txt file:

    cat original.txt
   

   

As we can notice, changes made through softlink.txt are reflected in original.txt.

Moving the target file

7. Move (or delete) the original.txt file to another location in the system:

    mv original.txt ./backup/
   

8. Again, check the content of the softlink.txt file:

    cat softlink.txt
   

   

As we can see, the soft link becomes a "dangling" link, as its target is moved to another location (or no longer exists in case it is deleted).

Linking to Directories

Because the soft link connection is a logical connection, not a duplication, soft links can point to entire directories or links to files on remote computers. Hard links cannot do this. Let's find out how it works.

9. Create a folder and a file:

    mkdir my-folder
    echo "Learn how to create soft links to folders" > my-folder/file
   

10. Create a soft link named soft-folder to the directory my-folder:

    ln -s my-folder soft-folder
    

11. Verify the link:

    ls -l
    

12. Access the directory my-folder using soft link soft-folder:

    ls soft-folder
    

13. Delete the original folder my-folder and list the soft-folder:

    rm -r my-folder
    ls soft-folder
    

    

As we can see, since the original folder was removed, listing soft-folder shows nothing in the output. The target was deleted and the link became useless.

Soft Link Advantages and Disadvantages

Soft links have the following advantages:

1. Flexibility: Soft links can link to both files and directories and span across different filesystems or partitions.

2. Convenience: It is easy to create shortcuts and organize files with soft links. They can link to non-existent files, which will work once the target is created.

3. Low Overhead: The size of soft link files is small since they only store the path to the target.

While useful, there are some drawbacks of soft links.

1. Fragility: They can become "dangling" links if the target is moved, renamed, or deleted.

2. Performance: They have slightly slower access than hard links because they require an extra path resolution step.

3. Security: They can potentially introduce security risks if not managed properly, as they can point to sensitive files.

4. Different Inodes: Soft links have their own inodes, which can add overhead to inode usage.

References

1. Linux Crash Course - Symbolic Links

2. Linux Hard Links versus Soft Links Explained

3. Create and change hard and symbolic links

4. Hard links and soft links in Linux explained

Type:

hostname

To see where it is stored, type:

cat /etc/hostname

To get the hostname and additional information about your machine:

hostnamectl

To change the hostname, login as root user or use the sudo command:

sudo hostnamectl set-hostname my-private-workstation

If you do not want to restart your terminal, you can force the current terminal to be changed by typing (assuming you use bash):

exec bash

Verify the hostname was changed:

cat /etc/hostname

Check the content of the /cat/hosts file

cat /etc/hosts

Even though the hostnamectl command allows us to set the new hostname, it does not update the /etc/hosts file. This is optional, but highly recommended.

To update the /etc/hosts file, use the vim command:

sudo vim /etc/hosts/

To ping this instance, run:

ping my-private-laptop

References:

1. Linux Crash Course - Setting the Hostname of your Linux Workstation or Server

2. How to Change Hostname on Ubuntu

3. How to configure a hostname on a Linux system

Viewing Permissions

There are two ways of viewing file permissions in Linux:

1. Using a graphical user interface (GUI) by right-clicking the file/directory -> Properties -> Permissions

   

2. Using a command-line interface (CLI) by typing ls -l command.

   

   The output of this command is explained further.

Permission Groups

Each file and directory in Linux can be accessed or modified by three different classes of users.

1. User (u): The user who owns the file.

2. Group : The group that the file belongs to.

3. Others: All other users.

/Permissions Types

Each file or directory has three basic permission types:

1. Read (r) - allows to read the contents of the file.

2. Write (w) - allows to write or modify a file or directory.

3. Execute (x) - allows to execute a file or view the contents of a directory.

How to Read File Permissions

The permissions are represented as a string of 10 characters, like -rw-rw-r--.

These characters can be broken down as follows:

• The first character indicates the type of file

  • - for regular files,

  • d for directories,

  • l for symbolic links,

• The next three characters indicate the permissions for the user/owner of the file/directory (rw- read, write).

• The following three characters indicate the permissions for the group the file/directory belongs to (rw- read, write).

• The last three characters indicate the permissions for others (r-- read).

So, the file above weather-report.service is owned by the user vagrant and belongs to the group vagrant . Its owner vagrant and all group members vagrant can read and write to the file while all others can only read the file.

Changing Permissions

Changing permissions in Linux can be done using the chmod command in 2 ways:

1. Symbolic mode;

2. Numeric (Octal) Mode:

Symbolic Mode

Symbolic mode allows to change file and directory permissions using symbolic representations of the

• User classes

  • User (u),

  • Group (g),

  • Others (o)

  • All (user, group, and others) (a)

• Permissions types

  • Read (r),

  • Write (w),

  • Execute (x).

• Operators

  • +: Adds the specified permissions to the existing ones without altering other permissions.

  • -: Removes the specified permissions.

  • =: Sets the specified permissions and removes any permissions not listed.

To add read, write, and execute permissions for the user:

chmod u+rwx file

To set read and write permissions for the group:

chmod g=rw file

To remove execute permission for others:

chmod o-x file

Numeric Mode

When chmod command is used with numerical values three digits are specified. The first digit is for the user, the second is for the group and the third is for others. Each digit represents a different set of permissions. The digits are calculated by adding up the values of:

• r (read) = 4,

• w (write) = 2,

• x (execute) = 1,

• no permission = 0

The numbers are summed up and depicted by one number. Therefore, the possibilities are:

• 7 - for read, write, and execute permission.

• 6 - for read and write privileges.

• 5 - for read and execute privileges.

• 4 - for read privileges.

To give the owner read and write permissions, the group read permissions, and no rights for all others, type.

chmod 640 filename

To change the permission for every file all in one shot:

chmod 640 Download/*

Changing Ownership

Aside from changing file and directory permissions, we can also change user ownership and group ownership of the file and directory. Both of these tasks require superuser privileges.

Changing the Owner

To change the owner of a file or directory:

sudo chown username /Downloads/

Changing the Owner and Group

To change both the owner and the group:

sudo chown username:groupname filename

Changing Only the Group

To change only the group:

sudo chown :groupname filename

Special Permissions

In addition to the standard read, write, and execute permissions, Linux also supports special permissions that provide additional security and functionality features for files and directories.

1. Setuid (Set User ID) - When set on an executable file, the file runs with the permissions of the file's owner, not the person running it.

   

   In order setuid to work, the file should be executable and owned by root in order to grant temporary elevated privileges to the person running it.

2. Setgid (Set Group ID) - Similar to setuid, the setgid permission allows a user to execute a file with the permissions of the file's group, rather than the permissions of the user executing the file. setgid is often used for directories to ensure that files created within the directory inherit the group ownership of the directory.

3. Sticky Bit: When the sticky bit is set on a directory, it restricts the deletion or renaming of files within that directory to only the file owner, the directory owner, or the root user, even if other users have write permissions on the directory. This is commonly used on directories like /tmp, where many users have write access, to prevent users from deleting or renaming each other's files. The sticky bit is more useful in shared directories where multiple users have write access.

   

References:

1. Linux Crash Course - Understanding File & Directory Permissions

2. Linux file permissions explained

3. Classic SysAdmin: Understanding Linux File Permissions

4. Linux Permissions Explained

find [path] [expression]

• [path]: The directory where the search begins. If omitted, it defaults to the current directory (.).

• [expression]: Criteria for searching (e.g., name, type, size, etc.).

Hands-on Exercise Overview

This hands-on exercise shows how to use find command and its different options to search for files and directories on Ubuntu 22.04. It focuses on basic file searching, advanced searching techniques, and executing commands on found files.

Hands-on Exercise

1. To search for a file with a specific name, use the -name option:

    sudo find / -name "*fish*"
   

   

   This command searches the entire filesystem (/) for files and directories whose names contain the substring "fish".

   sudo provides root access, ensuring no permission issues, and produces a clean output stream of matching file names. head receives a continuous and uninterrupted stream of matching filenames correctly displaying the first 5 lines from this stream.

   💡

   The search with -name option is case-sensitive. To make the search case-insensitive, use the -iname option instead.

2. To search for files of a specific type, use the -type option:

    sudo find /snap name "cat" -type f
   

   

   To search for both files and directories simultaneously, use -o operator which stands for OR operator:

    sudo find / -name "vagrant" \( -type f -o -type d\) -exec ls -ld {} \;
   

   

3. To find files of a specific size, use the -size option:

    sudo find /var/log -size 100c
    sudo find /var/log -size -1K
    sudo find /var/log -size +100M
    sudo find /var/log -size 2G
   

   

4. To search for files based on their modification, access, and change times use -mtime, -atime, and -ctime options:

    find /path/to/search -mtime +n
    find /path/to/search -atime -n
    find /path/to/search -ctime n
   

   • -mtime - Modification time, when the file content was last modified;

   • -atime - Access time, when the file was last accessed;

   • -ctime - Change time, when the file's metadata or content was last changed;

   • +n - more than n days ago;

   • -n - in the last n days;

   • n - exactly n days ago;

5. To find all .conf files in /etc and copy them to ~/backup/etc, use a new option -exec which executes a command against all files found in the result.

    sudo find /etc -type f -name "*.conf" -exec cp {} ~/backup/etc/ \;
   

   

• -exec executes the cp command against every item in the result;

• {} - a placeholder for the actual item itself. So every time a find command finds something, it puts the result here;

• \; terminates the find command. You can also use + instead of \; sign as a terminator. You should terminate the find command anytime you use the -exec option.

6. To change the permissions of several files to be readable and writable simultaneously, type:

    find ~ -type f -name "*.txt" -exec chmod 600 {} \;
    find ~ -name "*.txt" -exec ls -ld {} +
   

   

Best Practices

1. Use Quotes for Patterns: Always use quotes around patterns to avoid shell interpretation.

2. Combine Expressions: Use multiple expressions to narrow down your search.

3. Test Before Executing: Use -print to test your find command before adding -exec.

References:

1. How to use FIND in Linux

2. find(1) — Linux manual page

3. 10 ways to use the Linux find command

4. Linux Crash Course - The find command-

How fstab is Structured

The table itself is a 6-column structure, where each column designates a specific parameter and must be set up in the correct order. The fields of the table are as follows from left to right:

1. Block device - the device that should be mounted. The most typical way to reference a block device is by using its location /dev/sda1 or by using its LABEL or UUID (Universal Unique IDentifier). The latter is the preferred method since it guarantees to univocally reference a filesystem. On GPT partitioned disks it’s also possible to reference a filesystem by using PARTUUID or PARTLABEL.

2. Mount point - specifies the mountpoint for the filesystem. It is a directory in the system that should be used to access its content. This should always be provided except if the block device is used as a swap. In that case "none" should be used.

3. Filesystem type - the type of filesystem in use on the raw block device or partition. Linux supports many filesystem types: ext4, xfs, btrfs, f2fs, vfat, ntfs, hfsplus, tmpfs, sysfs, proc, iso9660, udf, squashfs, nfs, cifs, and many more.

4. Mount options - a list of options when mounting the filesystem. To use the default set of mount options we specify defaults as a value. The kernel default is usually rw, suid, dev, exec, auto, nouser, async.

   • suid - respect SETUID and SETGID bits;

   • exec - allow executing binaries and scripts;

   • noauto - do not mount when mount -a is given (e.g., at boot time);

   • user - allow a user to mount;

   • nouser - make the filesystem not mountable by a standard user;

   • async - perform I/O operations on the filesystem asynchronously;

   • owner - allow a device owner to mount;

   • nofail - do not report errors for this device if it does not exist.

5. Dump - enable (1) or disable (0) the backing up of the device/partition. The value is used by the dump backup program (if installed) to know what filesystem should be dumped. Usually, this field is set to 0, which disables it.

6. Pass - establishes the order by which another utility, fsck should check filesystems on boot. 0 means that fsck will not check the filesystem. Numbers higher than this represent the check order. The root filesystem should be set to 1 and other partitions set to 2.

An example of fstab table on a Vagrant Ubuntu virtual machine.

Each filesystem is described on a separate line. Lines starting with '#' are comments. Blank lines are ignored. Fields on each line are separated by tabs or spaces.

Hands-on Exercise Overview

The main purpose of this hands-on exercise is to gain practical experience in configuring filesystem mounts using the /etc/fstab file. This hands-on shows how to add a new entry to the /etc/fstab file to mount a filesystem and tests the configuration by mounting and unmounting the filesystem.

Hands-on Exercise

1. Run the command that allows us to find out whether there are storage volumes that are not mounted:

    lsblk
   

   

2. To get a listing of mounted devices and find out whether our disk device is mounted or not, run:

    mount | grep sdc
   

   

   💡

   Choose /mnt directory for permanent storage to mount the disk.

3. To add a new entry to the end of the /etc/fstab file:

    sudo vim /etc/fstab
    /dev/sdc /mnt/mydisk ext4 defaults 0 0
   

   

4. To create the directory for the storage volume to be mounted:

    sudo mkdir /mnt/mydisk
   

   💡

   Without this directory exists, the fstab file would not be able to find it and it will cause the problem.

5. To mount the device manually, run:

    sudo mount /mnt/mydisk
   

   

   This command assumes that the details of what to mount (like the device or remote filesystem) are already specified in the /etc/fstab file. The target directory /mnt/mydisk was found in the fstab file and the system knows that this directory is associated with the device /dev/sdc. Thus, the mount command was simplified.

6. To unmount the device, run:

    sudo unmount /mnt/mydisk
   

   

7. To ensure the new entry mounts correctly by simulating the boot-time mounting process, run:

    sudo mount -a
   

   

   This command automatically mounts all filesystems specified in the /etc/fstab file without requiring further human interaction. By running this command, you instruct the system to read /etc/fstab and mount all listed filesystems according to the defined parameters, making it useful for ensuring all necessary filesystems are mounted, especially after a system reboot.

   💡

   The option noauto in the mount options field of the fstab file does not mount when mount -a is given (e.g., at boot time). There are several reasons to use this option. 1. It allows you to control when these filesystems are mounted. 2. It simplifies the mount command since there is no need for the device name, and only the target directory can be referred to.

8. To find out the UUID of the storage volume, run:

    sudo blkid
   

   

   It is recommended to use the UUID (Universally Unique Identifier) of a storage volume instead of the device path (like /dev/sdc) in the /etc/fstab file, because UUIDs are unique and remain constant while device paths can change depending on the order in which devices are detected by the system.

References:

1. Linux Crash Course - The /etc/fstab file

2. fstab

3. fstab(5) — Linux manual page

4. An introduction to the Linux /etc/fstab file

5. How fstab works – introduction to the /etc/fstab file on Linux

Systemd timers are a part of this systmed suite and they enable us to schedule tasks to be executed at specified times or intervals, similar to cron jobs but with more features and tight integration with the rest of the systemd ecosystem.

Systemd Timers or Cron Jobs

1. Cron is more of a stand-alone thing whereas systemd timers are part of systemd. Therefore systemd timers can be used only on distributions with systemd systems while cron jobs can be used on distributions either with systemd systems or non-systemd systems.

2. Systemd timers have additional features such as support for complex calendar-based events, built-in support for catching up on missed jobs, a built-in mechanism for dependency management or condition checks before job execution, and integration with systemd’s journal which provides centralized and unified logging etc.

3. Additional features of systemd timers come with added complexity. Therefore, systemd timers are a bit more involved meaning more to learn while cron jobs are much simpler to understand and implement.

Which one to use depends on the complexity of the task and which of the two solutions is a better fit for your project on a case-by-case basis.

Components of Systemd Timers

To create a systemd timer two files are required: a .timer file to schedule the task and a .service file for the task.

• Timer units are the actual timer definitions. They define when and how often a task should be executed. The timer units have a .timer extension and are typically stored in the /etc/systemd/system directory. An example of weather-report.timer unit file is given below.

  

• Service units are the tasks that are executed when the timer is triggered. Service units have a .service extension and are typically stored in the /etc/systemd/system directory. An example of weather-report.service unit file is provided below.

  

When you have a .service file with the same name as the .timer file in the directory, with the only difference being the file extension, systemd automatically considers them to be related. Usually, for each timer, there is a corresponding service file that gets executed when the timer triggers.

Types of Timers

• Real-time timers are based on the actual real-world clock time. They can be set to trigger at specific times, such as "every day at 2:00 AM".

• Monotonic timers are based on a timer relative to a certain point, such as "15 minutes after boot".

Hands-on Exercise Overview

Understanding systemd timers involve knowing how to create and manage them. This hands-on exercise shows how to create a systemd timer that will run the wall command automatically on a specific schedule.

Hands-on Exercise

1. Create a new weather-report.service unit file:

    sudo vim /etc/systemd/system/weather-report.service
   

2. Copy and paste the following code into the file and save it:

    [Unit]
    Description=Weather Report Service
    After=network-online.target
    Wants=network-online.target
   
    [Service]
    ExecStart=/bin/sh -c 'sudo wall $(curl wttr.in?format=3)'
   
    [Install]
    WantedBy=default.target
   

   This simple file executes the wall command, which stands for "write to all". The wall command sends a message to all users currently logged into the system and the message is supposed to be the output of curl wttr.in?format=3, which queries the wttr.in service for weather information in a brief format. This can be particularly useful for system administrators who need to notify users about system maintenance, reboots, or other important events.

3. Refresh systemd so that it’s aware of the fact we added a new service file:

    sudo systemctl daemon-reload
   

4. Now, when we want to send a weather report to every user on the system, all we need to do is start the systemd service for it:

    sudo systemctl start weather-report.service
   

   

5. Create a new weather-report.timer unit file the type of which will be a timer:

    sudo vim /etc/systemd/system/weather-report.timer
   

6. Copy and paste the following code into the file and save it:

    [Unit]
    Description=Weather Report Service
   
    [Timer]
    OnCalendar=Mon..Fri 08:32
    Persistent=true
   
    [Install]
    WantedBy=timers.target
   

   Under the timer section, we have OnCalendar here. We’re giving it a range starting with Monday and ending with Friday. Also, we’re setting it for 08:32 am, which means the timer will trigger weather-report.service file every weekday at 08:32 am.

   If the server is down for any reason the timer won’t trigger. Enabling the Persistent=true option, allows the task to run once the server starts up next in case the OnCalendar time is missed.

   💡

   The timer will look for a file name that’s the same as its own but with a .service extension instead of a .timer extension. Therefore we don’t tell the timer which service file to start, it already knows what it is looking for. All we have to do is match the naming scheme.

   In the install section, targets are another type of unit file, and when you link them like this, the current file will inherit specific characteristics from the target.

7. Refresh systemd once more so that it knows we added a new timer unit file:

    sudo systemctl daemon-reload
   

8. Finally, start and enable the timer with one single command:

    sudo systemctl enable --now weather-report.timer
   

9. Check the status of your timer:

    systemctl status weather-report.timer
   

   

References:

1. Automate Your Tasks with systemd Timers: A Step-by-Step Guide

2. How to use systemd timers (cronjob alternative)

3. Understanding Systemd Services

4. systemd.time format

5. Systemd timers onCalendar (cron) format explained

systemd is a newer init system with a more modern approach to process management. It can enable the administrator to schedule tasks to run when specific criteria are met and also to ensure that a process is always running. It allows you to configure logging, how DNS is resolved, and much more.

Units in systemd are resources that systemd can manage for you. These include services, timers, mounts, automounts, and more. systemd runs in the background and is completely self-sufficient.

How systemd is Structured

systemd looks at unit files to understand how to interact with the processes. Service files are just text files that contain instructions that tell systemd how it needs to manage that particular service. Service files can be found in any of several directories and they will have a file extension of .service . Service unit files are typically located in:

When the Linux system starts, it will start the systemd and systemd will go into these directories looking for unit files. If it finds any, it is going to load those into memory.

A service unit file structure is divided into 3 sections ([Unit], [Service], [Install]), each containing key-value pairs that configure the behavior of the service. Each section name must be started with a capital letter since the file is case-sensitive. For example:

• [Unit] section: Describes the unit and its relationships to other units. Common options include Description, After, Requires, and Wants.

• [Service] section: Specifies how the service should be started, the command to use, and its behavior. Options include Type, ExecStart, Restart, and User.

• [Install] section: Defines how the service should be enabled or started as part of the system startup sequence. The most common option is WantedBy.

Dependencies between services are managed with directives like After=, Before=, Wants=, and Requires=. These directives help ensure services start in the correct order and that prerequisite services are running.

As an example, let's look at the apache2.service unit file which can be found in /lib/systemd/system directory:

Editing a systemd service unit file allows you to customize how a service starts, behaves, or stops on your system. To safely edit a service unit file, run:

sudo systemctl edit SERVICE_NAME.service

This command opens the service's override file in the default text editor. If the override file doesn’t exist, systemctl will create a new one. The override file allows you to specify only the changes or additions you want to apply to the service unit, without needing to copy and edit the entire original service file. It is stored in a directory named after the service within /etc/systemd/system/example.service.d/override.conf. This directory can contain multiple .conf files for granular overrides.

When systemd loads the service's configuration, it reads the original service file first and then applies the settings from the override file. If there are conflicting directives, the settings in the override file take precedence over those in the original service file. This merge happens in memory, leaving the original service file unchanged on disk.

To edit the entire service file, run:

sudo systemctl edit --full SERVICE_NAME.service

This command makes a copy of the entire service file and save it in /etc/systemd/system directory. This copy file will be in priority than the original service file.

Hands-on Exercise Overview

systemctl is the primary command-line utility for interacting with systemd. While it's used for managing various aspects of systemd, it's particularly focused on service management. This hands-on exercise shows how to use the six most important commands related to systemctl and systemd in general. It shows how to start, stop, restart, check the status, and enable, and disable the process using systemctl command. These are the bare minimum we should know when it comes to systemd.

Hands-on Exercise

1. Let's install a web server service for serving a web page.

    sudo apt install apache2
   

2. To check the status of the web service, run:

    systemctl status apache2.service
   

   

3. To confirm that Apache2 is serving its default page by accessing it from within the Vagrant box, run curl http://localhost in the command line. This should show the HTML content of the default Apache2 Ubuntu default page.

   

   or, enter http://localhost:8080/ on the web browser of your host machine:

   

4. To stop the web service with systemd immediately, run:

    sudo systemctl stop apache2.service
   

   

5. To start the service immediately:

    sudo systemctl start apache2.service
   

6. To restart the service, run:

    sudo systemctl restart apache2.service
   

   This command stops the service and then starts it again. This is useful for applying changes that require a complete stop and start to take effect. Restarting is more disruptive but ensures that the service runs with the latest configuration and code.

7. To reload the service, run:

    sudo systemctl reload apache2.service
   

   This command tells a service to reload its configuration files without interrupting ongoing operations. It is useful for applying non-disruptive changes. Not all services support reloading; it depends on whether the service can dynamically read new configurations without needing to stop and start again.

8. To enable a service to start at boot:

    sudo systemctl enable apache2.service
    sudo systemctl enable --now apache2.service
   

   --now option allows to start the service immediately and enable it in one shot.

   

9. To disable a service from starting at boot:

    sudo systemctl disable apache2.service
   

   

   preset: enabled means that the distribution has configured systemd such that new services will be enabled by default on boot. The vendor preset only indicates the initial recommended state. Debian and Ubuntu typically enable a newly started service for the user whereas Red Hat-based distributions such as Fedora, and CentOS disable it.

10. To list unit files that are of type service, run:

    systemctl list-unit-files --type=service
    

    

References:

1. Systemd Deep-Dive: A Complete, Easy to Understand Guide for Everyone

2. Linux Crash Course - systemd: Using the systemctl command

3. How To Use Systemctl to Manage Systemd Services and Units

• Hard Disk 1 and Hard Disk 2 are physical disks;

• "/dev/sda1", "/dev/sda2", and "/dev/sdb" are volumes or partitions;

• Any of these can be called a "drive".

In Linux, storage devices are typically represented as files in the /dev directory. Typically, files representing storage devices start with sd or hd followed by a letter.

For example, the first physical drive we attach to the system will be /dev/sda. For the subsequent physical storage drive we attach to the system, they’ll get the identifier in the form of sd{alphabet}, where the alphabet is the next letter in the line.

The kernel provides a name to every storage device plugged in or added to the computer or server on Linux.

Partitioning

Partitioning is the process of dividing the storage device into segments. To partition the disk in Linux utilities such as fdisk, gdisk, parted and the partitioning schemes such as MBR (Master Boot Record), GPT (GUID Partition Table) are used.

• MBR is over 30 years old. Because of its age, it has some serious limitations. For instance, it cannot be used for disks over 2TB, and can only have a maximum of four primary partitions.

• GPT is a more modern partitioning scheme that resolves some of the issues inherent in MBR. Systems running GPT can have many more partitions per disk. This is usually only limited by the restrictions imposed by the operating system itself. Additionally, the disk size limitation does not exist with GPT and the partition table information is available in multiple locations to guard against corruption. GPT can also write a “protective MBR” for compatibility with MBR-only tools. In most cases, GPT is the better choice unless your operating system prevents you from using it.

Each partition can be used for a different purpose, This gives a user more flexibility, allowing them to potentially segment a single disk for multiple operating systems, swap space, or specialized filesystems.

As a next step after partitioning, formatting and mounting of the storage volume should be done to prepare the disk for use.

Formatting

While the Linux kernel can recognize a raw disk, it must be formatted to be used. Formatting is the process of writing a filesystem to the disk and preparing it for file operations. A filesystem is a system that structures data and controls how information is written to and retrieved from the underlying disk. The storage device can not be used for standard filesystem operations without a filesystem. Common Linux filesystems include ext4, XFS, BTRFS, and NTFS (for Windows compatibility).

The general syntax for formatting disk partitions in Linux is:

sudo mkfs [options] [-t type ] [fs-options] device [size]

• mkfs - used to build a Linux filesystem on a device, usually a hard disk partition;

• -t, --type - Specify the type of filesystem to be built. If not specified, the default filesystem type (currently ext2) is used.

• device argument is either the device name (e.g. /dev/hda1, /dev/sdb2) or a regular file that shall contain the filesystem.

• size argument is the number of blocks to be used for the filesystem.

Mounting

Mounting is the process of attaching a formatted partition or drive to a directory within the Linux filesystem. In Linux and other Unix-like operating systems, the entire system, regardless of how many physical devices are involved, is represented by a single unified file tree. When a filesystem on a drive or partition is to be used, it must be attached to the existing tree. The drive’s contents can then be accessed from that directory.

Drives are almost always mounted on dedicated empty directories – mounting on a non-empty directory means that the directory’s usual contents will be inaccessible until the drive is unmounted. Many different mounting options can be set to alter the behavior of a mounted device. For example, the drive can be mounted in read-only (RO) mode to ensure its contents won’t be changed.

The standard form of the mount command is:

sudo mount -t type device dir

This tells the kernel to attach the filesystem found on device (which is of type type) at the directory dir. The option -t type is optional. The mount command is usually able to detect a filesystem. The root permissions are necessary to mount a filesystem by default.

Hands-on Exercise Overview

A disk partition must be formatted and mounted before use. The formatting process can also be done for several other reasons, such as changing the file system, fixing errors, or deleting all data.

This hands-on exercise shows how to format and mount disk partitions in Linux using ext4 file system.

Hands-on Exercise

1. To list all block devices attached to the local computer or server, use:

    lsblk
   

   The terminal prints out a list of all block devices which are files that represent devices such as hard drives, RAM disks, USB drives, and CD/ROM drives as well as information about them:

   • NAME – Device names

   • MAJ:MIN – Major or minor device numbers. These are the major and minor device numbers used by the Linux kernel to uniquely identify the type of device (major) and the specific instance of that device type (minor).

   • RM – Whether the device is removable (1 if yes, 0 if no)

   • SIZE – The size of the device

   • RO – Whether the device is read-only (1 if yes, 0 if no)

   • TYPE – The type of the device

   • MOUNTPOINT – Device’s mount point

• loop0 - Loop devices are a special block device that maps a normal file onto a virtual block device. This allows the file to be treated as if it were a physical disk, making it possible to mount a filesystem within a file onto the Linux file system hierarchy. Linux names these devices loop0, loop1, etc., based on their order of creation or mapping.

• sda - The name is derived from "SCSI Disk A," where "SCSI" stands for Small Computer System Interface, a set of standards for physically connecting and transferring data between computers and peripheral devices. Despite its origins, the naming convention is used broadly for devices using various interfaces, including SATA, which is very common in HDDs and SSDs. The letter a indicates that this is the first device recognized by the system. Additional partitions on this device would be listed as sda1, sda2, etc., representing different storage segments that were allocated for different purposes, such as system files, user data, or swap space. Subsequent devices would be named sdb, sdc, and so on.

• xvda - This naming convention suggests that this block device is a virtual disk, typically seen in environments using Xen or KVM virtualization, or cloud platforms like AWS EC2.

2. To display a list of all block devices containing file system information, add the -f option:

     lsblk -f
   

   

   This allows us to locate a partition for formatting and mounting. In this case, we can see that sdc partition is empty and not allocated for any specific purpose by the system or cloud provider. Therefore, it can be used for formatting and mounting.

   💡

   When working with block devices, if we target a device name without a number, we are targeting the entire disk. But if we add a number at the end of the device name, then we are targeting the partition.

3. To view what partitions are available on a storage device:

    sudo fdisk -l
   

   • fdisk - a dialog-driven program for the creation and manipulation of partition tables. It understands GPT and MBR partition tables.

   • -l, --list - lists the partition tables for the specified devices and then exit.

4. To format a disk or disk partition with the file system, use the following command:

    sudo mkfs -t ext4 /dev/sdc
   

   Make sure the correct disk is targeted. Then, verify the file system change using the command, lsblk -f

   

5. Create a new folder under the /mnt to mount the formatted storage device before using the disk:

    sudo mkdir /mnt/mountpoint
    ls -l /mnt
   

   This command creates a directory that will serve as the mount point for the new filesystem.

   

   💡

   It is recommended to use /media for temporary storage volumes and /mnt for permanent storage volumes

6. To mount the disk, use the following command and verify if the disk is mounted:

    sudo mount /dev/sdc /mnt/disk3
    df -h
   

   

7. To get a list of all mounted storage devices on the system:

    mount
    mount | grep sdc
   

8. To remove a filesystem from the directory hierarchy, unmount it with the umount command:

    sudo umount /mnt/mydisk
    lsblk -f
   

   

Once a disk is successfully formatted and mounted on the Linux system, it's ready for use! This new storage space can serve various purposes depending on your needs such as storing personal or application data, backing up storage, storing Virtual Machine (VM) disks etc;

References

1. Linux Crash Course - Formatting & Mounting Storage Volumes

2. Differences Between Drive, Partition, and Volumes in Linux

3. An Introduction to Storage Terminology and Concepts in Linux

4. How to Format Disk Partitions in Linux

How sed Works

The workflow of sed (Stream Editor) involves reading, executing commands, and displaying the output, operating in a cycle over the input stream.

Here's a step-by-step explanation of how sed works:

1. Reading: sed reads one line at a time from the input stream. This can be a file or input piped from another command.

2. Placing: Each line read into sed is placed into a space called the "pattern space," where the operations are performed. The end-of-line newline character is temporarily removed during this step.

3. Executing Commands: Once a line is in the pattern space, sed executes all the commands you've specified in the order they were given. These commands can modify the contents of the pattern space (like replacing text, deleting the line, etc.), and they can be conditioned to only run if the line matches a specified pattern. sed commands are powerful and can include:

   • Substitution (e.g., s/pattern/replacement/): Search for a pattern and replace it with the specified text.

   • Deletion (e.g., d): Delete lines that match a specified pattern or line number.

   • Insertion and Appending (e.g., i\text, a\text): Insert or append text before or after a matching pattern or line number.

   • Reading and Writing (e.g., r filename, w filename): Read content from a file into the pattern space, or write content from the pattern space to a file.

   • Line Addressing: Apply commands to specific lines or ranges of lines, either specified by line numbers or patterns.

4. Printing: After executing all commands on the line in the pattern space, sed by default prints the contents of the pattern space to the standard output. It can be redirected to a file or piped to another command for further processing.This behavior can be modified with options like -n, which suppresses automatic printing unless explicitly told to print (e.g., with the p command).

5. Cycle Continues: The pattern space is then cleared, and sed reads the next line of input, continuing the cycle until all lines from the input have been processed.

Linux sed Syntax

The main syntax for using Linux sed command is:

sed [OPTIONS]... [SCRIPT] [INPUTFILE...]

• [OPTIONS]: These are command-line options that modify how sed operates. Some of the most commonly used options include:

  • -e script: Allows you to specify a script containing one or more sed commands. This option is useful when chaining multiple commands.

  • -i: Enables in-place editing of files. When used, sed modifies the input file directly.

  • -n: Suppresses automatic printing of the pattern space. Typically used with the p command to print specific lines.

• [script]: This is where you specify the sed commands to be executed. The script usually contains one or more commands that tell sed how to process the input text.

• [input-file]: This is the source of input data for sed to process. It can be one or more filenames. If no input file is specified, sed reads from the standard input, which allows it to be used in pipelines with other commands.

Hands-on Exercise Overview

This hands-on exercise shows ten commonly used sed commands and a foxinbox.txt file with the following content was taken as a sample file:

Hands-on Exercise

1. To replace text, use the substitute command s and delimiters (in most cases, slashes - /) for separating text fields.

    sed 's/old_string/new_string/' filename.txt
   

   For example, to replace instances of box with the word bin, run:

   

   💡

   It does not matter what is used as the delimiter. Use any other delimiter instead of the forward slash ( / ) if the forward slash must be searched for.

2. To replace multiple instances of the same word within a single line, add the g flag to the command to change all of them:

    sed 's/old_string/new_string/g' filename.txt
   

   For example, to replace the word box with the word bin in the file foxinbox.txt every time, type:

   

   💡

   By default, sed only replaces the first occurrence of the specified string in each line. It searches for the first instance of the specified word in a line, replaces it, and moves on to the next line.

3. To replace a specific occurrence of the string in a line, add a number flag such as 1, 2 etc:

    sed 's/old_string/new_string/#' filename.txt
   

   For example, to substitute the second occurrence of the word box in each line of the file with the word bin, use this command:

   

4. To print out just the lines that have substitution under the given conditions, use the syntax:

    sed -n 's/old_string/new_string/p' filename.txt
   

   • -n option disables automatic printing.

   • -p instructs sed to print lines where substitution occurs.

For example, to replace the second instance of the word box in a line and print the line where the change took place:

5. To ignore case while substituting text, add the i subcommand at the end of the command:

    sed 's/old_string/new_string/i' filename.txt
   

   For example, the command for changing upper and lowercase instances of the word fox in the foxinbox.txt file is:

   

6. To substitute a string in a specific line, add the line number as a prefix to the s subcommand:

    sed '# s/old_string/new_string/' filename.txt
   

   For example, to replace the word socks with sandals only in the fourth line (4) of the text use the command:

   

7. To replace multiple instances of a string within a line range, but not the entire text, specify the range where you want sed to substitute. The syntax is:

    sed '#,# s/old_string/new_string/' filename.txt
   

   Replace the first # with the initial line number and the second # with the last line number you want to include.

   For instance, to replace the last two instances of the word socks in the file foxinbox.txt (located in the fourth and sixth line) with the word sandals, run:

   

8. To delete a line from a file with the sed command, use the d subcommand and the syntax:

    sed '#d' filename.txt
   

   Specify the line number you want to remove instead of the hash (#) symbol and run the command.

   For instance, to remove the second line from the foxinbox.txt file, type:

   

9. To use sed to delete lines within a line range, follow the syntax:

    sed '#,#d' filename.txt
   

   Replace the hash symbols with the beginning and end of the line range.

   For example, to delete lines 2, 3, and 4 from the foxinbox.txt file, run the command:

   

10. To delete empty lines (that contain no characters), run:

    sed '/^$/d' filename.txt
    

    The command uses a regular expression pattern ^$ to match empty lines. The ^ represents the start of a line, and $ represents the end of a line.

    For instance, to remove empty lines from the foxinbox.txt, run this command:

    

References:

1. Linux Crash Course - The sed Command

2. sed - wikipedia

3. sed, a stream editor - gnu

4. sed Command by IBM

5. Linux sed Command: How To Use the Stream Editor by phoenixNAP

6. How to Use sed Command to Delete a Line by phoenixNAP

7. How to Use Sed to Find and Replace a String in a File by phoenixNAP

Image source: Wikipedia

How awk Works

When you run awk, you specify an awk program that tells awk what to do. The program consists of a series of rules. Each rule specifies one pattern to search for and one action to perform upon finding the pattern.

Syntactically, a rule consists of a pattern followed by an action. The action is enclosed in braces to separate it from the pattern. Newlines usually separate rules. Therefore, a awk program looks like this:

awk [options] 'program' input-file(s)

OR

awk [options] 'pattern { action }' input-file(s)

• pattern: Specifies when the action should be performed. If omitted, the action is applied to every line.

• action: What to do when a line matches the pattern. Actions are enclosed in braces {}.

Single quotes around program makes the shell not to interpret any awk characters as special shell characters. The quotes also cause the shell to treat all of program as a single argument for awk, and allow program to be more than one line long.

Benefits of AWK

1. Supports complex pattern-matching and processing.

2. Designed for efficient text processing on both small and large files.

3. Easy to write and understand one-liners for basic tasks.

4. Changing data files.

5. Producing formatted reports.

6. Available on all Unix-like systems without the need for installation or setup.

Variables in AWK

Variables in AWK play a crucial role in processing text and data. They are used to store temporary data, manipulate fields, control the flow of the program, and customize output. AWK variables can be user-defined or built-in, with the latter providing access to various useful pieces of information or functionality. Some of the most commonly used built-in variables are:

• BEGIN and END - are not variables but special pattern blocks. The BEGIN block is executed before any input is read, and the END block is executed after all input has been processed. These blocks are useful for initialization and summary tasks, respectively.

• Arrays - AWK supports associative arrays, which can be indexed by string or number. Arrays are useful for collecting and organizing data dynamically during execution.

Hands-on Exercise Overview

The main goal of this hands-on exercise is to learn how to use awk utility to manipulate data.

The input file for the examples provided below is the mail-list.txt file, which represents a list of peoples’ names together with their email addresses and information about those people. Each record contains the name of a person, his/her phone number, his/her email address, and a code for his/her relationship with the author of the list. The columns are aligned using spaces. An ‘A’ in the last column means the person is an acquaintance. An ‘F’ in the last column means the person is a friend. An ‘R’ means that the person is a relative:

Hands-on Exercise

1. Print the 1st and 3rd columns:

    awk '{ print $1 "\t" $3}' mail-list.txt
   

   

2. Print lines that match a certain pattern. Search the input file mail-list.txt for the character string li:

    awk '/li/ { print $0 }' mail-list
   

   

   When lines containing ‘li’ are found, they are printed because print $0 means print the current line.

   The slashes indicate that ‘li’ is the pattern to search for. This type of pattern is called a regular expression. The pattern is allowed to match parts of words.

3. Print columns that match a specific pattern:

    awk '/.edu/ { print $3 }' mail-list.txt
   

   

   When awk locates a pattern match, the command will execute the whole record. You can change the default by issuing an instruction to display only certain fields.

4. Print every line that is longer than 55 characters:

    awk 'length($0) > 55' file
   

   awk has a built-in length function that returns the length of the string. From the command $0 variable stores the entire line and in the absence of a body block, the default action is taken, i.e., the print action. Therefore, in our mail-list.txt file, if a line has more than 55 characters, the comparison results to true, and the line is printed as shown below.

   

5. Print the total number of bytes used by mail-list.txt:

    ls -l mail-list | awk '{ x += $5 }
                       END { print "total bytes: " x }'
   

   

6. Count the lines in mail-list.txt:

    awk 'END { print NR }' mail-list.txt
   

   • END specifies that the action should be performed after all lines are processed.

   • NR is a built-in variable that keeps track of the number of records (lines) read.

7. Print the even-numbered lines in the mail-list.txt file:

    awk 'NR % 2 == 0' mail-list.txt
   

   

   • If you used the expression ‘NR % 2 == 1’ instead, the program would print the odd-numbered lines.

References:

1. Linux Crash Course - awk

2. A User’s Guide for GNU Awk

3. Getting Started with awk - O'REILLY

4. AWK Command in Linux with Examples (phoenixNAP)

5. Getting Started With AWK Command [Beginner's Guide - Linux Handbook]

6. AWK command in Linux/Unix - Digital Ocean

7. Advance your awk skills with two easy tutorials - opensource.com

8. A practical guide to learning awk - opensource.com

Types of Groups

There are 2 categories of groups in the Linux operating system:

• Primary group - When a new user is created on Linux using the useradd command, a group with the same name as the username is also created, and the user is added as the group's sole member. This group is the user's primary group. A user has only one primary group at any given time.

• Secondary (supplementary) groups - Additional groups are used to extend permissions and access rights of a user beyond what is provided by the primary group. For example, if users need access to a shared directory or need to execute tasks that require different group permissions, they can be added to the relevant secondary groups. Users can be members of multiple secondary groups.

Key Files for Group Management

• /etc/group: This file is a critical configuration file that defines the groups to which users belong. It's a text file that contains one entry per line, each describing a group.

  

  💡

  The password field stores a password for the group. It is usually empty (represented by an x or *), indicating that the group password is not used. Group passwords are a deprecated feature for managing group membership and access.

• /etc/gshadow: This file complements the /etc/group file by providing a secure way to store group passwords and manage group membership. Similar to /etc/group, it contains one entry per line for each group, but with a focus on security-sensitive information.

  

Other Common Groups

There are several common group names you might encounter in Linux:

• sudo – A member of this group can use the sudo command to elevate their privileges

• cdrom – Allows the user to mount the optical drive

• adm – Allows the user to monitor Linux system logs

• lpadmin – Allows the user to configure printers

• plugdev – Allows the user to access external storage devices

Hands-on Exercise Overview

This hands-on shows:

• how to add, delete, and modify groups;

• how to add users to and remove from the existing groups.

Hands-on Exercise

1. To find out which groups a user is a member of, run:

    groups
   

   

2. To find out which groups another user is a member of, run:

    groups username
   

   

3. To create a new group in Linux, run:

    sudo groupadd groupname
   

4. To confirm the new group is in the Linux /etc/group file:

    getent group groupname
   

   • getent: The command used to retrieve entries from databases configured in the system.

   • group: This specifies the database to query. In this case, group refers to the group database, which contains information about the groups on the system.

   • groupname: This is the name of the group you want to query.

5. To create a system group, run

    sudo groupadd -r groupname
   

   💡

   System groups are created for system purposes, such as running system services or daemon processes. They help in managing permissions and access rights for system processes and files.

   

6. To modify a group name, use the groupmod command:

    sudo groupmod -n new-name old-name
   

   

7. To add a user to an existing group while keeping them in their current groups, use the usermod command with the -a (append) and -G (groups) options:

    sudo usermod -aG groupname username
   

   Confirm if the user has been added by using the groups command.

   

8. An alternative for adding the user to an existing group is using the gpasswd command:

    sudo gpasswd -a username groupname
   

   • gpasswd - a command used for managing /etc/group and /etc/gshadow group files. It allows administrators to modify group passwords and memberships, including adding or removing users from groups and setting or changing group passwords.

   • -a: Option that stands for "add", indicating that you want to add a user to a group.

9. To remove a user from a specific group, use gpasswd command with -d option:

    sudo gpasswd -d username groupname
   

   

10. To delete a group, use the groupdel command:

    sudo groupdel groupname
    

    

References

1. Linux Crash Course - Managing Groups

2. User Management in Linux by phoenixNAP

3. Create New Groups in Linux With Groupadd Command

4. Linux sudo command explained

5. How to Add a User to a Linux Group

A user in Linux is associated with a user account, which consists of several properties defining their identity and privileges within the system. These properties are a username, UID (User ID), GID (Group ID), home directory, default shell, and password.

Types of Users in Linux

In Linux, users are categorized based on their roles, permissions, and the kind of access they have to the system's files and resources. Understanding the types of users is crucial for managing security and permissions on a Linux system. Here are the primary types of users found in Linux:

Understanding User Management Files

In Linux, user management revolves around several key files that store information about users, their passwords, groups, and group passwords. These files are critical for system security and user management tasks. Here's an overview of these essential files:

• /etc/passwd - The passwd file contains a list of user accounts and the corresponding user ID, group ID, home directory, and the default shell. It is readable by most users, but only root and sudo accounts can add new users or remove and modify existing user data.

  

• /etc/shadow - This file stores encrypted user password information and other password-related data such as the password expiration date, last change date, and account expiration date. It is only accessible by the root user or users with appropriate privileges. The restricted access and encryption add another security layer compared to the /etc/passwd file.

  

• /etc/sudoers- This file specifies which users have elevated permissions, on which machines, and for which directories. Admins can use this file to configure permissions for users and groups to use the sudo command.

• /etc/skel-The skel directory contains default configuration scripts and templates such as .bashrc and bash_profile. The templates are copied to the user's home directory when a new user is created, streamlining the provisioning of new user accounts.

  

Best Practices for User Management in Linux

Effective user management in Linux is essential for maintaining system security, performance, and user accessibility. Here are some best practices in short:

1. Avoid using the root account for daily tasks. Grant administrative privileges to trusted users via the sudo mechanism.

2. Follow the principle of least privilege (PoLP). Users should have only the permissions necessary for their roles.

3. Periodically review user accounts and remove or disable those that are no longer in use or are temporary.

4. Use strong password policies.

5. For temporary users or employees, set account expiration dates.

6. Keep track of user activities, especially for users with sudo privileges. Tools like auditd can be configured to monitor and report on specific user actions.

7. As users change roles within an organization, regularly update their group memberships and permissions to reflect their current needs.

8. Educate users about security best practices, including password security, phishing awareness, and safe internet usage.

9. Regularly backup user data and configuration files. This includes home directories and critical system files like /etc/passwd and /etc/shadow.

10. For environments with multiple Linux systems, consider centralizing user management using LDAP, Active Directory integration, or other directory services to streamline the process.

Hands-on Exercise Overview

Linux, being a multi-user system, provides a comprehensive set of tools and commands for user management. This hands-on shows how to add, modify, and remove a user on Ubuntu 20.04.

Hands-on Exercise

1. To find out which users and how many users you have on the server, run:

    cat /etc/passwd | wc -l
   

2. To determine which users are currently logged in a Linux system, use:

    who -H
   

   

3. To create a new user, run:

    sudo useradd username
   

   Root privileges are required when adding a new user to the system because system-wide changes happen. The command does not provide any output.

   💡

   By default, useradd command adds normal user accounts.

4. To verify if the new user is visible, run:

    cat /etc/passwd | grep username
   

   

5. To add a system user, add the -r option:

    sudo useradd -r sysuser
   

   💡

   The system user is useful when you automate any tasks. It does not log in interactively and is used for running in the background and scheduling tasks and processes. Desktop distributions will not show a system user on the login screen because if it did show, there would be a mess.

6. To add a user with a home directory, use the -m option:

    sudo useradd -m username
    ls -l /home/
   

   

   💡

   Always use the -m option with useradd if you want the home directory to be created automatically. Some distributions might have defaults that automatically create the home directory, but it's good practice to explicitly use the -m option to ensure consistency across different environments.

7. Each Linux distribution can set its own default values for the useradd command. These defaults are typically configured in the:

    cat /etc/default/useradd
   

   

8. To modify various attributes of an existing user account in Linux, use usermod command. Visit for more information: Linux usermod Command Explained

9. To remove a user account, use:

    sudo userdel username
   

   When a user is deleted from the system, its home directory is not deleted by default.

   

10. To remove a user account with its home directory, use -r option:

    sudo userdel -r username
    

12. To change the password for another user, run:

    sudo passwd username
    

    Since root privileges are accessed, the current password of the user is not asked.

References

1. Linux Crash Course - Managing Users

2. Linux sudo command explained

3. User Management in Linux

4. Linux usermod Command Explained

Crontab (Cron Table) is a configuration file that specifies the schedule of cron jobs. It contains commands to be executed along with their scheduled times. Each user on a system can have their crontab file, typically at /var/spool/cron/crontabs/, and there is also a system-wide crontab at /etc/crontab.

How 'cron' Works

Under the hood, the cron system is relatively simple yet powerful, and it operates by regularly checking for scheduled tasks and executing them at the specified times.

The algorithm used by cron is as follows:

1. On start-up, the cron daemon (crond) looks for the crontab files.

2. For each crontab file found, determine the next time in the future that each command must run.

3. Place those commands on the event list with their corresponding time and their "five field" time specifier.

4. Enter main loop:

   1. Examine the task entry at the head of the queue, and compute how far in the future it must run.

   2. Sleep for that period of time.

   3. On awakening and after verifying the correct time, execute the task at the head of the queue (in the background) with the privileges of the user who created it.

   4. Determine the next time in the future to run this command and place it back on the event list at that time value.

Understanding Crontab Format

The crontab file is composed of lines of comments and cron jobs and looks as follows:

Comments begin with a comment mark #, and must be on a line by themselves.

The syntax of a cron job line in a crontab file must use the following format:

MIN HOUR DOM MON DOW CMD

The first five fields, each separated by a single space, represent time intervals and tell Cron when to initiate the cron job. These fields are followed by the command (CMD), which is usually a path to a script or a system command.

Cron expression examples:

Nonstandard predefined scheduling definitions:

Some cron implementations support the following non-standard macros:

Crontab Commands

1. To edit your crontab file, run:

    crontab -e
   

2. To list your current cron jobs, run:

    crontab -l
   

3. To remove your crontab file, run:

    crontab -r
   

4. To edit another user's crontab (requires superuser privileges), run:

    sudo crontab -u username -e
   

   💡

   The direct editing of user crontab files in /var/spool/cron/crontabs/ is typically restricted to the system's cron service and should not be edited manually. Permissions are set to prevent unauthorized access. For managing cron jobs, the crontab utility provides a safe and standardized way for users to edit their cron schedules.

Examples of Cron Jobs

1. To run a script every Monday at 6:30 AM:

    30 6 * * 1 /path/to/script.sh
   

2. To delete temporary files every hour:

    0 * * * * /bin/rm -rf /tmp/*
   

3. To check disk space usage and email the results every Sunday at 7 PM:

    0 19 * * 0 df -h | mail -s "Disk Space Report" user@example.com
   

Tips for Writing Cron Jobs

1. Specify full paths to files and programs because cron jobs run in a limited environment, meaning that many of the environmental variables available in a full-user session might not be present. Also, it helps to prevent accidental or malicious execution of the wrong program.

2. By default, cron sends the output of jobs via email to the user's account. If you don't want this, redirect the output to a file or /dev/null.

3. Before setting up a cron task, run your script or command to see if you have any problems with permissions or paths.

4. If too many jobs run at the same time — especially the backups and compiles —the system would run out of RAM and nearly fill the swap file. As a result, the system thrashes and the performance decreases, so nothing gets done. In such a case, add more memory and improve how tasks are scheduled.

5. Remove a task that is poorly written and use large amounts of memory.

The crond service assumes that the host computer runs all the time. That means that if the computer is turned off during a period when cron jobs were scheduled to run, they will not run until the next time they are scheduled. This might cause problems if they are critical cron jobs. Fortunately, there is another option for running jobs at regular intervals: anacron.

anacron

The anacron program performs the same function as crond, but it adds the ability to run skipped jobs, such as if the computer was off or otherwise unable to run the job for one or more cycles. This is very useful for laptops and other computers that are turned off or put into sleep mode.

As soon as the computer is turned on and booted, anacron checks to see whether configured jobs missed their last scheduled run. If they have, those jobs run immediately, but only once (no matter how many cycles have been missed). For example, if a weekly job did not run for three weeks because the system was shut down while you were on vacation, it would be run soon after you turn the computer on, but only once, not three times.

References:

1. Linux Crash Course - Scheduling Tasks with Cron

2. Wikipedia: cron

3. How I use cron in Linux

4. How to schedule jobs using the Linux 'cron' utility

5. How to Create and Set Up a Cron Job in Linux

SSH public key authentication relies on asymmetric cryptographic algorithms that generate a pair of separate keys (a key pair), one "private" and the other "public". Keep the private key secret and store it on your local computer to connect to the remote system. Conceivably, share the public key with anyone without compromising the private key; you store it on the remote system in a .ssh/authorized_keys directory.

Hands-on Exercise Overview

This hands-on exercise uses public key authentication to connect to the Ubuntu 20.04/Vagrant server by generating an SSH key pair on the local Ubuntu 22.04 machine. The connection between the local machine and the Ubuntu/Vagrant server is established using the bridged mode that allows the two machines to be in the same network.

Prerequisites:

To use SSH public key authentication:

• The remote and local systems must have OpenSSH installed.

• You need to be able to transfer your public key to the remote system. Therefore, you must either be able to log into the remote system with an established account username and password/passphrase, or have an administrator on the remote system add the public key to the ~/.ssh/authorized_keys file in your account.

Steps:

1. Open the terminal (CTRL+ALT+T).

2. Verify that the ssh client is installed on the local machine:

    which ssh
   

   OR

    ssh -V
   

3. Check for existing keys by listing the ssh directory with on the local machine:

    ls -l ~/.ssh
   

   Generating new keys overwrites the current ones by default. However, indicating a new name for the keys saves them to different files.

   If there are no existing keys, the output indicates the directory does not exist.

   If some keys are found, they should be backed up before continuing.

4. Generate a key pair with ssh-keygen:

    ssh-keygen -b 4096
   

   • ssh-keygen is the tool used to create SSH keys, which are used for secure communication between machines.

   • -b specifies the number of bits in the key to create. More bits generally mean better security.

   • 4096 is the number of bits in the key. This is considered a good balance between security and performance for most SSH applications.

5. Verify the key pair exists:

   

6. View the public key:

   

7. Check whether the server works by connecting to it first:

    ssh username@ip_address
   

   

8. Add the public key to the server by copying it manually or using the ssh-copy-id tool:

    ssh-copy-id username@ip_address
   

9. Public keys are in the ~/.ssh/authorized_keys file and every time a new public is added, it appears as a new one line.

   

10. Now, connect to the server from the local machine by using the SSH private key and check whether the public key authentication works:

    

References:

1. Set up SSH public key authentication to connect to a remote system

2. About authentication

3. What is SSH Public Key Authentication?

4. How To Configure SSH Key-Based Authentication on a Linux Server

5. Setting up public key authentication over SSH

6. Use SSH Public Key Authentication on Linux, macOS, and Windows

Hands-on Exercise

1. To view the current password expiry information for a user, use:

    sudo chage -l username
   

   

   • -l, --list - shows account aging information. This will display various information like the last password change date, password expiry date, password inactive period, account expiry date, etc.

2. To set the maximum number of days during which a password is valid:

    sudo chage -M days username
   

   

   This allows to ensure that a user must change their password every 30 days.

3. To set a minimum number of days between password changes:

    sudo chage -m days username
   

   

   Following this command, once the user ‘karla’ changes its password, she will have to wait for a minimum of 20 days before they are allowed to change it again.

4. To start warning a user N days before their password expires, use:

    sudo chage -W days username
   

   

5. To ensure an account is automatically disabled after a certain date, for example, at the end of a contract or project, set:

    sudo chage -E YYYY-MM-DD username
   

   

   After the specified date, the user ‘karla’ will not be able to access her account.

6. To set the number of days of inactivity after a password has expired before the account is locked, use the -I option. For example, to lock the ‘karla’ account if its password is not changed within 10 days after its expiry, use:

    sudo chage -I 10 karla
   

   

   This policy ensures that inactive accounts are locked after a certain period, providing an additional layer of security.

7. The chage command also has an interactive mode, which can be used by not specifying any options. In this mode, the command will prompt you for all the information it needs. To enter the interactive mode, use:

    sudo chage karla
   

   

8. To remove the password expiry for a certain user, use the -M option with a value of -1:

    sudo chage -M -1 username
   

   

9. To remove account expiration for a certain user, similar to password expiration, use the -E option and passing -1 as the value:

    sudo chage -E -1 username
   

   

Exit Codes

The chage command exits with the following values:

• 0 - success;

  

• 1 - permission denied;

  

• 2 - invalid command syntax;

  

• 15 - cannot find the shadow password file

The passwd Command

There is another command that is not related to chage command. It is a passwd command which changes passwords for user accounts. Its primary use is for changing passwords rather than managing detailed aspects of password policies. It's the command used for regular password updates or if an account's password needs to be reset for security reasons. A normal user may only change the password for their own account, while the superuser may change the password for any account.

1. To lock a certain user, use:

    sudo passwd -l username
   

   

2. To view the password status of the named account, use:

    sudo passwd -S username
   

   

3. To unlock a certain user, use:

    sudo passwd -u username
   

   

Conclusion

The chage command is essential for maintaining security and compliance within a Linux environment. By enforcing regular password changes and setting minimum and maximum password ages, organizations can significantly reduce the risks associated with stale or compromised passwords. Additionally, the ability to disable accounts automatically helps manage access for temporary users or contractors, ensuring that only current, authorized users can access system resources.

While passwd plays a critical role in password management, its capabilities regarding the direct setting of account and password expiration dates are limited. For detailed management of password and account expiration policies, including setting specific expiration dates, use the chage command for password-related settings and usermod for account expiration. These tools offer the granularity and control needed for effective user account management and security policy enforcement.

References:

1. chage(1) — Linux manual page

2. Linux Crash Course - User Account & Password Expiration

3. chage Command in Linux with Examples

It is recommended not to open multiple terminals in one server. Instead, foreground and background processes should be used.

Background Processes

Running a process in the background means that a process is still running but doesn't have control over the terminal. It is associated with the specific terminal that started it but does not block access to the shell. Instead, it executes in the background, leaving the user able to interact with the system while the command runs. This is particularly useful for long-running tasks that don't require immediate user interaction. For example, a long-running task like a large file download can be initiated in the background, freeing up the terminal for other commands.

Because background processes return control to the shell immediately without waiting for the process to complete, many background processes can run at the same time.

There are two ways to send a process to the background:

1. Append & to the end of the command to send it to the background immediately. For example, cp largefile.txt backup & will copy the file while returning the terminal for other commands.

2. Press Ctrl+Z to suspend the running process. Then, type bg to send it to the background and resume execution.

Foreground Processes

By default, processes are started in the foreground. This means that until the program exits or changes state, it is not allowed to interact with the shell as it takes control of the terminal. This means that the user can interact with the process directly.

For example, if a text editor is run from the command line without backgrounding it, the terminal becomes occupied by the text editor, and it can't be used for other commands until the text editor is closed.

Because of the way that a foreground process interacts with its terminal, there can be only a single foreground process for every terminal window.

Job Identification

In a multitasking environment, multiple running processes in the background can be managed using job identifiers (job IDs) provided by the shell. The jobs command lists the running jobs and their statuses in the current shell session. A job can be a command or a pipeline of commands that are started as a single unit.

Here's a breakdown of the typical output format:

• Job Number: Each job is assigned a unique job number (e.g., [1], [2], [3]) for identification. This number is used when you want to refer to a specific job using other job control commands like fg (foreground) and bg (background).

• Current Job Indicators:

  • The + sign indicates the current job, which is the most recent job that was put into the background or stopped.

  • The - sign indicates the previous job, which is the second most recent job that was manipulated.

  • Jobs without these indicators are neither the current nor the previous job.

• State: This shows the current state of the job. Common states include:

  • Running: The job is actively running in the background.

  • Stopped: The job has been suspended (e.g., by pressing Ctrl+Z). It is not currently running but can be resumed.

  • Done: The job has been completed. It means the command finished its execution.

• Command Name: This is the command or pipeline that constitutes the job. It shows what command the job is running or was running.

Hands-on Exercise Overview

This hands-on will focus on managing foreground and background processes and will demonstrate how to leverage your shell’s job control functions to gain more flexibility in how you run commands.

Managing Foreground Processes

1. Start a command that runs indefinitely and suspend it with CTRL + Z:

    htop
   

   This utility will continue to run and update its display until it is terminated. Press CTRL + Z to suspend the running foreground process.

   

   When pressing CTRL + Z, the terminal registers a “suspend” command, which then sends the SIGTSTP (signal terminal stop) signal to the foreground process. Essentially, this will pause the execution of the command and return control to the terminal.

2. Run ping command to connect to google.com every 5 seconds and terminate it with CTRL + C.

    ping -i 5 google.com
   

   

   Linux terminals are usually configured to send the “SIGINT” signal (short for “signal interrupt”) to the current foreground process when the user presses the CTRL + C key combination. The SIGINT signal tells the program that the user has requested termination using the keyboard.

Managing Background Processes

3. To start the previous ping command as a background process, append a & to the end of the command:

    ping -i 5 google.com &
   

   This tells the shell not to wait for the process to complete, but instead to begin execution and immediately return the user to a prompt. The output of the command will still display in the terminal (unless redirected), but you can type additional commands as the background process continues.

   The bash job control system will return output like this:

   

   You’ll then receive the normal output from the ping command:

   

   However, you can also type commands at the same time. The background process’s output will be mixed among the input and output of your foreground processes, but it will not interfere with the execution of the foreground processes.

4. To list all stopped or backgrounded processes, use the jobs command:

    jobs
   

   If the previous ping command is still running in the background, the jobs command’s output will be similar to this:

   

   This indicates that you currently have two background processes, one is stopped and the second one is running. The [1], [2] represent the command’s job spec or job number. Reference this with other job and process control commands, like kill, fg, and bg by preceding the job number with a percentage sign. In this case, reference for instance, the second job as %2.

5. To stop a specific background process use the kill command with the associated job number.

    kill %2
   

   

Moving Processes between Foreground and Background

6. To move the currently running process to the background, first, the process must be stopped with CTRL+Z, and then bg command should be used to start it again in the background:

    bg
   

   The job status line will be shown again, this time with the ampersand appended:

   

   By default, the bg command operates on the most recently-stopped process. Because background processes return control to the shell immediately without waiting for the process to complete, many background processes can run at the same time. If multiple processes have been stopped in a row without starting them again, you can reference a specific process by its job number to move the correct process to the background.

   💡

   Note that not all commands can be backgrounded. Some processes will automatically terminate if they detect that they have been started with their standard input and output directly connected to an active terminal.

7. To move background processes to the foreground, type:

    fg
   

   This operates on your most recently backgrounded process (indicated by the + in the jobs command’s output). It immediately suspends the process and puts it into the foreground. To specify a different job, use its job number:

    fg %2
   

References:

1. Linux Crash Course: Background (bg) and Foreground (fg)

2. Processes

3. How To Use Bash's Job Control to Manage Foreground and Background Processes

Alias Benefits

• Save time by typing less.

• Improve command memorability through custom names.

• Increase efficiency by automating repetitive tasks.

Linux Alias Syntax

To create an alias, use the alias command and its syntax as follows:

alias [name]='[value]'

• alias: Invokes the alias command.

• [name]: the name of the alias is being created. This is what will be typed in the terminal instead of the full command. A name is a user-defined string, excluding special characters and 'alias', 'unalias' keywords that cannot be used as names.

• [value]: the actual command that the alias represents. This can be a simple command or a more complex command chain. Commands can also include options, arguments, and variables.

Create Aliases in Linux

There are two types of aliases to create in Linux:

• Temporary. Add them using the alias command.

• Permanent. These require editing the system files.

Create a Temporary Alias

Aliases defined directly in the terminal are temporary and last only for the current terminal session. Use the alias command to create a temporary alias. For example, adding move as an alias for the mv command with the option of asking for confirmation before overwriting:

alias move='mv -i'

Another use for aliases is to create a shortcut for running scripts. To do this, provide the absolute path to the script as the value:

alias frename='Example/Test/file_rename.sh'

In this example, using frename as a command runs the file_rename.sh bash script.

Create a Permanent Alias

To make an alias permanent, add it to the shell's configuration file, such as ~/.bashrc or ~/.bash_profile, and then source the file or restart your terminal. Depending on the type of shell you are using, use:

• Bash shell: ~/.bashrc

• Zsh shell: ~/.zshrc

• Fish shell: ~/.config/fish/config.fish

Start by opening the shell configuration file in a text editor. In this example, we are using the Bash shell and vim text editor:

sudo vim ~/.bashrc

Scroll down until you find a section that lists default system aliases. For ease of use, create a separate section with a descriptive comment and add your aliases using the alias command syntax.

In our example:

#Custom aliases
alias c='clear';
alias df='df -h -x squashfs -x tmpfs -x devtmpfs'

Once all of the new aliases are added, press ESC, type :wq! to save the changes to the configuration file.

The new aliases automatically load in the next terminal session. If you want to use them in the current session, load the configuration file using the source command:

source ~/.bashrc

The output of the df command before creating an alias for it:

The output of the df command after creating an alias for it:

List Aliases in Linux

Using the alias command on its own displays a list of all currently set aliases:

alias

Remove Aliases in Linux

To remove an alias, use the unalias command with the following syntax:

unalias [name]

Adding the -a option allows to remove all aliases:

unalias -a

References:

1. Linux Crash Course - Bash Aliases

2. Bash aliases you can’t live without

3. Linux alias Command: How to Use It With Examples

4. Linux Commands Cheat Sheet