The sed command in Linux stands for Stream Editor. It is a powerful utility that parses and transforms text, using a simple, compact programming language. sed is particularly useful for editing large files or streams without opening them in a traditional text editor. It operates line by line, applies operations specified by the user, and outputs the result to the standard output.
How sed Works
The workflow of sed (Stream Editor) involves reading, executing commands, and displaying the output, operating in a cycle over the input stream.
Here's a step-by-step explanation of how sed works:
Reading: sed reads one line at a time from the input stream. This can be a file or input piped from another command.
Placing: Each line read into sed is placed into a space called the "pattern space," where the operations are performed. The end-of-line newline character is temporarily removed during this step.
Executing Commands: Once a line is in the pattern space, sed executes all the commands you've specified in the order they were given. These commands can modify the contents of the pattern space (like replacing text, deleting the line, etc.), and they can be conditioned to only run if the line matches a specified pattern. sed commands are powerful and can include:
Substitution (e.g., s/pattern/replacement/): Search for a pattern and replace it with the specified text.
Deletion (e.g., d): Delete lines that match a specified pattern or line number.
Insertion and Appending (e.g., i\text, a\text): Insert or append text before or after a matching pattern or line number.
Reading and Writing (e.g., r filename, w filename): Read content from a file into the pattern space, or write content from the pattern space to a file.
Line Addressing: Apply commands to specific lines or ranges of lines, either specified by line numbers or patterns.
Printing: After executing all commands on the line in the pattern space, sed by default prints the contents of the pattern space to the standard output. It can be redirected to a file or piped to another command for further processing.This behavior can be modified with options like -n, which suppresses automatic printing unless explicitly told to print (e.g., with the p command).
Cycle Continues: The pattern space is then cleared, and sed reads the next line of input, continuing the cycle until all lines from the input have been processed.
💡
The sed command uses two workspaces for holding the line being modified: 1. The pattern space, where the selected line is held; 2. The hold space, where a line can be stored temporarily.
Linux sed Syntax
The main syntax for using Linux sed command is:
sed [OPTIONS]... [SCRIPT] [INPUTFILE...]
[OPTIONS]: These are command-line options that modify how sed operates. Some of the most commonly used options include:
-e script: Allows you to specify a script containing one or more sed commands. This option is useful when chaining multiple commands.
-i: Enables in-place editing of files. When used, sed modifies the input file directly.
-n: Suppresses automatic printing of the pattern space. Typically used with the p command to print specific lines.
[script]: This is where you specify the sed commands to be executed. The script usually contains one or more commands that tell sed how to process the input text.
[input-file]: This is the source of input data for sed to process. It can be one or more filenames. If no input file is specified, sed reads from the standard input, which allows it to be used in pipelines with other commands.
💡
sed does not affect the source file unless instructed. To overwrite the original file, use the -i option to save the modifications. However, such practice is not recommended before testing out the command output. Alternatively, save the edits to a different (or new) file. Redirect the output by adding > newfilename.txt at the end of the command.
Hands-on Exercise Overview
This hands-on exercise shows ten commonly used sed commands and a foxinbox.txt file with the following content was taken as a sample file:
Hands-on Exercise
To replace text, use the substitute command s and delimiters (in most cases, slashes - /) for separating text fields.
sed 's/old_string/new_string/' filename.txt
For example, to replace instances of box with the word bin, run:
💡
It does not matter what is used as the delimiter. Use any other delimiter instead of the forward slash ( / ) if the forward slash must be searched for.
To replace multiple instances of the same word within a single line, add the g flag to the command to change all of them:
sed 's/old_string/new_string/g' filename.txt
For example, to replace the word box with the word bin in the file foxinbox.txt every time, type:
💡
By default, sed only replaces the first occurrence of the specified string in each line. It searches for the first instance of the specified word in a line, replaces it, and moves on to the next line.
To replace a specific occurrence of the string in a line, add a number flag such as 1, 2 etc:
sed 's/old_string/new_string/#' filename.txt
For example, to substitute the second occurrence of the word box in each line of the file with the word bin, use this command:
To print out just the lines that have substitution under the given conditions, use the syntax:
sed -n 's/old_string/new_string/p' filename.txt
For example, to replace the second instance of the word box in a line and print the line where the change took place:
💡
By default, the sed command prints out the entire file content, along with the substitute text in its output. If you have a lot of text and want to focus on the lines with the applied changes, add the needed attributes to the command.
To ignore case while substituting text, add the i subcommand at the end of the command:
sed 's/old_string/new_string/i' filename.txt
For example, the command for changing upper and lowercase instances of the word fox in the foxinbox.txt file is:
To substitute a string in a specific line, add the line number as a prefix to the s subcommand:
sed '# s/old_string/new_string/' filename.txt
For example, to replace the word socks with sandals only in the fourth line (4) of the text use the command:
To replace multiple instances of a string within a line range, but not the entire text, specify the range where you want sed to substitute. The syntax is:
sed '#,# s/old_string/new_string/' filename.txt
Replace the first # with the initial line number and the second # with the last line number you want to include.
For instance, to replace the last two instances of the word socks in the file foxinbox.txt (located in the fourth and sixth line) with the word sandals, run:
To delete a line from a file with the sed command, use the d subcommand and the syntax:
sed '#d' filename.txt
Specify the line number you want to remove instead of the hash (#) symbol and run the command.
For instance, to remove the second line from the foxinbox.txt file, type:
To use sed to delete lines within a line range, follow the syntax:
sed '#,#d' filename.txt
Replace the hash symbols with the beginning and end of the line range.
For example, to delete lines 2, 3, and 4 from the foxinbox.txt file, run the command:
To delete empty lines (that contain no characters), run:
sed '/^$/d' filename.txt
The command uses a regular expression pattern ^$ to match empty lines. The ^ represents the start of a line, and $ represents the end of a line.
For instance, to remove empty lines from the foxinbox.txt, run this command:
References:
Linux Crash Course - The sed Command
sed - wikipedia
sed, a stream editor - gnu
sed Command by IBM
Linux sed Command: How To Use the Stream Editor by phoenixNAP
How to Use sed Command to Delete a Line by phoenixNAP
How to Use Sed to Find and Replace a String in a File by phoenixNAP
