Terminal Tricks using Stream Editor ( sed )

In this article, I have included some of the basic sed one-liners. These amazing one liners make difficult tasks look so simple and elegant. Moreover,they make you believe that you can perform such tedious tasks in a simple one-liner.

So, What is sed ?

It’s a non-interactive text editor. It reads an input from STDIN or a given file, executes a series of editing commands and returns the output to STDOUT. It is designed to be useful when the input files are too large and to perform multiple global editing functions efficiently.

Remember: sed is extremely powerful and a must-know command.

Indentation ( Useful in indenting C, C++ etc files )

1. Delete leading whitespace from front of each line

$ sed ‘s/^[ \t]*//’ sourcefile > destfile

2. Delete trailing whitespace from end of each line

$ sed ‘s/[ \t]*$//’ sourcefile > destfile

3. Insert a tab at beginning of each line

$ sed ‘s/^/\t/’ sourcefile > destfile

4. We can also perform these operations on a range of lines by specifying the line numbers

$ sed ’5,7 s/^/\t/’ test.c test1.c ( this will add a tab at the beginning of lines 5,6,7 )

---

Line Spacing

1. Delete all the blank lines from a file

$ sed ‘/^$/d’ sourcefile > destfile

2. Insert a blank line above every line which matches specified regular expression (regex)

$ sed ‘/regex/{x;p;x;}’ sourcefile > destfile

( $ sed ’4,6 {x;p;x}’ sourcefile > destfile // to add a line above line 4,5 and 6 )

3. Insert a blank line below every line which matches specified regular expression (regex)

$ sed ‘/regex/G’ sourcefile > destfile

( $ sed ’4,6 G’ sourcefile > destfile // to add a line below line 4,5 and 6 )

4. No more than one blank line between lines

$ sed ‘/^$/d;G’ sourcefile > destfile

5. Delete all leading blank lines at the top

$ sed ‘/./,$!d’ sourcefile > destfile

---

Numbering

1. Number each line of a file

$ sed = sourcefile | $ sed ‘N;s/\n/\t/’ > destfile

2. Number each line of file, but only print numbers if line is not blank

$ sed ‘/./=’ sourcefile | $ sed ‘/./N; s/\n/ /’ > destfile

---

Substitution

1. Substitute “foo” with “bar

$ sed ‘s/foo/bar/g’ sourcefile > destfile

2. Substitute “foo” with “bar” only for lines containing “jar”

$ sed ‘/jar/s/foo/bar/g’ sourcefile > destfile

3. Interchange columns of a text file ( especially useful for CSV files or any database )

for e.g. a text file has info seperated with a delimiter

ram:shyam:hari

one:two:three

a:b

$ sed ‘s/\(.*\):\(.*\):\(.*\)/\3:\2:\1/’ sourcefile > destfile

Output :

hari:shyam:ram

three:two:one

b:a

4. Retain the first word of a line

$ sed ‘s/\([a-z]*\).*/\1/’ sourcefile > destfile

5. Detect consecutive duplicated words

$ sed -n ‘/\([a-z][a-z]*\) \1/p’ sourcefile > destfile

6. Delete lines matching pattern

$ sed ‘/pattern/d’ sourcefile > destfile

---

Paragraph

1. Print paragraph if it contains AA

$ sed -e “/./{H;$!d;}” -e “x;/AA/!d;” sourcefile > destfile

2. Print paragraph if it contains AA and BB and CC (in any order)

$ sed -e “/./{H;$!d;}” -e “x;/AA/!d;/BB/!d;/CC/!d” sourcefile > destfile

3. Print paragraph if it contains AA or BB or CC

$ sed -e “/./{H;$!d;}” -e “x;/AA/b” -e “/BB/b” -e “/CC/b” -e d sourcefile > destfile

---

Join lines

1. Join pairs of lines side-by-side

$ sed “$!N;s/\n//” sourcefile > destfile

2. Join all lines

$ sed  “:a;$!N;s/\n//;ta;”  sourcefile > destfile

---

Pallindrome

1. Match a five letter pallindrome ( for e.g. radar )

$ sed ‘\([a-z]\)\([a-z]\)[a-z]\2\1′ sourcefile > destfile

---

Misc

1. Delete the first 10 lines of a file

$ sed ’1,10d’ sourcefile > destfile

2. Print only lines that are of 65 characters or longer

$ sed -n ‘/^.\{65\}/p’ sourcefile > destfile

Source : Sourceforge

---