3 Text Formatting Tricks: remove duplicate lines, remove empty lines, convert upper cases to lowers

In this article I will show you 3 basic text formatting tricks. It is very usefull to know how to manipulate text files.

Remove repeated lines in a file:

I have this file:

$ cat textfile
sample line1
sample line1
sample line2
sample line2
sample line3

I will remove the repeated lines with the uniq command:

Syntax: uniq repeated_lines_file no_repeated_lines_file

$ uniq textfile newtextfile
$ cat newtextfile
sample line1
sample line2
sample line3

Uniq can also be piped on a command: command | uniq

$ cat textfile | uniq
sample line1
sample line2
sample line3

OR, have the output redirected:

$ cat textfile | uniq > samplefile

Remove blank / empty lines from file:

This simple sed oneliner is very efficient for removing empty lines in files: cat samplefile | sed /^$/d

First creating the file with blank lines:
$ echo -e "\n\nHello\n" > samplefile

$ cat samplefile | sed /^$/d

OR, redirect it in a file:

$ cat samplefile | sed /^$/d > file1

3. Convert Upper to Lower cases:

$ echo "HELLO HELLO" > file1
$ cat file1 | tr '[A-Z]' '[a-z]'
hello hello

OR redirect to file:
$ cat file1 | tr '[A-Z]' '[a-z]' > file2

3. Convert Lower to Upper cases:

$ echo "hello hello" > file1
$ cat file1 | tr '[a-z]' '[A-Z]'

OR redirect to file:
$ cat file1 | tr '[a-z]' '[A-Z]'> file2

Tagged with: , , , , , , , , ,
Posted in The Linux and Unix Articles!

Leave a Reply

Your email address will not be published. Required fields are marked *




Subscribe to get the latest Linux news and how to guides directly on your e-mail!