Linux Uniq Command Overview

Linux Uniq Command return the uniq content from the input text or from the input file data. It filters out all the duplicate data and thus returning unique data. You can provide input data by using it with some other command such as sort -u | uniq or you can provide input data in a text file to uniq command. Similarly either you can provide its output to some other command as standard output or you can save the output in a file. To summarise

  • It will find the unique content from input data.
  • Input data can be provided to it in the form of standard output or by any file.
  • It can provide output data either in standard output or in a file.
Note: For Linux Uniq command to work the data inside the file or standard input should be sorted, otherwise uniq command will not work. To make the data sorted you can use sort -u before uniq.

Linux Uniq Command Syntax

Syntax for Uniq command is

UNIQ [--options] [--input_data]

Linux Uniq Command Example

Let us look more deeper into Uniq command with the help of an example

Consider a file with the name of hostel list - hostel_list.txt

ASHOK BHAVAN
ASHOK BHAVAN
ASHOK BHAVAN
BHAGIRATHI BHAVAN
GANGA BHAVAN
RUDRAKSH BHAVAN
RUDRAKSH BHAVAN
RUDRAKSH BHAVAN
RUDRAKSH BHAVAN
YAMUNA BHAVAN
KAILASH BHAVAN
NARMADA BHAVAN
NARMADA BHAVAN
KAVERI BHAVAN
KAVERI BHAVAN

Linux Uniq Command Example Case 1:

Let us fetch all the unique hostel names. Linux Uniq Command for that will be

UNIQ hostel_list.txt
ASHOK BHAVAN
BHAGIRATHI BHAVAN
GANGA BHAVAN
RUDRAKSH BHAVAN
YAMUNA BHAVAN
KAILASH BHAVAN
NARMADA BHAVAN
KAVERI BHAVAN

Linux Uniq Command Example Case 2:

Let us fetch all the unique hostel names with their duplicity count number as well. Linux Uniq Command for that will be

UNIQ -c hostel_list.txt

"-c" parameter is used for finding the count of occurrences

3 ASHOK BHAVAN
1 BHAGIRATHI BHAVAN
1 GANGA BHAVAN
4 RUDRAKSH BHAVAN
1 YAMUNA BHAVAN
1 KAILASH BHAVAN
2 NARMADA BHAVAN
2 KAVERI BHAVAN

Linux Uniq Command Example Case 3:

Let us only fetch duplicate hostel names. Command for that will be

UNIQ -d hostel_list.txt

"-d" parameter is used for fetching only duplicate entries

ASHOK BHAVAN
RUDRAKSH BHAVAN
NARMADA BHAVAN
KAVERI BHAVAN

Linux Uniq Command Example Case 4:

Let us fetch all the hostel names with single occurrences in hostel_list.txt file. Command for that will be

UNIQ -u hostel_list.txt

"-u" parameter is used for fetching single occurrences hostel names.

BHAGIRATHI BHAVAN
GANGA BHAVAN
YAMUNA BHAVAN
KAILASH BHAVAN

Linux Uniq Command Example Case 4:

Suppose you want to find unique hostel names by ignoring letter cases. Command for that will be

UNIQ -u -i hostel_list.txt

"-i" parameter is used for applying ignore letter case to uniq query.

BHAGIRATHI BHAVAN
GANGA BHAVAN
YAMUNA BHAVAN
KAILASH BHAVAN

Linux Uniq Command Example Case 5:

Suppose you want to skip first "N" fields for comparison and count duplicates using "-c" . Command for that will be

UNIQ -c -f 1 hostel_list.txt

In "-f 1" "-f" will be used to skip fields and " 1" in "-f 1" will skip first field in each line.

15 ASHOK BHAVAN

Linux Uniq Command Example 6:

Suppose you want to skip first "N" characters for comparison and count duplicates using "-c". Command for that will be

UNIQ -c -s 3 hostel_list.txt

In "-s 1" "-s" will be used to skip chars and " 3" in "-f 3" will skip first 3 chars in each line.

3 ASHOK BHAVAN
1 BHAGIRATHI BHAVAN
1 GANGA BHAVAN
4 RUDRAKSH BHAVAN
1 YAMUNA BHAVAN
1 KAILASH BHAVAN
2 NARMADA BHAVAN
2 KAVERI BHAVAN



This article is contributed by Ankur Rastogi on Fri Apr 15 2016 22:15:44 GMT+0530 (IST)

If you want to add more comments to the article or you see any thing incorrect please write a comment below and we will surely get back to you.