web analytics

logo Meta Data Science

By Massoud Seifi, Ph.D. Data Scientist

Add Unique ID Column to CSV Files

If you want to add a unique id column to the end of a CSV file, you can simply do it via a bash script following the steps below:

  • Count the number of lines in the input file using wc command and save it to a variable.
  • Create a temoprary file using tempfile command.
  • Using uuidgen command, generate n(=#lines) unique ids and save them to the temporary file.
  • Merge the input file with the temporary file containing unique ids using paste command.
  • Remove the temporary file.
Add unique ids to a CSV file (gen_uid.sh) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

#!/bin/bash
# Author: Massoud Seifi, Ph.D. @ MetaDataScience.com

# Count the number of lines in the input file
lines=$(wc -l < "$1")

# Generate unique ids and store them in a temporary file
uids=$(tempfile)
for i in `seq 1 $lines`; do uuidgen -t; done > "$uids"

# Insert the unique ids at the end of each line of the input file
paste --delimiter "," "$1" "$uids"

# Remove the temporary file
rm "$uids"

For example, running the script on the sample CSV file below:

Sample CSV file (test.csv) download
1
2
3

Mike,Smith,29
Jane,Clark,18
Brian,Jones,21

1

$./gen_uid.sh test.csv

The output will be:

1
2
3

Mike,Smith,29,e802dbee-617a-11e5-9c46-0862664fe7a7
Jane,Clark,18,e802ef30-617a-11e5-9c46-0862664fe7a7
Brian,Jones,21,e80302e0-617a-11e5-9c46-0862664fe7a7

If first line of your CSV file is the column headings, you can use code below:

Add unique ids to a CSV file with headers (gen_uid_header.sh) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

#!/bin/bash
# Author: Massoud Seifi, Ph.D. @ MetaDataScience.com

# Count the number of lines in the input file
lines=$(wc -l < "$1")

# Generate unique ids and store them in a temporary file
uids=$(tempfile)
for i in `seq 1 $lines`; do if [ $i == 1 ]; then echo "uid"; else uuidgen -t; fi; done > "$uids"

# Insert the unique ids at the end of each line of the input file
paste --delimiter "," "$1" "$uids"

# Remove the temporary file
rm "$uids"
Sample CSV file with headers (test_header.csv) download
1
2
3
4

first_name,last_name,age
Mike,Smith,29
Jane,Clark,18
Brian,Jones,21

1

$./gen_uid_heade.sh test_header.csv

1
2
3
4

first_name,last_name,age,uid
Mike,Smith,29,76de1cc4-617c-11e5-9c46-0862664fe7a7
Jane,Clark,18,76de2de0-617c-11e5-9c46-0862664fe7a7
Brian,Jones,21,76de3f2e-617c-11e5-9c46-0862664fe7a7

Comments