17.15. Shell Programming

Shell scripts are often used to group a set of commands that should execute together. A shell script is an executable text file that consists of a sequence of shell commands that, when run, execute those commands in order. Shells also have language constructs such as loops and conditionals, which can be used to create complex featureful scripts. These shell programming constructs can also be used at the command line and in shell scripts. Below, we present a few examples of Bash shell programming. Other shells (e.g. zsh, sh, tcsh) have their own shell programming language syntax, but they all provide similar functionality.

The first line of a Bash script starts with a special comment # followed by a bang ! and the path to the program that executes the contents of the script file. Below is an example of a simple Bash script, simplescript.sh (note that by convention shell scripts are named with a .sh suffix).

#!/bin/bash
pwd
whoami
date

Before it can be run, the simplescript.sh file needs to be given the executable permission. A user can check its permissions by running ls -l and set the executable permission using chmod if it is not set. For example:

$ ls -l simplescript.sh
-rw------- 1 sam users 31 Mar 28 18:26 simplescript.sh

$ chmod u+x simplescirpt.sh

$ ls -l simplescript.sh
-rwx------ 1 sam users 31 Mar 28 18:26 simplescript.sh

Once it is marked as executable, the script can be run on the command line like any executable file, and the sequence of commands listed in the script will execute in order:

$ ./simplescript.sh
/home/sam/
sam
Tue 28 Mar 2023 06:27:38 PM EDT

Unix shell programs support language constructs such as iteration (i.e., for and while loops) and conditionals (i.e., if-else). Although these constructs can be used at the command line, they are more commonly used in shell scripts to specify more complicated execution structure. Shell scripts can also make use of command line arguments and variables, which help the script work with different input values.

Below are a couple examples of Bash shell scripts using some of these more advanced features. The first, runsh.sh, is a script that repeats some timed executions of a program. In this example, the executable file name and the number of executions to repeat are given as command line arguments to the script. This example also shows how to test for some constraints on command line arguments. Here the program executable command line argument is required to run the script, but the number of executions to repeat is an optional command line argument (the script uses a default value of 5 when a user runs the script without specifying this command line option). The script uses the if-elif construct to test for command line options. It also shows an example of a for loop used to repeat the number of runs specified by the caller.

#!/bin/bash

# this script performs a number of timed runs of a
# program given as a required command line argument
# the number of runs is an optional cmdln arg

# script variable N for the number of runs
# assigning N default value of 5
N=5

# get command line arguments
#  1: name of program is required
#  2: number of runs is optional
# $@ is the array of command line args (like argv)
# $# is the number of command line args (like argc)

# get the program executable name
# note: space between "[ $" and between "0 ]" is important!
if [[ $# -eq 0 ]]
then
  echo "Error, usage: ./run.sh ./a.out [num times]"
  exit 1
else
  PROG=$1
fi

# if they gave the optional cmdln arg, set N to it
if [[ $# -gt 1 ]]
then
  N=$2
fi

# it's useful to output some info about what is run:
echo "running $PROG $N times: "
date
echo "======================= "

# do N timed runs of PROG:
for((n=1; n <= $N; n++))
do
echo " "
echo "run $n:"
time $PROG
done

There are a few things to note in this script. First, there are no types in Bash scripts, so it is up to the user to use values appropriately based on their implied types.

Second, note that Bash shell variables are set using = operator and their values are referenced using $var_name:

N=5      # set N to 5
n <= $N  # use value of N

Third, note the syntax for accessing command line arguments in the Bash script:

  • $@ is the array of command line arguments, which is similar to argv in C.

  • $# is the number of command line arguments, which is similar to argc in C except that the name of the Bash script file does not count as one of the arguments (i.e., a Bash script run with no command line arguments has a value of 0 for $#, whereas a C program run with no command line arguments has a value of 1 for argc).

Fourth, note the if-else syntax, and its general form:

if [[ cond ]]
then
   # if true stmts
else
   # if false stmts
fi

Bash uses then, else, and fi to denote the start of the if block, the start of the else block, and the end of the if-else statement. The [[ and ]] symbols are used to denote the conditional statement, and the space chars between them and the condition are very important (without these Bash will not correctly parse this statement). The conditional operator syntax is -gt represents greater than, -lt represents less than, -eq represents equal to, and others use similar notation.

Finally, note the syntax of the for loop is similar to for loops in C, where the body of the for loop is denoted by do and done and the and contain the init; cond; step parts. Here is the general form of this type of Bash for loop (Bash actually has more than one form of for loop):

for((init; cond; step))
do
  # for loop body statements
done

Below are some sample runs of the run.sh script:

$ ./run.sh
Error, usage: ./run.sh ./a.out [num times]

$ ./run.sh ./myprog 2
running ./myprog 2 times:
Wed Mar 29 11:34:34 AM EDT 2023
=======================

run 1:
myprog result = 300000

real	0m0.060s
user	0m0.003s
sys	0m0.006s

run 2:
myprog result = 300000

real	0m0.059s
user	0m0.003s
sys	0m0.006s


$ ./run.sh ./myprog
running ./myprog 5 times:
Wed Mar 29 11:34:37 AM EDT 2023
=======================

run 1:
myprog result = 300000

real	0m0.057s
user	0m0.008s
sys	0m0.000s

run 2:
myprog result = 300000

real	0m0.057s
user	0m0.004s
sys	0m0.004s

run 3:
myprog result = 300000

real	0m0.057s
user	0m0.010s
sys	0m0.000s

run 4:
myprog result = 300000

real	0m0.056s
user	0m0.009s
sys	0m0.000s

run 5:
myprog result = 300000

real	0m0.055s
user	0m0.004s
sys	0m0.004s

Often when using scripts like run.sh that produce a lot of output, it is helpful to save the output to a file that can be examined after completion. One way to do this is to use I/O redirection to redirect the script’s output to a file. Below is an example run of the run.sh script with the command line options ./myprog and 10 that redirects its stderr and stdout ouput to a file named results:

$ ./run.sh ./myprog 10 &> results

In addition to supporting syntax similar to a C for loop, Bash supports for loop syntax for iterating over a set of elements. The forloops.sh Bash script (listed below), has a few examples of this type of for loop, showing different ways in which the set of elements iterated over is obtained.

#!/bin/bash

echo "for loop over set of values"
# iterate over a set of given values
# repeats once for each element in the list
for i in cat dog bunny
  do
    echo "next animal is: $i"
  done

echo

echo "for loop over sequence"
# {1..5} is the set of values in the sequence: 1,2,3,4,5
# this iterates over "infilei" where i: 1,2,3,4,5
for i in infile{1..5}
  do
    echo "next file is: $i"
  done

echo

echo "for loop over set created with ls"
# this iterates over all the files in a subdirectory named input
# $(ls input/): creates a set of value that are all the file
# and directory names in the input/ subdirectory
for i in $(ls input/)
  do
    echo "next input/ file is: $i"
  done

When run, the output is:

$ ./forloops.sh

for loop over set of values
next animal is: cat
next animal is: dog
next animal is: bunny

for loop over sequence
next file is: infile1
next file is: infile2
next file is: infile3
next file is: infile4
next file is: infile5

for loop over set created with ls
next input/ file is: albums
next input/ file is: artists
next input/ file is: bands
next input/ file is: songs

In general, combining Unix commands with Bash shell language constructs like loops, conditionals, command lines, and variables, allow a user to write powerful shell scripts to perform complex actions. We have shown just a few examples in this section. See a Bash shell programming guide for more information.

17.15.1. References

For more information see: