Jeffimgcls Hi, I'm Jeff! Linkedin Mail

Comprehensive Bash Scripting Guide

1. What is Bash?

Bash (Bourne Again SHell) is the GNU Project’s shell—a command-language interpreter and scripting language.

It lets you automate repetitive tasks, glue together system utilities, and build powerful command-line tools.

2. Creating & Running a Script

Shebang Line

At the top of your script, specify the interpreter:

#!/usr/bin/env bash

# !/usr/bin/env bash is more portable than hard-coding /bin/bash.

Make Executable

chmod +x myscript.sh

Run It

./myscript.sh       # current directory
bash myscript.sh    # explicitly with bash

3. Comments

# This is a single-line comment

: <<'COMMENT'
This is a
multi-line
comment
COMMENT

4. Variables

Assignment

No spaces around =:

name="Jeff"
count=42

Usage

Prefix with $:

echo "Hello, $name! You have $count new messages."

Read-only / Export

readonly PI=3.14159     # cannot be reassigned
export PATH="$PATH:/opt/bin"  # makes it available to child processes

5. Positional Parameters & Special Variables

  • $0 – script name
  • $1…$9 – first through ninth argument
  • $@ – all args as separate words
  • $* – all args as a single word
  • $# – number of args
  • $? – exit status of last command
  • $$ – PID of current shell
  • $! – PID of last background job
echo "Script name: $0"
echo "First arg: $1, second: $2"
echo "All args: $@ (count=$#)"

6. Quoting

  • Unquoted: word-splitting & globbing apply
  • Double quotes "": expand variables & command substitution, but preserve spaces
  • Single quotes '': literal (no expansion)
file="My Documents/report.txt"
echo $file        # -> My Documents/report.txt  (split into two words)
echo "$file"      # -> My Documents/report.txt  (one argument)

echo "Today is $(date)"    # command substitution
echo 'Today is $(date)'    # literally $(date)

7. Command Substitution

now=$(date +%Y-%m-%d_%H%M)
files=$(ls *.log)
echo "Logs: $files"

8. Conditional Tests

8.1 test / [ ] vs [[ ]]

[ ] is POSIX-compatible; more limited. [[ ]] is Bash-built, safer for regex and pattern matching.

8.2 Numeric & String Checks

if [[ $age -ge 18 && $age -le 65 ]]; then
  echo "Working age"
fi

if [[ -z $name ]]; then
  echo "Name is empty"
fi

if [[ $file == *.txt ]]; then
  echo "Text file"
fi

8.3 File Tests

TestDescription
-e fileexists
-f fileregular file
-d dirdirectory
-r filereadable
-w filewritable
-x fileexecutable
if [[ -x /usr/bin/grep ]]; then
  echo "grep is executable"
fi

8.4 case Statement

case $fruit in
  apple)  echo "Red or green";;
  banana) echo "Yellow";;
  *)      echo "Unknown fruit";;
esac

9. Loops

9.1 for Loop

for file in *.sh; do
  echo "Script: $file"
done

9.2 C-style for

for ((i=1; i<=5; i++)); do
  echo "Iteration $i"
done

9.3 while & until

count=1
while (( count <= 3 )); do
  echo "Count: $count"
  ((count++))
done

until [[ -f /tmp/ready.flag ]]; do
  echo "Waiting..."
  sleep 5
done

10. Functions

say_hello() {
  local name="$1"   # local scope
  echo "Hello, $name!"
}

say_hello "World"

local confines variables to the function.

11. Arrays

# Declare
fruits=(apple banana cherry)
# Access
echo "${fruits[1]}"      # banana
# All elements
echo "${fruits[@]}"      # apple banana cherry
# Loop
for f in "${fruits[@]}"; do
  echo "Fruit: $f"
done
# Length
echo "Count: ${#fruits[@]}"

12. Arithmetic

a=5; b=3
sum=$(( a + b ))
(( a++ ))         # a = a + 1
if (( a > b )); then echo "a is larger"; fi
echo "Result: $sum"

13. I/O Redirection & Pipes

cmd > out.txt          # stdout → file (overwrite)
cmd >> out.txt         # stdout → file (append)
cmd 2> err.log         # stderr → file
cmd &> both.log        # stdout+stderr → file
cmd < input.txt        # stdin ← file
cmd1 | cmd2            # pipe stdout(cmd1) → stdin(cmd2)

14. Here Documents & Strings

cat < greeting.txt
Dear $name,

Welcome to our system!
Regards,
Admin
EOF

15. Error Handling & Debugging

Exit on error:

set -o errexit    # or set -e
set -o nounset    # unset variables are errors
set -o pipefail   # pipeline fails if any command fails

Trace execution:

set -o xtrace     # or set -x

trap to catch signals or script exit:

trap 'echo "Interrupted! Cleaning up..."; cleanup; exit 1' INT TERM
trap 'echo "Exiting normally";' EXIT

16. Best Practices

  • Always quote variables ("$var").
  • Use #!/usr/bin/env bash.
  • Use [[ … ]] over [ … ] for tests.
  • Name functions and variables clearly; use local inside functions.
  • Add comments and a usage/help section.
  • Validate inputs, check return codes, and fail fast.
  • Keep scripts idempotent where possible.

17. Example: Simple Backup Script

#!/usr/bin/env bash
set -euo pipefail

# Defaults
SRC="/home/jeff/data"
DEST="/mnt/backup/$(date +%F)"
LOG="/var/log/backup_$(date +%F).log"

# Usage info
usage() {
  echo "Usage: $0 [-s source_dir] [-d dest_dir]"
  exit 1
}

# Parse options
while getopts ":s:d:h" opt; do
  case $opt in
    s) SRC="$OPTARG" ;;
    d) DEST="$OPTARG" ;;
    h) usage ;;
    *) usage ;;
  esac
done

# Create backup directory
mkdir -p "$DEST"

echo "Backing up $SRC → $DEST" | tee -a "$LOG"
rsync -av --delete "$SRC/" "$DEST/" | tee -a "$LOG"

echo "Backup complete at $(date)" | tee -a "$LOG"

Advanced Bash Scripting Topics

1. “Strict Mode” and Shell Options

#!/usr/bin/env bash
set -euo pipefail
shopt -s inherit_errexit  # Bash ≥4.4: make errexit work in functions & subshells

2. Advanced Parameter Expansion

# Default / fallback values
: "${VAR:-default}"   # returns "default" if VAR unset or empty
: "${VAR:=default}"   # sets VAR="default" if unset or empty

# Substring removal
path="/usr/local/bin/script.sh"
echo "${path##*/}"    # → script.sh
echo "${path%/*}"     # → /usr/local/bin

# Search & replace
text="foo bar foo"
echo "${text//foo/baz}"  # → baz bar baz

# Length & slicing
str="abcdef"
echo "${#str}"        # → 6
echo "${str:2:3}"     # → cde

3. Arrays & Associative Arrays

# Indexed arrays
colors=(red green blue)
echo "${colors[@]:1:2}"     # → green blue

# Associative arrays
declare -A user
user[name]="Jeff"
user[id]=1001
for key in "${!user[@]}"; do
  printf "%s → %s\n" "$key" "${user[$key]}"
done

4. Process Substitution & Here-Strings

# Compare two files without temporary files
diff <(sort file1) <(sort file2)

# Feed a string into stdin
grep foo <<<"$some_long_string"

5. Coprocesses

coproc nc_server { nc -l 1234; }
# Read/write via ${nc_server[0]} and ${nc_server[1]}
echo "Hello" >&"${nc_server[1]}"
read line <&"${nc_server[0]}"
echo "Client said: $line"

6. Advanced I/O Redirection

# Open a new FD 3 for writing
exec 3> output.log
echo "Log entry" >&3
# Close FD 3
exec 3>&-

# Redirect stderr to stdout, but keep stdout separate
command 2>&1 1>/dev/null

# Prevent accidental overwrites
set -o noclobber
echo "data" >| file.txt  # force overwrite despite noclobber

7. Robust Temporary Files

tmp=$(mktemp -d) || { echo "Failed to create temp dir" >&2; exit 1; }
trap 'rm -rf "$tmp"' EXIT  # always clean up

8. Signal Handling & Traps

cleanup() {
  echo "Cleaning up…"
  # remove temp files, stop background jobs, etc.
}
trap cleanup EXIT          # always run, even on errors
trap 'echo "Interrupted!"; exit 1' SIGINT SIGTERM

# Capture errors with line number
trap 'echo "Error on line $LINENO"; cleanup; exit 1' ERR

9. In-Depth Debugging

export PS4='+(${BASH_SOURCE##*/}:${LINENO}) '
bash -x yourscript.sh 2> debug.log

# Send xtrace output to separate FD (Bash ≥4.1)
exec 5>trace.log
BASH_XTRACEFD=5
set -o xtrace

10. Modularization & Libraries

# lib.sh
say() { printf "[%s] %s\n" "$(date +'%T')" "$*"; }

# main script
source /path/to/lib.sh
say "Starting backup…"

11. Advanced getopts & Long Options

options=$(getopt -o hv -l help,verbose,output: -- "$@") || exit 1
eval set -- "$options"

verbose=0; output=""
while true; do
  case $1 in
    -h|--help)    show_help; exit;;
    -v|--verbose) verbose=1; shift;;
    -o|--output)  output="$2"; shift 2;;
    --) shift; break;;
  esac
done

12. Concurrency & Parallelism

# Run up to 4 jobs in parallel
maxjobs=4; njobs=0
for file in *.txt; do
  process_file "$file" &
  ((++njobs>=maxjobs)) && wait -n && ((njobs--))
done
wait

# Using xargs
ls *.img | xargs -n1 -P4 convert -resize 800x600

13. Advanced Pattern Matching (extglob)

shopt -s extglob
for f in !(*.bak|*.tmp); do
  echo "Processing $f"
done

14. Error‐Resilient Pipelines

# Without pipefail, this might succeed even if grep fails
grep foo bigfile | head -n5

# With errexit + pipefail, script exits on failure
set -o errexit -o pipefail
grep foo bigfile | head -n5

15. Logging Framework

logfile="/var/log/myscript.log"
log() {
  local lvl="$1"; shift
  printf '%s [%s] %s\n' "$(date +'%F %T')" "$lvl" "$*" >>"$logfile"
}
log INFO "Script started"
log ERROR "Could not connect to DB"

16. Testing & Linting

# Lint with ShellCheck
shellcheck yourscript.sh

# Unit tests with bats or shUnit2
bats test_suite.bats

17. Packaging & Distribution

  • Add --version and --help flags
  • Provide make install into /usr/local/bin
  • Use #!/usr/bin/env bash for portability
  • Sign scripts with GPG for public release

18. Performance Tips

  • Avoid external processes in tight loops—use built-ins when possible.
  • Coalesce I/O: write once to file descriptors rather than multiple echos.
  • Benchmark with time and optimize hotspots.

19. Further Learning