Bash Basics #7: Advanced Bash Techniques

Welcome to the seventh and final part of our comprehensive Bash basics course!

In the first six modules, we mastered the fundamentals of script structure and shebang, variables and parameter expansions, control structures, functions and scopes, I/O data streams, and error handling and signal traps.

In this concluding lesson, we combine all learned building blocks and elevate your scripting to enterprise level: We implement professional command-line argument parsing with getopts, use process substitution <(...) for comparing streams in memory, extract text components via $BASH_REMATCH, protect scripts against race conditions, and establish a binding best-practice checklist for production use.

Professional CLI Argument Parsing with `getopts`

Many simple scripts rely on rigid positional parameters ($1, $2). For professional tools, however, administrators expect flexible command-line switches (flags like -v, -h, or -f /path).

The built-in Bash tool getopts parses options in a standards-compliant and robust way:


┌─────────────────────────────────────────────────────────────┐
│                 GETOPTS PARSING FLOW                        │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  Call: ./backup.sh -v -t /mnt/data -p 8080                  │
│         │                                                   │
│         ▼                                                   │
│  Option string "vt:p:" (colon = value required!)            │
│         │                                                   │
│         ├── -v ──> Sets VERBOSE=true                         │
│         ├── -t ──> Stores argument in TARGET_DIR            │
│         ├── -p ──> Stores argument in PORT                  │
│         └── ?  ──> Shows help & exits on errors             │
│                                                             │
└─────────────────────────────────────────────────────────────┘

The Options Syntax of `getopts`:

h: The switch -h expects no subsequent argument (boolean flag).
f:: The colon indicates: The switch -f requires a mandatory value (stored in $OPTARG).
A leading colon (e.g. :hf:v) enables silent error mode, so you can output custom error messages.

Complete Practical Example with `getopts`:


#!/usr/bin/env bash
set -euo pipefail

# Initialize defaults
VERBOSE=false
TARGET_FILE=""
PORT=80

show_help() {
    cat <<EOF
Usage: $(basename "$0") [OPTIONS]

Options:
  -h            Show this help
  -v            Enable verbose mode
  -f FILE       Path to target file (required)
  -p PORT       Server port (default: 80)
EOF
    exit 0
}

# Parse options
while getopts ":hvf:p:" opt; do
    case "$opt" in
        h)
            show_help
            ;;
        v)
            VERBOSE=true
            ;;
        f)
            TARGET_FILE="$OPTARG"
            ;;
        p)
            PORT="$OPTARG"
            ;;
        \?)
            echo "Error: Invalid option -$OPTARG" >&2
            show_help
            ;;
        :)
            echo "Error: Option -$OPTARG requires an argument!" >&2
            exit 1
            ;;
    esac
done

# Remove processed options from argument list
shift $(( OPTIND - 1 ))

# Validate required parameters
if [[ -z "$TARGET_FILE" ]]; then
    echo "Error: The option -f FILE is mandatory!" >&2
    exit 1
fi

echo "Starting with target file '$TARGET_FILE' on port $PORT (verbose: $VERBOSE)..."

Text Extraction with Regular Expressions and `$BASH_REMATCH`

When you use parentheses ( ) for grouping in a [[ ... =~ ... ]] condition, Bash automatically stores all found sub-patterns (capture groups) in the special array $BASH_REMATCH:


#!/usr/bin/env bash
set -euo pipefail

URL="https://api.admindocs.de:8443/v1/status"

# Regex with capture groups for protocol, host, port, and path:
REGEX='^(https?)://([^:/]+):?([0-9]*)(/.*)$'

if [[ "$URL" =~ $REGEX ]]; then
    echo "Valid URL detected!"
    echo "Protocol: ${BASH_REMATCH[1]}"  # https
    echo "Hostname : ${BASH_REMATCH[2]}"  # api.admindocs.de
    echo "Port     : ${BASH_REMATCH[3]:-default (80/443)}" # 8443
    echo "Path     : ${BASH_REMATCH[4]}"  # /v1/status
else
    echo "Invalid URL format!" >&2
fi

Process Substitution: Treating Data Streams as Files

With process substitution (<(command) or >(command)), Bash provides the output of a program to the calling command as a temporary file descriptor (/dev/fd/X).

1. Directly Comparing Output of Two Commands (`diff`)

Without process substitution, you would need to create intermediate files on disk. With <(...), you compare outputs directly in memory:


# Compare installed packages of two servers via SSH:
diff -u <(ssh server01 "dpkg -l | sort") <(ssh server02 "dpkg -l | sort")

2. Avoiding Subshell Variable Loss in Loops

When you feed a loop over a pipe with data (cat file | while ...), the loop runs in a subshell – all variables set inside it are lost. With process substitution, the loop stays in the main shell:


#!/usr/bin/env bash
set -euo pipefail

total_bytes=0

# The while loop runs in the current shell (no subshell!):
while IFS= read -r bytes; do
    total_bytes=$(( total_bytes + bytes ))
done < <(awk '{print $5}' /var/log/nginx/access.log)

echo "Total transfer volume: $total_bytes bytes"

Subshells `( ... )` vs. Command Groups `{ ...; }`

Bash distinguishes between two types of command grouping:

Construct	Syntax	Behavior & Impact
Subshell	`( command1; command2 )`	Starts a new child process. Changes to directory (`cd`) or variables have no effect on the main script.
Command Group	`{ command1; command2; }`	Executes commands in the current shell. Ideal for redirecting multiple commands together into a file.


# Example 1: Temporarily change directory without affecting main script (subshell):
(
    cd /tmp
    tar -czf backup.tar.gz data/
)
# pwd is still the original directory afterwards!

# Example 2: Redirect entire block together to a log file (command group):
{
    echo "=== SYSTEM REPORT ==="
    uptime
    free -h
    echo "====================="
} > /var/log/system_report.log

The Ultimate Best-Practice Checklist for Enterprise Scripts

Before you put an administration script into production, check the following quality criteria:


┌─────────────────────────────────────────────────────────────┐
│                 ENTERPRISE BASH CHECKLIST                   │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  [x] Shebang portable: #!/usr/bin/env bash                  │
│  [x] Strict Mode: set -euo pipefail & IFS set               │
│  [x] Signal traps: trap cleanup EXIT INT TERM defined       │
│  [x] Quoting: Variables consistently protected with "${VAR}"│
│  [x] Scoping: Function variables encapsulated with "local"  │
│  [x] Test operators: Modern [[ ... ]] tests used            │
│  [x] I/O: Errors sent to stderr via echo "..." >&2          │
│  [x] Static check: shellcheck reports 0 warnings            │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Exercises & Practice Check

❗ Practice tasks for lesson #7:

Task 1 (CLI Backup Tool with getopts):

Write a complete administration script backup_tool.sh that processes options -s SOURCE, -t TARGET, and optionally -v (verbose) with getopts, cleans up temporary directories via trap, and uses the Bash Strict Mode.

Task 2 (Regex Parsing with BASH_REMATCH):

Write a script that accepts an email address in the format user@domain.tld and outputs the username and domain separately using BASH_REMATCH.

Task 3 (Process Substitution Diff):

Create a script that compares the output of df -h / before and after creating a test file using diff -u in memory.

Sample Solution for Task 1:


#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

SOURCE=""
TARGET=""
VERBOSE=false

show_help() {
    echo "Usage: $(basename "$0") -s SOURCE -t TARGET [-v]"
    exit 0
}

while getopts ":s:t:vh" opt; do
    case "$opt" in
        s) SOURCE="$OPTARG" ;;
        t) TARGET="$OPTARG" ;;
        v) VERBOSE=true ;;
        h) show_help ;;
        \?) echo "Error: Unknown option -$OPTARG" >&2; exit 1 ;;
        :)  echo "Error: Option -$OPTARG requires argument!" >&2; exit 1 ;;
    esac
done

[[ -z "$SOURCE" || -z "$TARGET" ]] && { echo "Error: -s and -t are mandatory!" >&2; exit 1; }
[[ -d "$SOURCE" ]] || { echo "Error: Source directory $SOURCE does not exist!" >&2; exit 1; }

TEMP_DIR=$(mktemp -d /tmp/backup.XXXXXX)
cleanup() { rm -rf "$TEMP_DIR"; }
trap cleanup EXIT INT TERM

$VERBOSE && echo "Creating archive in temporary folder $TEMP_DIR..."
ARCHIVE_NAME="backup_$(date '+%Y%m%d_%H%M%S').tar.gz"
tar -czf "$TEMP_DIR/$ARCHIVE_NAME" -C "$SOURCE" .

mkdir -p "$TARGET"
cp "$TEMP_DIR/$ARCHIVE_NAME" "$TARGET/"
echo "Success: Backup saved to $TARGET/$ARCHIVE_NAME"

Command Reference (Cheatsheet)

Construct / Command	Function & Description
`getopts ":ab:c" opt`	Standard parser for CLI command-line options
`$OPTARG` / `$OPTIND`	Contains the current option argument / the argument index
`[[ $str =~ $regex ]]`	Performs regex comparison and populates `$BASH_REMATCH`
`${BASH_REMATCH[1]}`	Contains the first capture group of the regular expression
`cmd <(subcmd)`	Process substitution: Passes output of `subcmd` as file descriptor
`( command )`	Executes commands in an isolated subshell
`{ command; }`	Groups commands in the current shell context
`mktemp -d /tmp/app.XXXXXX`	Creates a collision-free temporary directory
`shellcheck script.sh`	Runs static source code and security analysis

Further Resources

Resource	Description
Bash Hackers Wiki: getopts Tutorial	Comprehensive guide to CLI option parsing
ShellCheck Tool & Documentation	Official linter and static analysis tool for Bash
Google Shell Style Guide	Binding industry standard for clean shell scripts
Linux Command Line Processor Guide	Fundamental knowledge on shells, I/O streams, and pipes
chmod and File Permissions	Linux permission concepts in detail

Conclusion: Your Path to Professional Bash Scripting

Congratulations on completing our 7-part Bash basics course!

From the first lines with shebang and file permissions, through variables, control structures, and functions, all the way to signal traps, getopts parsing, and process substitution – you have learned the complete toolkit for writing modern, secure, and maintainable automation scripts for productive Linux server environments.

💡 Tip: Good scripts are created through discipline: Activate Strict Mode (set -euo pipefail) in every new script, secure temporary resources with trap, and always run your scripts through shellcheck before production use.

For in-depth administration tasks and server architectures, we recommend getting started with our course modules on Linux Administration and the Arch Linux Series.

👉 Course overview: All lessons of the Bash basics course

Professional CLI Argument Parsing with getopts

The Options Syntax of getopts:

Complete Practical Example with getopts:

Text Extraction with Regular Expressions and $BASH_REMATCH