Bash Basics #5: Input and Output in Bash

Welcome to the fifth part of our technical series on Linux and Bash programming!

After learning how to create and secure scripts, work with variables, use control structures and loops, and modularize code into functions in the previous modules, today we focus on the lifeline of every Linux system: input and output data streams (I/O).

The fundamental Unix principle is: "Everything is a stream / file" and "Write programs that do one thing and do it well. Write programs to work together". A script that cannot flexibly ingest, process, redirect, and persistently store data is useless for practical system administration.

In this lesson, you will learn how Linux manages data streams through file descriptors (0, 1, 2), how to deliberately separate standard output and error messages (2>, 2>&1, &>), how pipelines (|) and tee chain data in memory, how to create multi-line configurations via here-documents, and how to safely read user input with read -r.

File Descriptors: stdin (0), stdout (1), and stderr (2)

Every process that the Linux shell starts automatically receives three open data channels from the kernel (file descriptors):


┌─────────────────────────────────────────────────────────────┐
│                 STANDARD DATA STREAMS IN LINUX               │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  [ Keyboard / File / Stream ]                               │
│         │                                                   │
│         ▼                                                   │
│  0: Standard Input (stdin)                                  │
│         │                                                   │
│         ▼                                                   │
│  ┌──────────────┐                                           │
│  │ Linux Process│                                           │
│  └──────┬───────┘                                           │
│         │                                                   │
│         ├──────────────────────────┐                        │
│         ▼                          ▼                        │
│  1: Standard Output (stdout)  2: Standard Error (stderr)    │
│  Normal program output        Error messages & warnings     │
│         │                          │                        │
│         ▼                          ▼                        │
│  [ Screen / File / Pipe ]     [ Screen / Error Log ]        │
│                                                             │
└─────────────────────────────────────────────────────────────┘

The Three Channels at a Glance:

0 – stdin (Standard Input): The input stream through which programs receive data from the keyboard, from files, or from other programs.
1 – stdout (Standard Output): The output stream for regular program results.
2 – stderr (Standard Error): The separate channel for error messages, status, and warning notices.

💡 Why two separate output channels? The separation of stdout (1) and stderr (2) ensures that a program can forward usable data through a pipe, while warnings or errors remain visible on the screen instead of corrupting the downstream program.

I/O Redirection: Controlling Output and Errors

With redirection operators, you redirect data streams into files or pseudo-devices like /dev/null:


┌─────────────────────────────────────────────────────────────┐
│                 REDIRECTION MATRIX OVERVIEW                  │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  Operator       Meaning & Behavior                          │
│  ─────────────────────────────────────────────────────────  │
│  > file         Redirects stdout to file (overwrites file)  │
│  >> file        Appends stdout to file (append mode)        │
│  2> file        Redirects ONLY stderr (errors) to file      │
│  2>> file       Appends ONLY stderr to file                 │
│  &> file        Redirects stdout AND stderr together        │
│  2>&1           Duplicates stderr to stdout's channel       │
│  < file         Reads stdin from file                       │
│  >/dev/null 2>&1 Discards all output & errors completely    │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Practical Examples for Redirections:


# 1. Save regular output to file (overwriting):
echo "Server initialized" > /var/log/init.log

# 2. Append output to existing log:
echo "[$(date '+%F %T')] Service started" >> /var/log/app.log

# 3. Write only error messages to separate error log:
find /var/ -name "*.conf" 2> /tmp/find_errors.log

# 4. Completely silence error messages (/dev/null):
grep -r "SECRET_KEY" /etc/ 2>/dev/null

# 5. Write stdout and stderr together to the same log file:
./backup.sh &> /var/log/backup.log
# POSIX-compliant alternative:
./backup.sh > /var/log/backup.log 2>&1

Outputting Custom Error Messages to `stderr` (`>&2`)

When your script encounters an error, you should explicitly send the message to stderr:


#!/usr/bin/env bash
set -euo pipefail

if [[ ! -f "/etc/app.conf" ]]; then
    # '> &2' redirects the output of this echo command to stderr (channel 2):
    echo "CRITICAL ERROR: Configuration file /etc/app.conf missing!" >&2
    exit 1
fi

Pipelines (`|`) and Parallel Output with `tee`

A pipeline (|) connects the standard output (stdout) of the left command directly to the standard input (stdin) of the right command in memory – without temporary intermediate files.


# Filter and count all running Nginx processes:
ps aux | grep nginx | grep -v grep | wc -l

# Find the 5 largest files in the home directory:
du -ah ~/ | sort -rh | head -n 5

The `tee` Tool: T-Junction for Data Streams

Often you want to see a command's output on the screen AND save it to a file simultaneously. For this, use tee:


┌─────────────────────────────────────────────────────────────┐
│                 HOW TEE WORKS                                │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  Command ──> [ stdout ] ──► [ tee -a app.log ]              │
│                                   │                         │
│                    ┌──────────────┴─────────────┐           │
│                    ▼                            ▼           │
│             [ Screen ]                [ File app.log ]       │
│                                                             │
└─────────────────────────────────────────────────────────────┘


# Execute deployment, monitor on screen, and archive in log:
./deploy.sh | tee -a /var/log/deploy.log

Multi-line Text: Here-Documents (`<<EOF`) and Here-Strings (`<<<`)

When you want to generate multi-line configuration files, SQL queries, or emails directly in your script, here-documents are the cleanest tool.

1. Standard Here-Document with Variable Expansion


#!/usr/bin/env bash
set -euo pipefail

DOMAIN="example.com"
PORT=8080

# Generate an Nginx configuration file:
cat <<EOF > /tmp/nginx_${DOMAIN}.conf
server {
    listen 80;
    server_name ${DOMAIN};

    location / {
        proxy_pass http://127.0.0.1:${PORT};
        proxy_set_header Host \$host;
    }
}
EOF

💡 Prevent variable expansion (<<'EOF'): If the text contains quotation marks, $ characters, or Bash code that should not be expanded by the script, enclose the delimiter in quotes (<<'EOF').

2. Tab Indentation with `<<-EOF`

The minus sign (<<-EOF) ensures that leading tab characters (not spaces!) in the generated text are automatically stripped, so your script code remains cleanly indented.

3. Here-Strings (`<<<`)

A here-string passes a single variable or a short string directly to the stdin of a program:


IP="192.168.1.1"
# Passes $IP directly to bc or cut without an echo pipe:
cut -d'.' -f1 <<< "$IP"  # Output: 192

Interactive Input with `read -r`

The read command reads user input from the keyboard or from data streams.

Important Parameters of `read`:

Option	Function	Example
`-r`	Mandatory: Prevents misinterpretation of backslashes	`read -r text`
`-p "text"`	Displays an input prompt	`read -r -p "Name: " name`
`-s`	Silent mode: Hides input (ideal for passwords)	`read -rs -p "Password: " pw`
`-t sec`	Timeout: Automatically aborts after x seconds	`read -t 10 -p "Continue? (10s): " a`
`-n num`	Reads maximum num characters (without pressing Enter)	`read -n 1 -p "Y/N? " answer`
`-a arr`	Reads words directly into an indexed array	`read -ra words`

Practical Example: Secure Password and Confirmation Dialog


#!/usr/bin/env bash
set -euo pipefail

# 1. Ask for username
read -r -p "Enter username: " USERNAME

# 2. Ask for password silently
read -rs -p "Enter password: " PASSWORD
echo "" # Line break after silent input

# 3. Single-key confirmation (1 character without Enter)
read -n 1 -r -p "Do you want to continue? (y/n): " CONFIRMATION
echo ""

if [[ "$CONFIRMATION" =~ ^[jJyY]$ ]]; then
    echo "Executing action for user '$USERNAME'..."
else
    echo "Operation cancelled."
    exit 0
fi

Safely Reading Files Line by Line

To process log files or CSV files line by line, the combination of while and read -r is the absolute standard:


#!/usr/bin/env bash
set -euo pipefail

CSV_FILE="/tmp/users.csv"

# Create example data
cat <<'EOF' > "$CSV_FILE"
max;admin;/home/max
anna;developer;/home/anna
lisa;support;/home/lisa
EOF

# Read line by line with custom delimiter (semicolon):
while IFS=';' read -r name role home_dir || [[ -n "$name" ]]; do
    [[ -z "$name" || "$name" =~ ^# ]] && continue
    echo "User: $name | Role: $role | Directory: $home_dir"
done < "$CSV_FILE"

Exercises & Practice Check

❗ Practice tasks for lesson #5:

Task 1 (Structured Log Routing):

Write a script log_router.sh that writes normal output to /tmp/output.log and error messages to /tmp/error.log, while the user sees both outputs in color on the terminal.

Task 2 (Configuration Generator):

Create a script that asks the user for DB_NAME, DB_USER, and DB_PASS and generates a file database.env with permissions 600 (chmod 600) via a here-document.

Task 3 (Timeout Query):

Write a script that asks for confirmation with read -t 5 and automatically continues with a default value after 5 seconds if no input is provided.

Sample Solution for Task 2:


#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

read -r -p "Database Name  : " DB_NAME
read -r -p "Database User  : " DB_USER
read -rs -p "Database Pass  : " DB_PASS
echo ""

ENV_FILE="/tmp/database.env"

cat <<EOF > "$ENV_FILE"
# Automatically generated database configuration
DB_NAME="${DB_NAME}"
DB_USER="${DB_USER}"
DB_PASS="${DB_PASS}"
DB_HOST="127.0.0.1"
DB_PORT="3306"
EOF

# Restrict permissions to owner only
chmod 600 "$ENV_FILE"
echo "Configuration successfully created at $ENV_FILE (permissions: 600)."

exit 0

Command Reference (Cheatsheet)

Syntax / Command	Function & Description
`cmd > file`	Redirects stdout to file (overwrites file)
`cmd >> file`	Appends stdout to file
`cmd 2> file`	Redirects error messages (stderr) to file
`cmd &> file`	Redirects stdout AND stderr together to file
`echo "msg" >&2`	Explicitly outputs text on the error channel (stderr)
`cmd1 \| cmd2`	Connects stdout of cmd1 directly with stdin of cmd2
`cmd \| tee -a file`	Writes stdout to file and displays it on screen simultaneously
`cat <<EOF > file`	Creates file from multi-line here-document
`read -r -p "prompt" var`	Reads user input with a displayed prompt text
`read -rs var`	Reads passwords silently without screen output
`while IFS= read -r line; do`	Safely reading files line by line

Further Resources

Resource	Description
GNU Bash Redirections Manual	Complete GNU documentation on data streams and descriptors
Bash Basics #4: Functions in Bash	The previous part: Modularization, parameters, and scopes
Linux Command Line Processor Guide	Fundamental knowledge on shells, I/O streams, and pipes
chmod and File Permissions	Linux permission concepts in detail

Conclusion

Mastering input and output streams, file descriptors, and redirections is the key to seamlessly integrating your scripts into the Linux ecosystem. Through deliberate separation of stdout and stderr, using tee for gapless logging, and robust input dialogs with read -r, you make your automations transparent and user-friendly.

💡 Tip: Always explicitly send error messages in scripts to stderr (echo "Error..." >&2). This allows other admins and CI/CD pipelines to cleanly filter errors into separate log files via 2> without contaminating the regular data streams.

In the next lesson, we make our scripts indestructible and learn professional error handling, signal trapping, and debugging: 👉 Next up: Bash Basics #6: Error Handling and Debugging in Bash

👉 Course overview: All lessons of the Bash basics course