---
id: 2025-07-13-lpic-1-streams-pipes-and-redirections
slug: lpic-1-streams-pipes-and-redirections
title: "LPIC-1: streams, pipes and redirections"
excerpt: "Linux streams, pipes and redirections: standard streams, redirection, pipelines, tee and xargs for LPIC-1."
date: "2025-07-13T20:11:30+02:00"
updated: "2025-07-13T20:11:30+02:00"
author:
  name: "Sebastian Palencsár"
  handle: "spalencsar"
category: "lpic-1-serie"
tags: ["lpic-1", "lpic-1-serie", "streams", "pipes", "redirection", "shellscripting", "xargs", "teecommand"]
reading_time: 45
toc: true
---

The first four [LPIC-1 modules](/en/category/lpic-1-serie){.badge-link-text} covered the [Linux command line](/en/lpic-1-serie/lpic-1-understanding-the-linux-command-line-shell-terminal-and-first-commands){.badge-link-text}, [filesystem navigation](/en/lpic-1-serie/lpic-1-basic-navigation-and-filesystem-commands){.badge-link-text} and [viewing and editing file contents](/en/lpic-1-serie/lpic-1-viewing-and-editing-file-contents){.badge-link-text}.

Streams, pipes and redirections are the backbone of the Unix idea “**Do one thing and do it well**”. They let you join small programs into stronger workflows. Handling data streams well is what separates a competent Linux administrator from a beginner.

<blockquote class="infobox infobox--practice">
❗ **Important note:** As in the previous modules, this series does not replace an official exam-prep course for the [LPIC-1 certification (LPI 101-500)](https://www.lpi.org/our-certifications/exam-101-objectives/){.badge-link-text}. It is a practice-oriented companion for self-study.
</blockquote>

## In Linux everything is a data stream

Program input, output, errors and even device I/O travel on standard streams. That abstraction lets you connect programs, store output in files and build complex processing without each tool knowing the next.

**Advantages:**

* **Modularity:** small specialised programs combine into larger solutions
* **Flexibility:** input and output can be redirected at will
* **Efficiency:** data flows between programs without extra copies on disk
* **Automation:** workflows map cleanly into scripts
* **Debugging:** intermediate results are easy to inspect

### Place in the LPIC-1 certification

Streams, pipes and redirections sit in several exam objectives:

* **103.4:** use streams, pipes and redirections
* **103.1:** work on the command line (using pipes)
* **103.2:** process text streams with filters

Those areas are about 15–20% of LPIC-1 exam 101 and underpin many other tasks.

**In daily administration they are required for:**

* Log analysis and monitoring
* Automated backup and maintenance scripts
* Data processing and reports
* Fault diagnosis and debugging
* Configuration and tuning

### Practical relevance for day-to-day admin work

Typical jobs:

* Automated log rotation with archive and compression
* Monitoring with live alerts
* Extracting data from several sources for reports
* Batch file processing with error logs
* Debugging by redirecting output on purpose

<blockquote class="infobox infobox--info">
💡 **Practice note:** Exam hints and sysadmin examples sit throughout the module. Run the commands and pipelines in your own shell.
</blockquote>

### How to get the most from this module

Try the ideas on a real Linux system. Mix redirections and pipes, watch what happens on errors, and run the examples against your own data.

Streams and pipes only stick when you type them. The more you use the data flow, the more automatic it becomes.

## Standard streams (stdin, stdout, stderr)

The three standard streams sit under almost every input and output operation and under the chaining of programs that is the core of Unix.

### What are standard streams?

<span class="nb-accent">The three data streams on Linux</span>

Programs talk to each other and to you through standard streams. Every process gets three streams opened at start:

* **Standard input (stdin)** — input data
* **Standard output (stdout)** — normal output
* **Standard error (stderr)** — errors and diagnostics

A program does not need to know whether input comes from the keyboard, a file or another program — it just reads stdin.

🔧 **Practical example:**

```bash
# cat reads stdin and writes stdout
cat
Here I type something
Here I type something
^D  # Ctrl+D ends input
```

<span class="nb-accent">File descriptors: 0 (stdin), 1 (stdout), 2 (stderr)</span>

Internally the streams are file descriptors — numeric IDs for open files and streams:

| **Stream** | **File descriptor** | **Default target** | **Description** |
|---|---|---|---|
| **stdin** | 0 | Keyboard | Input stream |
| **stdout** | 1 | Terminal/screen | Normal program output |
| **stderr** | 2 | Terminal/screen | Errors and warnings |

Those numbers matter for advanced redirection:

```bash
# Explicit file descriptors
command 1>output.txt 2>errors.txt 0<input.txt
```

<blockquote class="infobox infobox--info">
💡 **Tip:** stdout and stderr both go to the terminal by default, but they are separate streams. You can treat normal output and errors differently.
</blockquote>

<span class="nb-accent">How the streams differ</span>

**Functional split:**

* `stdout`: the actual results, often fed into the next command
* `stderr`: metadata, warnings and errors, usually not piped onward

**Practical effect:**

```bash
# both streams hit the terminal
ls /exists /does-not-exist
ls: cannot access '/does-not-exist': No such file or directory  # stderr
/exists:  # stdout (if the directory exists)

# they can be redirected separately
ls /exists /does-not-exist >output.txt 2>errors.txt
```

### Standard input (stdin)

<span class="nb-accent">Input from the keyboard and other sources</span>

stdin is how programs receive data. By default it is the keyboard, but it can come from:

* Interactive typing
* Files (redirection)
* Another program’s output (pipes)
* Here documents and here strings

🔧 **Practical examples for stdin sources:**

**1. Interactive keyboard:**

```bash
sort
zebra
apple
banana
^D
apple
banana
zebra
```

**2. Input from a file:**

```bash
sort < unsorted_list.txt
```

**3. Input from another program:**

```bash
ls | sort
```

<span class="nb-accent">Interactive vs. non-interactive input</span>

Programs can tell whether they are attached to a terminal:

**Interactive:**

* Prompts are allowed
* Input is line by line
* Keys such as Ctrl+C and Ctrl+D work

**Non-interactive:**

* No prompts (they would pollute output)
* The whole input is processed in one go
* The program runs unattended

```bash
# Interactive — shows a prompt
mysql -u root -p
Enter password:

# Non-interactive — no prompt
mysql -u root -p < backup.sql
```

🔧 **Practical examples for using stdin**

**1. Processing data from stdin:**

```bash
# Average of numbers
cat numbers.txt | awk '{sum += $1} END {print sum/NR}'

# Filter and sort user names
cut -d: -f1 /etc/passwd | sort | head -10
```

**2. Here documents for multi-line input:**

```bash
cat << EOF > configuration.txt
# Generated configuration
ServerName example.com
DocumentRoot /var/www/html
EOF
```

**3. Batch processing:**

```bash
# Several MySQL statements
mysql -u root -p database << 'SQL'
SELECT COUNT(*) FROM users;
SHOW TABLES;
SQL
```

### Standard output (stdout)

<span class="nb-accent">Normal program output</span>

stdout is the primary result stream. Whatever the program produces as its main result should go there.

**Traits of stdout:**

* Holds the actual results
* Often piped to the next program
* Can be sent to a file without taking errors with it
* Is buffered — output can be delayed

🔧 **Typical stdout:**

```bash
ls -la                    # file list
ps aux                    # process list
cat file.txt              # file contents
grep "pattern" *.txt      # search hits
awk '{print $1}' data.csv # extracted fields
```

<span class="nb-accent">stdout vs. stderr</span>

**stdout should hold:**

* Main results
* Data meant for further processing
* Structured output

**stderr should hold:**

* Errors
* Warnings
* Progress
* Debug output

```bash
# Correct split
find /etc -name "*.conf" 2>/dev/null | head -5
# hits on stdout, errors on stderr

# Poor mix: everything on one stream
some_bad_program 2>&1 | grep -v "WARNING"
```

🔧 **Typical admin uses**

**1. Data for reports:**

```bash
ps aux | awk '{print $3}' | awk '{sum += $1} END {print "Average CPU:", sum/NR "%"}'
grep "ERROR" /var/log/application.log | cut -d' ' -f1-3 > error_timestamps.txt
```

**2. Monitoring:**

```bash
free -h | grep "Mem:" | awk '{print $3 "/" $2}' > memory_usage.txt
netstat -an | grep ESTABLISHED | wc -l > active_connections.txt
```

**3. Automated reports:**

```bash
{
    echo "=== System report $(date) ==="
    echo "Disk use:"
    df -h | grep -v tmpfs
    echo "Top 5 processes:"
    ps aux | sort -k3 -nr | head -5
} > daily_report.txt
```

### Standard error (stderr)

<span class="nb-accent">Errors and diagnostics</span>

stderr is dedicated to errors, warnings and diagnostics. Splitting it from normal output is a core Unix design.

**What belongs on stderr:**

* Errors
* Warnings
* Progress on long jobs
* Debug information
* Usage hints

🔧 **stderr examples:**

```bash
cat missing_file.txt
cat: missing_file.txt: No such file or directory

cp file.txt /read-only-directory/
cp: cannot create regular file '/read-only-directory/file.txt': Permission denied

rsync -av --progress source/ destination/
# progress on stderr, file list on stdout
```

<span class="nb-accent">Why stderr is separate from stdout</span>

**1. Clean data processing:**

```bash
find /etc -name "*.conf" 2>/dev/null | wc -l
# errors do not pollute the count
```

**2. Separate logs:**

```bash
backup_script.sh >backup.log 2>backup_errors.log
```

**3. Usability:**

```bash
long_running_command >results.txt
# errors still appear on the terminal
```

**4. Script robustness:**

```bash
if ! command >output.txt 2>errors.txt; then
    echo "Command failed, see errors.txt"
    exit 1
fi
```

🔧 **Practical meaning for logging and debugging**

**1. Structured logging:**

```bash
log_info() {
    echo "INFO: $1" >&2
}

log_error() {
    echo "ERROR: $1" >&2
}

process_data() {
    log_info "Starting data processing..."

    if [ ! -f "$1" ]; then
        log_error "File $1 not found"
        return 1
    fi

    log_info "Processing $1..."
    cat "$1"
    log_info "Processing finished"
}
```

**2. Debugging without polluting stdout:**

```bash
debug() {
    [ "$DEBUG" = "1" ] && echo "DEBUG: $1" >&2
}

DEBUG=1 ./my_script.sh >data.txt
# debug on the terminal, data in the file
```

**3. Errors in pipelines:**

```bash
command1 2>errors1.log | command2 2>errors2.log | command3 2>errors3.log >final_output.txt
(command1 | command2 | command3 >output.txt) 2>all_errors.log
```

<blockquote class="infobox infobox--info">
💡 **Practice tip:** In your own scripts, split stdout and stderr consistently. The scripts become easier to reuse.
</blockquote>

<blockquote class="infobox infobox--warn">
⚠️ **Note:** Some older programs dump everything on stdout. You may need extra filtering.
</blockquote>

<blockquote class="infobox infobox--practice">
❗ **Typical mistakes:**
</blockquote>

**Mixing data and messages:**

```bash
# BAD: messages on stdout
echo "Processing file filename"
cat "filename"

# BETTER: messages on stderr
echo "Processing file filename" >&2
cat "filename"
```

**Ignoring stderr:**

```bash
# DANGEROUS: errors are swallowed
result=$(command 2>/dev/null)

# BETTER: handle errors
if ! result=$(command 2>error_log); then
    echo "Command failed, see error_log" >&2
    exit 1
fi
```

## Redirection with > and >>

Redirection lets the shell send streams to files instead of the terminal, or take input from files instead of the keyboard. That is essential for automation, logging and batch work.

### Basics of output redirection

<span class="nb-accent">The idea of redirection</span>

The shell interprets operators (`>`, `>>`, `<`, …), opens the files, then starts the program with those streams. The program itself does not know about the redirect.

🔧 **Simple example:**

```bash
echo "Hello World"
Hello World

echo "Hello World" > greeting.txt
cat greeting.txt
Hello World
```

<span class="nb-accent">Syntax</span>

```bash
[n]operator[file]
```

* `n` is the file descriptor (optional; default 1 for `>`)
* `operator` is `>`, `>>`, `<`, …
* `file` is the target or source

| **Operator** | **Description** | **Example** |
|---|---|---|
| `>` | stdout, overwrite | `command > file.txt` |
| `>>` | stdout, append | `command >> file.txt` |
| `<` | stdin from file | `command < input.txt` |
| `2>` | stderr | `command 2> errors.txt` |
| `2>>` | stderr, append | `command 2>> errors.txt` |
| `&>` | stdout and stderr | `command &> all_output.txt` |

### Simple redirection with >

<span class="nb-accent">Sending stdout to files</span>

`>` creates the file if missing and overwrites it if it exists.

🔧 **Practical examples:**

**1. Collect system stats:**

```bash
ps aux > processes_$(date +%Y%m%d).txt
df -h > disk_status.txt
ip addr show > network_config.txt
```

**2. Write configuration:**

```bash
echo "ServerName example.com" > apache_config.txt
echo "DocumentRoot /var/www/html" >> apache_config.txt

cat > ~/.ssh/config << EOF
Host production
    HostName prod.example.com
    User admin
    Port 2222
EOF
```

**3. Reports:**

```bash
{
    echo "=== System report $(date) ==="
    echo "Uptime:"
    uptime
    echo "Memory:"
    free -h
    echo "Disk:"
    df -h
} > daily_report_$(date +%Y%m%d).txt
```

<span class="nb-accent">Overwrite vs. create</span>

```bash
echo "First line" > new_file.txt
echo "Second line" > new_file.txt
cat new_file.txt
Second line  # the first line is gone
```

<blockquote class="infobox infobox--warn">
⚠️ **Warning:** `>` overwrites without asking. Previous content is lost.
</blockquote>

**Guard against accidental overwrite:**

```bash
set -o noclobber
echo "Test" > existing_file.txt
bash: existing_file.txt: cannot overwrite existing file

echo "Test" >| existing_file.txt
set +o noclobber
```

🔧 **Admin examples**

**1. Log rotation:**

```bash
LOGFILE="/var/log/application.log"
ARCHIVE_DIR="/var/log/archive"

if [ -f "$LOGFILE" ]; then
    mv "$LOGFILE" "$ARCHIVE_DIR/application_$(date +%Y%m%d_%H%M%S).log"
fi

touch "$LOGFILE"
echo "Log rotated on $(date)" > "$LOGFILE"
```

**2. Back up configuration:**

```bash
tar -czf config_backup_$(date +%Y%m%d).tar.gz /etc/apache2/ /etc/nginx/ /etc/ssh/ 2> backup_errors.log
```

### Appending with `>>`

<span class="nb-accent">Difference from simple redirection</span>

`>>` appends. If the file is missing, it is created.

| **Operator** | **Existing file** | **New file** | **Use** |
|---|---|---|---|
| `>` | Overwrites | Creates | One-shot reports |
| `>>` | Appends | Creates | Logs, continuous collection |

🔧 **Demo:**

```bash
echo "Line 1" > testfile.txt
echo "Line 2" >> testfile.txt
echo "Line 3" >> testfile.txt
```

<span class="nb-accent">Logging use cases</span>

```bash
while true; do
    echo "$(date): CPU: $(top -bn1 | grep "Cpu(s)" | awk '{print $2}')" >> system_monitor.log
    sleep 60
done
```

```bash
LOGFILE="/var/log/my_script.log"
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOGFILE"
}
log_message "Script started"
```

### Avoiding data loss

```bash
CURRENT_LOG="/var/log/application.log"
ARCHIVE_LOG="/var/log/archive/application_$(date +%Y%m%d_%H%M%S).log"
cp "$CURRENT_LOG" "$ARCHIVE_LOG"
> "$CURRENT_LOG"
echo "$(date): log rotated, archive: $(basename "$ARCHIVE_LOG")" >> "$CURRENT_LOG"
```

### Redirecting stderr

```bash
command 2> error_file.txt
command 2>> error_log.txt
command > output.txt 2> errors.txt
```

🔧 **Practical examples:**

```bash
find /etc -name "*.conf" 2>/dev/null > config_files.txt
gcc -o program source.c 2> compile_errors.txt
rsync -av /home/ /backup/ > backup_success.log 2> backup_errors.log
command 2>/dev/null
```

<span class="nb-accent">Combined stdout and stderr</span>

```bash
command > output.txt 2>&1
command &> output.txt
command >> output.txt 2>&1
command &>> output.txt
```

<blockquote class="infobox infobox--warn">
⚠️ **Important:** Order of `2>&1` matters:
</blockquote>

```bash
# RIGHT: redirect stdout first, then send stderr where stdout now goes
command > file.txt 2>&1

# WRONG: stderr still points at the original stdout (the terminal)
command 2>&1 > file.txt
```

🔧 **Demo:**

```bash
ls /exists /does-not-exist > test1.txt 2>&1
# both streams in the file

ls /exists /does-not-exist 2>&1 > test2.txt
# error still on the terminal; only stdout in the file
```

### Input redirection with <

```bash
sort < unsorted_list.txt
awk -F',' '{print $1, $3}' < data.csv
mysql -u root -p database_name < backup.sql
```

<span class="nb-accent">Here documents and here strings</span>

```bash
command << DELIMITER
Line 1
Line 2
DELIMITER
```

🔧 **Practical uses:**

```bash
cat << 'EOF' > /etc/apache2/sites-available/example.conf
<VirtualHost *:80>
    ServerName example.com
    DocumentRoot /var/www/example
</VirtualHost>
EOF
```

```bash
mysql -u root -p << 'SQL'
CREATE DATABASE IF NOT EXISTS myapp;
SQL
```

```bash
tr '[:lower:]' '[:upper:]' <<< "hello world"
HELLO WORLD
```

<blockquote class="infobox infobox--info">
💡 **Practice tip:** Use input redirection when you want the shell to open the file, or when the program does not take a filename argument.
</blockquote>

<blockquote class="infobox infobox--practice">
❗ **Typical mistakes:**
</blockquote>

```bash
# WRONG direction
echo "test" < output.txt

# RIGHT
echo "test" > output.txt
```

```bash
# Variables expand
cat << EOF
Current user: $USER
EOF

# Quoted delimiter: no expansion
cat << 'EOF'
Current user: $USER
EOF
```

## Using pipes

Pipes join simple programs into stronger chains without temporary files. `|` connects one command’s stdout to the next command’s stdin.

### What pipes are and how they work

<span class="nb-accent">Connecting commands</span>

```markdown
┌─────────────────────────────────────────────────────────────┐
│                   UNIX PIPELINE DATA FLOW                   │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│   [ Program 1 ]        [ Program 2 ]        [ Program 3 ]   │
│         │                    │                    │         │
│      stdout               stdout               stdout       │
│         │                    │                    │         │
│         ├───► (Pipe: |) ────►│                    │         │
│               stdin          ├───► (Pipe: |) ────►│         │
│                                    stdin          │         │
│                                                   ▼         │
│                                            stdout (terminal)│
│                                                             │
└─────────────────────────────────────────────────────────────┘
```

The shell starts every program in the pipeline at once and connects their streams. Data flows in real time; nothing has to hit the disk.

🔧 **Simple example:**

```bash
# Without a pipe: clumsy temp files
ps aux > temp_processes.txt
grep apache temp_processes.txt > apache_processes.txt
wc -l apache_processes.txt
rm temp_processes.txt apache_processes.txt

# With a pipe
ps aux | grep apache | wc -l
```

### Data flow between processes

Pipes use a kernel buffer (typically 64KB). A full buffer blocks the writer; an empty buffer blocks the reader.

```bash
# Slow producer, fast consumer
find / -name "*.log" 2>/dev/null | head -10
# find may keep running after head exits

# Fast producer, slow consumer
cat large_file.txt | sleep 10
# cat waits until the reader drains the pipe
```

<blockquote class="infobox infobox--info">
💡 **Tip:** Pipes are lazy — they only process as much data as the downstream command needs. That is efficient on large files.
</blockquote>

<span class="nb-accent">Pipes vs. file redirection</span>

| **Property** | **Pipes** | **File redirection** |
|---|---|---|
| **Storage** | RAM | Disk |
| **Speed** | Very fast | Slower (I/O) |
| **Persistence** | Temporary | Permanent |
| **Parallelism** | Concurrent | Sequential |
| **Disk use** | Minimal | Can be large |
| **Debugging** | Harder | Easier (intermediate files) |

🔧 **Comparison:**

```bash
grep "ERROR" /var/log/huge.log > temp1.txt
sort temp1.txt > temp2.txt
uniq temp2.txt > temp3.txt
wc -l temp3.txt
rm temp1.txt temp2.txt temp3.txt

grep "ERROR" /var/log/huge.log | sort | uniq | wc -l
```

### Basic pipe syntax

```bash
command1 | command2
```

🔧 **Common patterns:**

```bash
ps aux | grep apache | grep -v grep | wc -l
who | wc -l
ls -1 | wc -l
cut -d: -f7 /etc/passwd | sort | uniq
history | awk '{print $2}' | sort | uniq -c | sort -nr | head -10
ps aux | awk '{print $2, $3, $4, $11}' | column -t
du -h /var/log/* | sort -hr
```

<span class="nb-accent">Multiple pipes</span>

🔧 **Practical chains:**

```bash
cat /var/log/apache2/access.log | \
  awk '{print $1}' | \
  sort | \
  uniq -c | \
  sort -nr | \
  head -10
```

```bash
ps aux | awk 'NR>1 {print $4, $11}' | sort -nr | head -5 | column -t
```

```bash
netstat -an | grep ESTABLISHED | awk '{print $4}' | sed 's/.*://' | sort -n | uniq -c | sort -nr
```

### Advanced pipe uses

Filter early for speed:

```bash
# BAD: sort the whole file first
cat huge_file.txt | sort | uniq | grep "pattern" | head -10

# BETTER: filter first
grep "pattern" huge_file.txt | sort | uniq | head -10
```

**Debugging a chain:**

```bash
ps aux
ps aux | grep apache
ps aux | grep apache | grep -v grep
ps aux | grep apache | grep -v grep | wc -l
ps aux | tee debug1.txt | grep apache | tee debug2.txt | wc -l
```

```bash
command1 | command2 | command3
echo "Pipeline status: ${PIPESTATUS[@]}"
set -o pipefail
command1 | command2 | command3 || echo "Pipeline failed"
```

### Named pipes (FIFOs)

<span class="nb-accent">Concept and mkfifo</span>

Named pipes persist as special files and let unrelated processes talk.

```bash
mkfifo my_pipe
ls -l my_pipe
prw-rw-r-- 1 user user 0 Jul 13 10:14 my_pipe
```

```bash
# Terminal 1
echo "Hello world" > my_pipe

# Terminal 2
cat < my_pipe
Hello world
```

🔧 **Logging collector:**

```bash
mkfifo /tmp/app_logs
echo "App1: $(date) - Startup complete" > /tmp/app_logs &
while read log_entry; do
    echo "$log_entry" >> /var/log/applications.log
done < /tmp/app_logs
```

| **Property** | **Anonymous pipes** | **Named pipes (FIFOs)** |
|---|---|---|
| **Lifetime** | Only while the pipeline runs | Persist in the filesystem |
| **Access** | Related processes | Any processes |
| **Visibility** | Invisible | Visible as a file |
| **Cleanup** | Automatic | Delete by hand |
| **Use** | Command chaining | IPC |
| **Performance** | Very fast | Slightly slower |

<blockquote class="infobox infobox--warn">
⚠️ **Named-pipe notes:** Reads and writes block until both ends are connected. Delete FIFOs when you are done. Permissions work like normal files.
</blockquote>

<blockquote class="infobox infobox--info">
💡 **Practice tip:** Named pipes fit microservice IPC, log aggregation, parallel build workers and alerting.
</blockquote>

<blockquote class="infobox infobox--practice">
❗ **Typical mistakes:** Deadlocks if both sides wait. Always `rm` the FIFO, or use `trap 'rm -f /tmp/my_pipe' EXIT`. Create with `mkfifo -m 660 /tmp/secure_pipe` when you need tighter rights. Prefer `timeout 5 cat my_pipe` over hanging forever.
</blockquote>

## Redirection combinations

### Combined stdout/stderr redirection

| **Syntax** | **Description** | **Bash** | **Example** |
|---|---|---|---|
| `command > file 2>&1` | POSIX classic | All | `ls /tmp > output.txt 2>&1` |
| `command &> file` | Short form | Bash 4.0+ | `ls /tmp &> output.txt` |
| `command >& file` | Alternate short form | Bash/csh | `ls /tmp >& output.txt` |
| `command >> file 2>&1` | Append, classic | All | `ls /tmp >> output.txt 2>&1` |
| `command &>> file` | Append, modern | Bash 4.0+ | `ls /tmp &>> output.txt` |

🔧 **Practical scenarios:**

```bash
./install_script.sh > install.log 2>&1
./install_script.sh &> install.log
./monitoring_script.sh &>> system_monitor.log
find /var -name "*.log" -exec grep "ERROR" {} \; &> debug_output.txt
rsync -av /home/ /backup/home/ &> backup_$(date +%Y%m%d_%H%M%S).log
```

### Combining pipes and redirections

```bash
{
    grep "ERROR" /var/log/application.log | \
    awk '{print $1, $2, $NF}' | \
    sort | uniq -c | sort -nr
} > error_analysis.txt 2> analysis_errors.log
```

```bash
ps aux | \
  tee process_list.debug | \
  grep apache | \
  tee apache_processes.debug | \
  awk '{sum += $3} END {print "Total CPU:", sum "%"}'
```

### Advanced redirection techniques

```bash
exec 3> logfile.txt
echo "Message 1" >&3
echo "Message 2" >&3
exec 3>&-
```

🔧 **Practical applications:**

```bash
exec 3> /var/log/debug.log
exec 4> /var/log/errors.log
exec 5> /var/log/audit.log

log_debug() { echo "$(date '+%Y-%m-%d %H:%M:%S') DEBUG: $*" >&3; }
log_error() { echo "$(date '+%Y-%m-%d %H:%M:%S') ERROR: $*" >&4; echo "$(date '+%Y-%m-%d %H:%M:%S') ERROR: $*" >&2; }
log_audit() { echo "$(date '+%Y-%m-%d %H:%M:%S') AUDIT: $*" >&5; }

exec 3>&- 4>&- 5>&-
```

```bash
exec 6>&1 7>&2
exec 1> temp_output.log 2> temp_errors.log
echo "This goes to temp_output.log"
exec 1>&6 2>&7
exec 6>&- 7>&-
echo "This appears on the terminal"
```

### Redirecting to /dev/null

```bash
find /etc -name "*.conf" 2>/dev/null
command > /dev/null
command &> /dev/null
command < /dev/null
```

🔧 **Practical scenarios:**

```bash
if systemctl is-active apache2 &> /dev/null; then
    echo "Apache is running"
else
    echo "Apache is stopped"
fi

for file in *.txt; do
    process_file "$file" > /dev/null || echo "Error on $file"
done
```

### Combining pipes with redirection

<span class="nb-accent">Complex processing pipelines</span>

Pipes plus redirections give you staged workflows with error files and intermediate `tee` snapshots:

```bash
{
    grep "ERROR" /var/log/application.log | \
    awk '{print $1, $2, $NF}' | \
    sort | uniq -c | sort -nr
} > error_analysis.txt 2> analysis_errors.log
```

🔧 **Practical example:**

```bash
ps aux | \
  tee process_list.debug | \
  grep apache | \
  tee apache_processes.debug | \
  awk '{sum += $3} END {print "Total CPU:", sum "%"}' | \
  tee cpu_summary.debug
```

### Error handling in combined operations

🔧 **Practical example:**

```bash
set -o pipefail
command1 | command2 | command3 || echo "Pipeline failed"
```

### Practical scenarios for system administrators

<span class="nb-accent">Log rotation and archiving</span>

🔧 **Practical example:**

```bash
LOGFILE="/var/log/application.log"
ARCHIVE_DIR="/var/log/archive"
cp "$LOGFILE" "$ARCHIVE_DIR/application_$(date +%Y%m%d_%H%M%S).log"
> "$LOGFILE"
find "$ARCHIVE_DIR" -name "*.log_*" -mtime +7 ! -name "*.gz" -exec gzip {} \;
```

<span class="nb-accent">Backups with logging</span>

🔧 **Practical example:**

```bash
rsync -av --progress /home/ /backup/home/ > /var/log/backup_success.log 2> /var/log/backup_errors.log
```

<span class="nb-accent">Monitoring and alerting</span>

CPU, memory and disk checks that `tee` into a monitor log and append alerts.

<blockquote class="infobox infobox--info">
💡 **Practice tip:** Combine redirection techniques step by step. Start simple, then grow the workflow.
</blockquote>

<blockquote class="infobox infobox--warn">
⚠️ **Important:** On complex combinations, understand operator order and test it. Use `set -x` when debugging.
</blockquote>

<blockquote class="infobox infobox--practice">
❗ **Typical mistakes:** Wrong order of combined redirects; forgetting to close extra file descriptors; race conditions in parallel work; unhandled errors in long pipelines.
</blockquote>

## tee and xargs

`tee` splits a stream to stdout and to files. `xargs` turns stdin into command arguments for tools that do not read a pipe.

### The tee command

<span class="nb-accent">How it splits a stream</span>

```markdown
┌─────────────────────────────────────────────────────────────┐
│                   TEE COMMAND: T-JUNCTION                   │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│                       Standard Input                        │
│                             │                               │
│                             ▼                               │
│                      ┌─────────────┐                        │
│                      │  tee command│                        │
│                      └──────┬──────┘                        │
│                             │                               │
│              ┌──────────────┴──────────────┐                │
│              ▼                             ▼                │
│      Standard Output                  File(s)               │
│   (terminal / pipe)                (log / backup)           │
│                                                             │
└─────────────────────────────────────────────────────────────┘
```

🔧 **Simple example:**

```bash
ps aux > processes.txt   # save, but you do not see it
ps aux                   # see it, but do not save
ps aux | tee processes.txt
```

| **Option** | **Description** | **Example** |
|---|---|---|
| `-a` | Append | `command | tee -a logfile.txt` |
| `-i` | Ignore interrupt signals | `command | tee -i output.txt` |
| (none) | Overwrite | `command | tee output.txt` |

🔧 **Practical examples:**

```bash
df -h | tee disk_usage_human.txt | awk '{print $1, $5}' > disk_usage_simple.txt

cat /var/log/apache2/access.log | \
  tee step1_raw.log | \
  grep "404" | \
  tee step2_404s.log | \
  awk '{print $1}' | \
  sort | uniq -c | sort -nr
```

<span class="nb-accent">Using tee with sudo</span>

```bash
# FAILS: sudo applies to echo, not the redirect
echo "new line" | sudo > /etc/hosts

# WORKS: tee runs as root
echo "127.0.0.1 test.local" | sudo tee -a /etc/hosts
echo "127.0.0.1 test.local" | sudo tee -a /etc/hosts > /dev/null
```

🔧 **Privileged writes:**

```bash
cat << 'EOF' | sudo tee /etc/apache2/sites-available/example.conf > /dev/null
<VirtualHost *:80>
    ServerName example.com
    DocumentRoot /var/www/example
</VirtualHost>
EOF

echo "0 2 * * * /usr/local/bin/backup.sh" | sudo tee -a /etc/crontab
echo "vm.swappiness=10" | sudo tee -a /etc/sysctl.conf
```

### The xargs command

<span class="nb-accent">The problem: commands that do not read a pipe</span>

```bash
find /tmp -name "*.tmp" | rm
rm: missing operand

ls *.txt | cp /backup/
cp: missing destination file operand
```

`xargs` reads stdin and builds argument lists:

```bash
stdin → xargs → command arg1 arg2 arg3 ...
```

🔧 **Simple examples:**

```bash
find /tmp -name "*.tmp" | xargs rm
ls *.txt | xargs -I {} cp {} /backup/
find /var/www -name "*.php" | xargs chmod 644
```

| **Option** | **Description** | **Example** |
|---|---|---|
| `-n NUM` | At most NUM arguments per call | `echo "1 2 3 4" | xargs -n 2 echo` |
| `-I REPLACE` | Replace the placeholder | `ls *.txt | xargs -I {} cp {} backup/` |
| `-0` | NUL-terminated input | `find . -name "*.txt" -print0 | xargs -0 rm` |
| `-P NUM` | Run NUM processes in parallel | `ls *.jpg | xargs -P 4 -I {} convert {} {}.png` |
| `-t` | Print the command before running it | `find /tmp -name "*.tmp" | xargs -t rm` |
| `-r` | Do nothing if stdin is empty | `find /tmp -name "*.tmp" | xargs -r rm` |

🔧 **Option details:**

```bash
echo "file1 file2 file3 file4" | xargs -n 2 echo "Processing:"
find . -name "*.txt" -print0 | xargs -0 rm
ls *.jpg | xargs -P 4 -I {} convert {} {}.png
```

<span class="nb-accent">Safety and special characters</span>

```bash
# DANGEROUS with spaces
echo "important file.txt" | xargs rm

# SAFE
find . -name "*.txt" -print0 | xargs -0 rm
find . -name "*.txt" | xargs -I {} rm "{}"
```

### Advanced tee uses

🔧 **Practical example:**

```bash
ps aux | tee processes_backup1.txt processes_backup2.txt processes_current.txt
```

### Monitoring and logging scenarios

```bash
{
    while true; do
        timestamp=$(date '+%Y-%m-%d %H:%M:%S')
        cpu_usage=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | sed 's/%us,//')
        echo "$timestamp CPU:${cpu_usage}%"
        sleep 5
    done
} | tee -a /var/log/realtime_monitoring/realtime_$(date +%Y%m%d).log
```

### Advanced xargs uses

```bash
find /tmp -type f -mtime +7 -print0 | xargs -0 -r rm -v
find /var/www/html -type d -print0 | xargs -0 chmod 755
find /var/log -name "*.log" -type f -mtime +7 ! -name "*.gz" -print0 | xargs -0 -r -P 2 gzip -v
```

### Combining tee and xargs

🔧 **Practical example:**

```bash
find /var/log -name "*.log" -print0 | \
  xargs -0 grep -l "ERROR" | \
  tee error_logs.txt | \
  xargs -r wc -l
```

<blockquote class="infobox infobox--info">
💡 **Practice tip:** `tee` plus `xargs` is strong when you need both intermediate files and parallel work.
</blockquote>

<blockquote class="infobox infobox--warn">
⚠️ **Important:** With `xargs -P`, watch system load. Start with few parallel jobs and raise the count step by step.
</blockquote>

### Why these ideas matter for LPIC-1

<blockquote class="infobox infobox--info">
💡 **Exam weight:** On LPIC-1 exam 101, streams, pipes and redirections are about 15–20% of the marks, especially objectives 103.4 and 103.1.
</blockquote>

They underpin log analysis, backup scripts, data pipelines and professional error handling. They also show that you understand the Unix model: small tools, joined cleanly.

<blockquote class="infobox infobox--info">
💡 **Note:** Further exam information is in [LPIC-1: basic navigation and filesystem commands](/en/lpic-1-serie/lpic-1-basic-navigation-and-filesystem-commands){.badge-link-text}.
</blockquote>

## Command Reference (Cheatsheet)

| Operator / command | Syntax example | Description and LPIC-1 relevance |
|---|---|---|
| `>` | `command > file.txt` | Redirect `stdout` (overwrite) |
| `>>` | `command >> file.txt` | Append `stdout` |
| `<` | `command < input.txt` | File as `stdin` |
| `2>` | `command 2> error.log` | Redirect `stderr` (FD 2) only |
| `2>&1` | `command > all.log 2>&1` | Duplicate `stderr` onto `stdout` |
| `&>` | `command &> all.log` | Bash short form for both streams |
| `\|` | `command1 \| command2` | Pipe `stdout` of command1 into `stdin` of command2 |
| `<< EOF` | `cat << EOF > file` | Here-document until the delimiter |
| `<<<` | `grep "word" <<< "$VAR"` | Here-string into `stdin` |
| `<()` | `diff <(cmd1) <(cmd2)` | Process substitution |
| `tee` | `cmd \| tee -a log.txt` | Copy `stdin` to `stdout` and a file (`-a` append) |
| `xargs` | `find . -print0 \| xargs -0 rm` | Build arguments from `stdin` (`-n`, `-I`, `-0`, `-P`) |
| `mkfifo` | `mkfifo /tmp/named_pipe` | Create a named FIFO |
| `exec` | `exec 3> file.log` | Open or reshape file descriptors for the shell |

## Further Resources

| Resource | Description |
|---|---|
| [GNU Bash Reference Manual: Redirections](https://www.gnu.org/software/bash/manual/html_node/Redirections.html){.badge-link-text} | Official docs on redirections, file descriptors and here-docs |
| [GNU Coreutils: tee invocation](https://www.gnu.org/software/coreutils/manual/html_node/tee-invocation.html){.badge-link-text} | GNU manual for `tee` |
| [GNU Findutils: xargs manual](https://www.gnu.org/software/findutils/manual/html_node/find_html/xargs-options.html){.badge-link-text} | `xargs` options and parallel runs |
| [LPI: LPIC-1 Exam 101 Objectives](https://www.lpi.org/our-certifications/exam-101-objectives/){.badge-link-text} | Official objectives for topic 103.4 |

## Conclusion

Streams, pipes and redirections are the core of the Linux command line and of the Unix model: small tools joined on standard channels into efficient pipelines. Simple `>`, parallel `xargs` and splitting with `tee` are daily admin skills.

<blockquote class="infobox infobox--info">
💡 **Practice tip:** On complex pipelines, check intermediate results with `tee` or process substitution `<()` without breaking the flow.
</blockquote>

The next LPIC-1 module covers archiving and compression: [LPIC-1: archiving and compressing files](/en/lpic-1-serie/lpic-1-archiving-and-compressing-files){.badge-link-text} — `tar`, `gzip`, `bzip2`, `xz` and backup pipelines.

**Course overview:** [All LPIC-1 articles and modules](/en/category/lpic-1-serie){.badge-link-text}