LPIC-1: streams, pipes and redirections

The first four LPIC-1 modules covered the Linux command line, filesystem navigation and viewing and editing file contents.

Streams, pipes and redirections are the backbone of the Unix idea “Do one thing and do it well”. They let you join small programs into stronger workflows. Handling data streams well is what separates a competent Linux administrator from a beginner.

❗ Important note: As in the previous modules, this series does not replace an official exam-prep course for the LPIC-1 certification (LPI 101-500). It is a practice-oriented companion for self-study.

In Linux everything is a data stream

Program input, output, errors and even device I/O travel on standard streams. That abstraction lets you connect programs, store output in files and build complex processing without each tool knowing the next.

Advantages:

Modularity: small specialised programs combine into larger solutions
Flexibility: input and output can be redirected at will
Efficiency: data flows between programs without extra copies on disk
Automation: workflows map cleanly into scripts
Debugging: intermediate results are easy to inspect

Place in the LPIC-1 certification

Streams, pipes and redirections sit in several exam objectives:

103.4: use streams, pipes and redirections
103.1: work on the command line (using pipes)
103.2: process text streams with filters

Those areas are about 15–20% of LPIC-1 exam 101 and underpin many other tasks.

In daily administration they are required for:

Log analysis and monitoring
Automated backup and maintenance scripts
Data processing and reports
Fault diagnosis and debugging
Configuration and tuning

Practical relevance for day-to-day admin work

Typical jobs:

Automated log rotation with archive and compression
Monitoring with live alerts
Extracting data from several sources for reports
Batch file processing with error logs
Debugging by redirecting output on purpose

💡 Practice note: Exam hints and sysadmin examples sit throughout the module. Run the commands and pipelines in your own shell.

How to get the most from this module

Try the ideas on a real Linux system. Mix redirections and pipes, watch what happens on errors, and run the examples against your own data.

Streams and pipes only stick when you type them. The more you use the data flow, the more automatic it becomes.

Standard streams (stdin, stdout, stderr)

The three standard streams sit under almost every input and output operation and under the chaining of programs that is the core of Unix.

What are standard streams?

The three data streams on Linux

Programs talk to each other and to you through standard streams. Every process gets three streams opened at start:

Standard input (stdin) — input data
Standard output (stdout) — normal output
Standard error (stderr) — errors and diagnostics

A program does not need to know whether input comes from the keyboard, a file or another program — it just reads stdin.

🔧 Practical example:


# cat reads stdin and writes stdout
cat
Here I type something
Here I type something
^D  # Ctrl+D ends input

File descriptors: 0 (stdin), 1 (stdout), 2 (stderr)

Internally the streams are file descriptors — numeric IDs for open files and streams:

Stream	File descriptor	Default target	Description
stdin	0	Keyboard	Input stream
stdout	1	Terminal/screen	Normal program output
stderr	2	Terminal/screen	Errors and warnings

Those numbers matter for advanced redirection:


# Explicit file descriptors
command 1>output.txt 2>errors.txt 0<input.txt

💡 Tip: stdout and stderr both go to the terminal by default, but they are separate streams. You can treat normal output and errors differently.

How the streams differ

Functional split:

stdout: the actual results, often fed into the next command
stderr: metadata, warnings and errors, usually not piped onward

Practical effect:


# both streams hit the terminal
ls /exists /does-not-exist
ls: cannot access '/does-not-exist': No such file or directory  # stderr
/exists:  # stdout (if the directory exists)

# they can be redirected separately
ls /exists /does-not-exist >output.txt 2>errors.txt

Standard input (stdin)

Input from the keyboard and other sources

stdin is how programs receive data. By default it is the keyboard, but it can come from:

Interactive typing
Files (redirection)
Another program’s output (pipes)
Here documents and here strings

🔧 Practical examples for stdin sources:

1. Interactive keyboard:


sort
zebra
apple
banana
^D
apple
banana
zebra

2. Input from a file:


sort < unsorted_list.txt

3. Input from another program:


ls | sort

Interactive vs. non-interactive input

Programs can tell whether they are attached to a terminal:

Interactive:

Prompts are allowed
Input is line by line
Keys such as Ctrl+C and Ctrl+D work

Non-interactive:

No prompts (they would pollute output)
The whole input is processed in one go
The program runs unattended


# Interactive — shows a prompt
mysql -u root -p
Enter password:

# Non-interactive — no prompt
mysql -u root -p < backup.sql

🔧 Practical examples for using stdin

1. Processing data from stdin:


# Average of numbers
cat numbers.txt | awk '{sum += $1} END {print sum/NR}'

# Filter and sort user names
cut -d: -f1 /etc/passwd | sort | head -10

2. Here documents for multi-line input:


cat << EOF > configuration.txt
# Generated configuration
ServerName example.com
DocumentRoot /var/www/html
EOF

3. Batch processing:


# Several MySQL statements
mysql -u root -p database << 'SQL'
SELECT COUNT(*) FROM users;
SHOW TABLES;
SQL

Standard output (stdout)

Normal program output

stdout is the primary result stream. Whatever the program produces as its main result should go there.

Traits of stdout:

Holds the actual results
Often piped to the next program
Can be sent to a file without taking errors with it
Is buffered — output can be delayed

🔧 Typical stdout:


ls -la                    # file list
ps aux                    # process list
cat file.txt              # file contents
grep "pattern" *.txt      # search hits
awk '{print $1}' data.csv # extracted fields

stdout vs. stderr

stdout should hold:

Main results
Data meant for further processing
Structured output

stderr should hold:

Errors
Warnings
Progress
Debug output


# Correct split
find /etc -name "*.conf" 2>/dev/null | head -5
# hits on stdout, errors on stderr

# Poor mix: everything on one stream
some_bad_program 2>&1 | grep -v "WARNING"

🔧 Typical admin uses

1. Data for reports:


ps aux | awk '{print $3}' | awk '{sum += $1} END {print "Average CPU:", sum/NR "%"}'
grep "ERROR" /var/log/application.log | cut -d' ' -f1-3 > error_timestamps.txt

2. Monitoring:


free -h | grep "Mem:" | awk '{print $3 "/" $2}' > memory_usage.txt
netstat -an | grep ESTABLISHED | wc -l > active_connections.txt

3. Automated reports:


{
    echo "=== System report $(date) ==="
    echo "Disk use:"
    df -h | grep -v tmpfs
    echo "Top 5 processes:"
    ps aux | sort -k3 -nr | head -5
} > daily_report.txt

Standard error (stderr)

Errors and diagnostics

stderr is dedicated to errors, warnings and diagnostics. Splitting it from normal output is a core Unix design.

What belongs on stderr:

Errors
Warnings
Progress on long jobs
Debug information
Usage hints

🔧 stderr examples:


cat missing_file.txt
cat: missing_file.txt: No such file or directory

cp file.txt /read-only-directory/
cp: cannot create regular file '/read-only-directory/file.txt': Permission denied

rsync -av --progress source/ destination/
# progress on stderr, file list on stdout

Why stderr is separate from stdout

1. Clean data processing:


find /etc -name "*.conf" 2>/dev/null | wc -l
# errors do not pollute the count

2. Separate logs:


backup_script.sh >backup.log 2>backup_errors.log

3. Usability:


long_running_command >results.txt
# errors still appear on the terminal

4. Script robustness:


if ! command >output.txt 2>errors.txt; then
    echo "Command failed, see errors.txt"
    exit 1
fi

🔧 Practical meaning for logging and debugging

1. Structured logging:


log_info() {
    echo "INFO: $1" >&2
}

log_error() {
    echo "ERROR: $1" >&2
}

process_data() {
    log_info "Starting data processing..."

    if [ ! -f "$1" ]; then
        log_error "File $1 not found"
        return 1
    fi

    log_info "Processing $1..."
    cat "$1"
    log_info "Processing finished"
}

2. Debugging without polluting stdout:


debug() {
    [ "$DEBUG" = "1" ] && echo "DEBUG: $1" >&2
}

DEBUG=1 ./my_script.sh >data.txt
# debug on the terminal, data in the file

3. Errors in pipelines:


command1 2>errors1.log | command2 2>errors2.log | command3 2>errors3.log >final_output.txt
(command1 | command2 | command3 >output.txt) 2>all_errors.log

💡 Practice tip: In your own scripts, split stdout and stderr consistently. The scripts become easier to reuse.

⚠️ Note: Some older programs dump everything on stdout. You may need extra filtering.

❗ Typical mistakes:

Mixing data and messages:


# BAD: messages on stdout
echo "Processing file filename"
cat "filename"

# BETTER: messages on stderr
echo "Processing file filename" >&2
cat "filename"

Ignoring stderr:


# DANGEROUS: errors are swallowed
result=$(command 2>/dev/null)

# BETTER: handle errors
if ! result=$(command 2>error_log); then
    echo "Command failed, see error_log" >&2
    exit 1
fi

Redirection with > and >>

Redirection lets the shell send streams to files instead of the terminal, or take input from files instead of the keyboard. That is essential for automation, logging and batch work.

Basics of output redirection

The idea of redirection

The shell interprets operators (>, >>, <, …), opens the files, then starts the program with those streams. The program itself does not know about the redirect.

🔧 Simple example:


echo "Hello World"
Hello World

echo "Hello World" > greeting.txt
cat greeting.txt
Hello World

Syntax


[n]operator[file]

n is the file descriptor (optional; default 1 for >)
operator is >, >>, <, …
file is the target or source

Operator	Description	Example
`>`	stdout, overwrite	`command > file.txt`
`>>`	stdout, append	`command >> file.txt`
`<`	stdin from file	`command < input.txt`
`2>`	stderr	`command 2> errors.txt`
`2>>`	stderr, append	`command 2>> errors.txt`
`&>`	stdout and stderr	`command &> all_output.txt`

Simple redirection with >

Sending stdout to files

> creates the file if missing and overwrites it if it exists.

🔧 Practical examples:

1. Collect system stats:


ps aux > processes_$(date +%Y%m%d).txt
df -h > disk_status.txt
ip addr show > network_config.txt

2. Write configuration:


echo "ServerName example.com" > apache_config.txt
echo "DocumentRoot /var/www/html" >> apache_config.txt

cat > ~/.ssh/config << EOF
Host production
    HostName prod.example.com
    User admin
    Port 2222
EOF

3. Reports:


{
    echo "=== System report $(date) ==="
    echo "Uptime:"
    uptime
    echo "Memory:"
    free -h
    echo "Disk:"
    df -h
} > daily_report_$(date +%Y%m%d).txt

Overwrite vs. create


echo "First line" > new_file.txt
echo "Second line" > new_file.txt
cat new_file.txt
Second line  # the first line is gone

⚠️ Warning: > overwrites without asking. Previous content is lost.

Guard against accidental overwrite:


set -o noclobber
echo "Test" > existing_file.txt
bash: existing_file.txt: cannot overwrite existing file

echo "Test" >| existing_file.txt
set +o noclobber

🔧 Admin examples

1. Log rotation:


LOGFILE="/var/log/application.log"
ARCHIVE_DIR="/var/log/archive"

if [ -f "$LOGFILE" ]; then
    mv "$LOGFILE" "$ARCHIVE_DIR/application_$(date +%Y%m%d_%H%M%S).log"
fi

touch "$LOGFILE"
echo "Log rotated on $(date)" > "$LOGFILE"

2. Back up configuration:


tar -czf config_backup_$(date +%Y%m%d).tar.gz /etc/apache2/ /etc/nginx/ /etc/ssh/ 2> backup_errors.log

Appending with `>>`

Difference from simple redirection

>> appends. If the file is missing, it is created.

Operator	Existing file	New file	Use
`>`	Overwrites	Creates	One-shot reports
`>>`	Appends	Creates	Logs, continuous collection

🔧 Demo:


echo "Line 1" > testfile.txt
echo "Line 2" >> testfile.txt
echo "Line 3" >> testfile.txt

Logging use cases


while true; do
    echo "$(date): CPU: $(top -bn1 | grep "Cpu(s)" | awk '{print $2}')" >> system_monitor.log
    sleep 60
done


LOGFILE="/var/log/my_script.log"
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOGFILE"
}
log_message "Script started"

Avoiding data loss


CURRENT_LOG="/var/log/application.log"
ARCHIVE_LOG="/var/log/archive/application_$(date +%Y%m%d_%H%M%S).log"
cp "$CURRENT_LOG" "$ARCHIVE_LOG"
> "$CURRENT_LOG"
echo "$(date): log rotated, archive: $(basename "$ARCHIVE_LOG")" >> "$CURRENT_LOG"

Redirecting stderr


command 2> error_file.txt
command 2>> error_log.txt
command > output.txt 2> errors.txt

🔧 Practical examples:


find /etc -name "*.conf" 2>/dev/null > config_files.txt
gcc -o program source.c 2> compile_errors.txt
rsync -av /home/ /backup/ > backup_success.log 2> backup_errors.log
command 2>/dev/null

Combined stdout and stderr


command > output.txt 2>&1
command &> output.txt
command >> output.txt 2>&1
command &>> output.txt

⚠️ Important: Order of 2>&1 matters:


# RIGHT: redirect stdout first, then send stderr where stdout now goes
command > file.txt 2>&1

# WRONG: stderr still points at the original stdout (the terminal)
command 2>&1 > file.txt

🔧 Demo:


ls /exists /does-not-exist > test1.txt 2>&1
# both streams in the file

ls /exists /does-not-exist 2>&1 > test2.txt
# error still on the terminal; only stdout in the file

Input redirection with <


sort < unsorted_list.txt
awk -F',' '{print $1, $3}' < data.csv
mysql -u root -p database_name < backup.sql

Here documents and here strings


command << DELIMITER
Line 1
Line 2
DELIMITER

🔧 Practical uses:


cat << 'EOF' > /etc/apache2/sites-available/example.conf
<VirtualHost *:80>
    ServerName example.com
    DocumentRoot /var/www/example
</VirtualHost>
EOF


mysql -u root -p << 'SQL'
CREATE DATABASE IF NOT EXISTS myapp;
SQL


tr '[:lower:]' '[:upper:]' <<< "hello world"
HELLO WORLD

💡 Practice tip: Use input redirection when you want the shell to open the file, or when the program does not take a filename argument.

❗ Typical mistakes:


# WRONG direction
echo "test" < output.txt

# RIGHT
echo "test" > output.txt


# Variables expand
cat << EOF
Current user: $USER
EOF

# Quoted delimiter: no expansion
cat << 'EOF'
Current user: $USER
EOF

Using pipes

Pipes join simple programs into stronger chains without temporary files. | connects one command’s stdout to the next command’s stdin.

What pipes are and how they work

Connecting commands


┌─────────────────────────────────────────────────────────────┐
│                   UNIX PIPELINE DATA FLOW                   │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│   [ Program 1 ]        [ Program 2 ]        [ Program 3 ]   │
│         │                    │                    │         │
│      stdout               stdout               stdout       │
│         │                    │                    │         │
│         ├───► (Pipe: |) ────►│                    │         │
│               stdin          ├───► (Pipe: |) ────►│         │
│                                    stdin          │         │
│                                                   ▼         │
│                                            stdout (terminal)│
│                                                             │
└─────────────────────────────────────────────────────────────┘

The shell starts every program in the pipeline at once and connects their streams. Data flows in real time; nothing has to hit the disk.

🔧 Simple example:


# Without a pipe: clumsy temp files
ps aux > temp_processes.txt
grep apache temp_processes.txt > apache_processes.txt
wc -l apache_processes.txt
rm temp_processes.txt apache_processes.txt

# With a pipe
ps aux | grep apache | wc -l

Data flow between processes

Pipes use a kernel buffer (typically 64KB). A full buffer blocks the writer; an empty buffer blocks the reader.


# Slow producer, fast consumer
find / -name "*.log" 2>/dev/null | head -10
# find may keep running after head exits

# Fast producer, slow consumer
cat large_file.txt | sleep 10
# cat waits until the reader drains the pipe

💡 Tip: Pipes are lazy — they only process as much data as the downstream command needs. That is efficient on large files.

Pipes vs. file redirection

Property	Pipes	File redirection
Storage	RAM	Disk
Speed	Very fast	Slower (I/O)
Persistence	Temporary	Permanent
Parallelism	Concurrent	Sequential
Disk use	Minimal	Can be large
Debugging	Harder	Easier (intermediate files)

🔧 Comparison:


grep "ERROR" /var/log/huge.log > temp1.txt
sort temp1.txt > temp2.txt
uniq temp2.txt > temp3.txt
wc -l temp3.txt
rm temp1.txt temp2.txt temp3.txt

grep "ERROR" /var/log/huge.log | sort | uniq | wc -l

Basic pipe syntax


command1 | command2

🔧 Common patterns:


ps aux | grep apache | grep -v grep | wc -l
who | wc -l
ls -1 | wc -l
cut -d: -f7 /etc/passwd | sort | uniq
history | awk '{print $2}' | sort | uniq -c | sort -nr | head -10
ps aux | awk '{print $2, $3, $4, $11}' | column -t
du -h /var/log/* | sort -hr

Multiple pipes

🔧 Practical chains:


cat /var/log/apache2/access.log | \
  awk '{print $1}' | \
  sort | \
  uniq -c | \
  sort -nr | \
  head -10


ps aux | awk 'NR>1 {print $4, $11}' | sort -nr | head -5 | column -t


netstat -an | grep ESTABLISHED | awk '{print $4}' | sed 's/.*://' | sort -n | uniq -c | sort -nr

Advanced pipe uses

Filter early for speed:


# BAD: sort the whole file first
cat huge_file.txt | sort | uniq | grep "pattern" | head -10

# BETTER: filter first
grep "pattern" huge_file.txt | sort | uniq | head -10

Debugging a chain:


ps aux
ps aux | grep apache
ps aux | grep apache | grep -v grep
ps aux | grep apache | grep -v grep | wc -l
ps aux | tee debug1.txt | grep apache | tee debug2.txt | wc -l


command1 | command2 | command3
echo "Pipeline status: ${PIPESTATUS[@]}"
set -o pipefail
command1 | command2 | command3 || echo "Pipeline failed"

Named pipes (FIFOs)

Concept and mkfifo

Named pipes persist as special files and let unrelated processes talk.


mkfifo my_pipe
ls -l my_pipe
prw-rw-r-- 1 user user 0 Jul 13 10:14 my_pipe


# Terminal 1
echo "Hello world" > my_pipe

# Terminal 2
cat < my_pipe
Hello world

🔧 Logging collector:


mkfifo /tmp/app_logs
echo "App1: $(date) - Startup complete" > /tmp/app_logs &
while read log_entry; do
    echo "$log_entry" >> /var/log/applications.log
done < /tmp/app_logs

Property	Anonymous pipes	Named pipes (FIFOs)
Lifetime	Only while the pipeline runs	Persist in the filesystem
Access	Related processes	Any processes
Visibility	Invisible	Visible as a file
Cleanup	Automatic	Delete by hand
Use	Command chaining	IPC
Performance	Very fast	Slightly slower

⚠️ Named-pipe notes: Reads and writes block until both ends are connected. Delete FIFOs when you are done. Permissions work like normal files.

💡 Practice tip: Named pipes fit microservice IPC, log aggregation, parallel build workers and alerting.

❗ Typical mistakes: Deadlocks if both sides wait. Always rm the FIFO, or use trap 'rm -f /tmp/my_pipe' EXIT. Create with mkfifo -m 660 /tmp/secure_pipe when you need tighter rights. Prefer timeout 5 cat my_pipe over hanging forever.

Redirection combinations

Combined stdout/stderr redirection

Syntax	Description	Bash	Example
`command > file 2>&1`	POSIX classic	All	`ls /tmp > output.txt 2>&1`
`command &> file`	Short form	Bash 4.0+	`ls /tmp &> output.txt`
`command >& file`	Alternate short form	Bash/csh	`ls /tmp >& output.txt`
`command >> file 2>&1`	Append, classic	All	`ls /tmp >> output.txt 2>&1`
`command &>> file`	Append, modern	Bash 4.0+	`ls /tmp &>> output.txt`

🔧 Practical scenarios:


./install_script.sh > install.log 2>&1
./install_script.sh &> install.log
./monitoring_script.sh &>> system_monitor.log
find /var -name "*.log" -exec grep "ERROR" {} \; &> debug_output.txt
rsync -av /home/ /backup/home/ &> backup_$(date +%Y%m%d_%H%M%S).log

Combining pipes and redirections


{
    grep "ERROR" /var/log/application.log | \
    awk '{print $1, $2, $NF}' | \
    sort | uniq -c | sort -nr
} > error_analysis.txt 2> analysis_errors.log


ps aux | \
  tee process_list.debug | \
  grep apache | \
  tee apache_processes.debug | \
  awk '{sum += $3} END {print "Total CPU:", sum "%"}'

Advanced redirection techniques


exec 3> logfile.txt
echo "Message 1" >&3
echo "Message 2" >&3
exec 3>&-

🔧 Practical applications:


exec 3> /var/log/debug.log
exec 4> /var/log/errors.log
exec 5> /var/log/audit.log

log_debug() { echo "$(date '+%Y-%m-%d %H:%M:%S') DEBUG: $*" >&3; }
log_error() { echo "$(date '+%Y-%m-%d %H:%M:%S') ERROR: $*" >&4; echo "$(date '+%Y-%m-%d %H:%M:%S') ERROR: $*" >&2; }
log_audit() { echo "$(date '+%Y-%m-%d %H:%M:%S') AUDIT: $*" >&5; }

exec 3>&- 4>&- 5>&-


exec 6>&1 7>&2
exec 1> temp_output.log 2> temp_errors.log
echo "This goes to temp_output.log"
exec 1>&6 2>&7
exec 6>&- 7>&-
echo "This appears on the terminal"

Redirecting to /dev/null


find /etc -name "*.conf" 2>/dev/null
command > /dev/null
command &> /dev/null
command < /dev/null

🔧 Practical scenarios:


if systemctl is-active apache2 &> /dev/null; then
    echo "Apache is running"
else
    echo "Apache is stopped"
fi

for file in *.txt; do
    process_file "$file" > /dev/null || echo "Error on $file"
done

Combining pipes with redirection

Complex processing pipelines

Pipes plus redirections give you staged workflows with error files and intermediate tee snapshots:


{
    grep "ERROR" /var/log/application.log | \
    awk '{print $1, $2, $NF}' | \
    sort | uniq -c | sort -nr
} > error_analysis.txt 2> analysis_errors.log

🔧 Practical example:


ps aux | \
  tee process_list.debug | \
  grep apache | \
  tee apache_processes.debug | \
  awk '{sum += $3} END {print "Total CPU:", sum "%"}' | \
  tee cpu_summary.debug

Error handling in combined operations

🔧 Practical example:


set -o pipefail
command1 | command2 | command3 || echo "Pipeline failed"

Practical scenarios for system administrators

Log rotation and archiving

🔧 Practical example:


LOGFILE="/var/log/application.log"
ARCHIVE_DIR="/var/log/archive"
cp "$LOGFILE" "$ARCHIVE_DIR/application_$(date +%Y%m%d_%H%M%S).log"
> "$LOGFILE"
find "$ARCHIVE_DIR" -name "*.log_*" -mtime +7 ! -name "*.gz" -exec gzip {} \;

Backups with logging

🔧 Practical example:


rsync -av --progress /home/ /backup/home/ > /var/log/backup_success.log 2> /var/log/backup_errors.log

Monitoring and alerting

CPU, memory and disk checks that tee into a monitor log and append alerts.

💡 Practice tip: Combine redirection techniques step by step. Start simple, then grow the workflow.

⚠️ Important: On complex combinations, understand operator order and test it. Use set -x when debugging.

❗ Typical mistakes: Wrong order of combined redirects; forgetting to close extra file descriptors; race conditions in parallel work; unhandled errors in long pipelines.

tee and xargs

tee splits a stream to stdout and to files. xargs turns stdin into command arguments for tools that do not read a pipe.

The tee command

How it splits a stream


┌─────────────────────────────────────────────────────────────┐
│                   TEE COMMAND: T-JUNCTION                   │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│                       Standard Input                        │
│                             │                               │
│                             ▼                               │
│                      ┌─────────────┐                        │
│                      │  tee command│                        │
│                      └──────┬──────┘                        │
│                             │                               │
│              ┌──────────────┴──────────────┐                │
│              ▼                             ▼                │
│      Standard Output                  File(s)               │
│   (terminal / pipe)                (log / backup)           │
│                                                             │
└─────────────────────────────────────────────────────────────┘

🔧 Simple example:


ps aux > processes.txt   # save, but you do not see it
ps aux                   # see it, but do not save
ps aux | tee processes.txt

Option	Description	Example
`-a`	Append	`command \| tee -a logfile.txt`
`-i`	Ignore interrupt signals	`command \| tee -i output.txt`
(none)	Overwrite	`command \| tee output.txt`

🔧 Practical examples:


df -h | tee disk_usage_human.txt | awk '{print $1, $5}' > disk_usage_simple.txt

cat /var/log/apache2/access.log | \
  tee step1_raw.log | \
  grep "404" | \
  tee step2_404s.log | \
  awk '{print $1}' | \
  sort | uniq -c | sort -nr

Using tee with sudo


# FAILS: sudo applies to echo, not the redirect
echo "new line" | sudo > /etc/hosts

# WORKS: tee runs as root
echo "127.0.0.1 test.local" | sudo tee -a /etc/hosts
echo "127.0.0.1 test.local" | sudo tee -a /etc/hosts > /dev/null

🔧 Privileged writes:


cat << 'EOF' | sudo tee /etc/apache2/sites-available/example.conf > /dev/null
<VirtualHost *:80>
    ServerName example.com
    DocumentRoot /var/www/example
</VirtualHost>
EOF

echo "0 2 * * * /usr/local/bin/backup.sh" | sudo tee -a /etc/crontab
echo "vm.swappiness=10" | sudo tee -a /etc/sysctl.conf

The xargs command

The problem: commands that do not read a pipe


find /tmp -name "*.tmp" | rm
rm: missing operand

ls *.txt | cp /backup/
cp: missing destination file operand

xargs reads stdin and builds argument lists:


stdin → xargs → command arg1 arg2 arg3 ...

🔧 Simple examples:


find /tmp -name "*.tmp" | xargs rm
ls *.txt | xargs -I {} cp {} /backup/
find /var/www -name "*.php" | xargs chmod 644

Option	Description	Example
`-n NUM`	At most NUM arguments per call	`echo "1 2 3 4" \| xargs -n 2 echo`
`-I REPLACE`	Replace the placeholder	`ls *.txt \| xargs -I {} cp {} backup/`
`-0`	NUL-terminated input	`find . -name "*.txt" -print0 \| xargs -0 rm`
`-P NUM`	Run NUM processes in parallel	`ls *.jpg \| xargs -P 4 -I {} convert {} {}.png`
`-t`	Print the command before running it	`find /tmp -name "*.tmp" \| xargs -t rm`
`-r`	Do nothing if stdin is empty	`find /tmp -name "*.tmp" \| xargs -r rm`

🔧 Option details:


echo "file1 file2 file3 file4" | xargs -n 2 echo "Processing:"
find . -name "*.txt" -print0 | xargs -0 rm
ls *.jpg | xargs -P 4 -I {} convert {} {}.png

Safety and special characters


# DANGEROUS with spaces
echo "important file.txt" | xargs rm

# SAFE
find . -name "*.txt" -print0 | xargs -0 rm
find . -name "*.txt" | xargs -I {} rm "{}"

Advanced tee uses

🔧 Practical example:


ps aux | tee processes_backup1.txt processes_backup2.txt processes_current.txt

Monitoring and logging scenarios


{
    while true; do
        timestamp=$(date '+%Y-%m-%d %H:%M:%S')
        cpu_usage=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | sed 's/%us,//')
        echo "$timestamp CPU:${cpu_usage}%"
        sleep 5
    done
} | tee -a /var/log/realtime_monitoring/realtime_$(date +%Y%m%d).log

Advanced xargs uses


find /tmp -type f -mtime +7 -print0 | xargs -0 -r rm -v
find /var/www/html -type d -print0 | xargs -0 chmod 755
find /var/log -name "*.log" -type f -mtime +7 ! -name "*.gz" -print0 | xargs -0 -r -P 2 gzip -v

Combining tee and xargs

🔧 Practical example:


find /var/log -name "*.log" -print0 | \
  xargs -0 grep -l "ERROR" | \
  tee error_logs.txt | \
  xargs -r wc -l

💡 Practice tip: tee plus xargs is strong when you need both intermediate files and parallel work.

⚠️ Important: With xargs -P, watch system load. Start with few parallel jobs and raise the count step by step.

Why these ideas matter for LPIC-1

💡 Exam weight: On LPIC-1 exam 101, streams, pipes and redirections are about 15–20% of the marks, especially objectives 103.4 and 103.1.

They underpin log analysis, backup scripts, data pipelines and professional error handling. They also show that you understand the Unix model: small tools, joined cleanly.

💡 Note: Further exam information is in LPIC-1: basic navigation and filesystem commands.

Command Reference (Cheatsheet)

Operator / command	Syntax example	Description and LPIC-1 relevance
`>`	`command > file.txt`	Redirect `stdout` (overwrite)
`>>`	`command >> file.txt`	Append `stdout`
`<`	`command < input.txt`	File as `stdin`
`2>`	`command 2> error.log`	Redirect `stderr` (FD 2) only
`2>&1`	`command > all.log 2>&1`	Duplicate `stderr` onto `stdout`
`&>`	`command &> all.log`	Bash short form for both streams
`\|`	`command1 \| command2`	Pipe `stdout` of command1 into `stdin` of command2
`<< EOF`	`cat << EOF > file`	Here-document until the delimiter
`<<<`	`grep "word" <<< "$VAR"`	Here-string into `stdin`
`<()`	`diff <(cmd1) <(cmd2)`	Process substitution
`tee`	`cmd \| tee -a log.txt`	Copy `stdin` to `stdout` and a file (`-a` append)
`xargs`	`find . -print0 \| xargs -0 rm`	Build arguments from `stdin` (`-n`, `-I`, `-0`, `-P`)
`mkfifo`	`mkfifo /tmp/named_pipe`	Create a named FIFO
`exec`	`exec 3> file.log`	Open or reshape file descriptors for the shell

Further Resources

Resource	Description
GNU Bash Reference Manual: Redirections	Official docs on redirections, file descriptors and here-docs
GNU Coreutils: tee invocation	GNU manual for `tee`
GNU Findutils: xargs manual	`xargs` options and parallel runs
LPI: LPIC-1 Exam 101 Objectives	Official objectives for topic 103.4

Conclusion

Streams, pipes and redirections are the core of the Linux command line and of the Unix model: small tools joined on standard channels into efficient pipelines. Simple >, parallel xargs and splitting with tee are daily admin skills.

💡 Practice tip: On complex pipelines, check intermediate results with tee or process substitution <() without breaking the flow.

The next LPIC-1 module covers archiving and compression: LPIC-1: archiving and compressing files — tar, gzip, bzip2, xz and backup pipelines.

Course overview: All LPIC-1 articles and modules

In Linux everything is a data stream

Place in the LPIC-1 certification

Practical relevance for day-to-day admin work

How to get the most from this module

Standard streams (stdin, stdout, stderr)

What are standard streams?

Standard input (stdin)

Standard output (stdout)

Standard error (stderr)

Redirection with > and >>

Basics of output redirection

Simple redirection with >

Appending with >>

Avoiding data loss

Redirecting stderr

Input redirection with <

Using pipes

What pipes are and how they work

Data flow between processes

Basic pipe syntax

Advanced pipe uses

Named pipes (FIFOs)

Redirection combinations

Combined stdout/stderr redirection

Combining pipes and redirections

Advanced redirection techniques

Redirecting to /dev/null

Combining pipes with redirection

Error handling in combined operations

Practical scenarios for system administrators

tee and xargs

The tee command

The xargs command

Advanced tee uses

Monitoring and logging scenarios

Advanced xargs uses

Combining tee and xargs

Why these ideas matter for LPIC-1

Command Reference (Cheatsheet)

Further Resources

Conclusion

Appending with `>>`