Linux Administration #7: System Security and Hardening

Welcome to the seventh part of our technical wiki series on Linux administration!

After we covered data backup and recovery in the previous article, we now turn to a critical aspect of system administration: system security and hardening.

⚠️ Note: In this article, we use Ubuntu/Debian as the example distribution. The basic security concepts are the same on all Linux systems, but package installation and some configuration paths may vary depending on the distribution. If you use a different distribution, please consult the relevant documentation for the specific installation commands and paths.

As a Linux administrator, securing your systems is one of your most important tasks.

Think of your system like a house – you need sturdy locks, good lighting, and a security system to protect against intruders. System hardening is the process of reducing attack vectors by disabling unnecessary services, applying security policies, and monitoring system activity.

Security Fundamentals

Before diving into specific security tools and techniques, you need to understand the fundamental principles that guide all security work on Linux systems.

The Principle of Least Privilege

The principle of least privilege means granting only the minimum permissions necessary to perform a task. This applies to users, processes, and services alike.

Users should have only the permissions they need
Services should run with minimal privileges
Files should have restrictive permissions
Administrative tasks should require explicit elevation

Defense in Depth

Defense in depth means implementing multiple layers of security controls. No single security measure is perfect – multiple layers ensure that if one fails, others still provide protection.


┌─ Defense in Depth ───────────────────────────────────────────┐
│ - Physical Security                                          │
├─ Network Security ───────────────────────────────────────────┤
│ - Firewalls, IDS/IPS                                         │
├─ Host Security ──────────────────────────────────────────────┤
│ - Hardening, ACLs, SELinux                                   │
├─ Application Security ───────────────────────────────────────┤
│ - Secure coding, updates                                     │
├─ Data Security ──────────────────────────────────────────────┤
│ - Encryption, backups                                        │
└─────────────────────────────────────────────────────────────┘

Attack Surface Reduction

Every running service, open port, or enabled feature represents a potential attack vector. Reducing the attack surface means disabling everything that isn't strictly necessary.


# Show all listening ports and services
sudo netstat -tulpn

# List all enabled services
systemctl list-unit-files --type=service | grep enabled

# Check for unnecessary services
sudo systemctl status cups    # Printer service
sudo systemctl status avahi-daemon  # mDNS/DNS-SD

Firewall Configuration

A firewall is your first line of defense against network-based attacks. It controls incoming and outgoing traffic based on predefined rules.

UFW (Uncomplicated Firewall)

UFW is the recommended firewall management tool for Ubuntu/Debian systems. It provides a simple command-line interface for managing netfilter rules.

Basic UFW configuration:


# Enable UFW
sudo ufw enable

# Set default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow SSH (important - don't lock yourself out!)
sudo ufw allow 22/tcp

# Allow web server
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Check status
sudo ufw status verbose

Advanced UFW rules:


# Allow from specific IP
sudo ufw allow from 192.168.1.100

# Allow from subnet
sudo ufw allow from 192.168.1.0/24

# Rate limiting (protects against brute force)
sudo ufw limit 22/tcp

# Delete a rule
sudo ufw delete allow 80/tcp

# Reset all rules
sudo ufw reset

iptables/nftables

For more advanced firewall configurations, you can use iptables (legacy) or nftables (modern replacement) directly.

iptables basics:


# List all rules
sudo iptables -L -n -v

# Allow incoming SSH
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# Block an IP address
sudo iptables -A INPUT -s 192.168.1.100 -j DROP

# Allow established connections
sudo iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

nftables basics:


# List all rules
sudo nft list ruleset

# Add a rule
sudo nft add rule inet filter input tcp dport 22 accept

# Flush all rules
sudo nft flush ruleset

Firewall Best Practices

Default deny: Block all incoming traffic by default
Explicit allow: Only open ports that are necessary
Document rules: Keep a record of why each rule exists
Regular audits: Review rules periodically
Test changes: Always test firewall changes before applying in production

Intrusion Prevention

Intrusion prevention systems detect and block malicious activity before it can harm your system.

Fail2ban

Fail2ban monitors log files for repeated failed login attempts and bans offending IP addresses by updating firewall rules.

Installation and configuration:


# Install fail2ban
sudo apt install fail2ban

# Create local configuration
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

# Edit configuration
sudo nano /etc/fail2ban/jail.local

Example jail configuration:


[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 5

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3

Managing fail2ban:


# Check status
sudo fail2ban-client status

# Check specific jail
sudo fail2ban-client status sshd

# Manually ban an IP
sudo fail2ban-client set sshd banip 192.168.1.100

# Unban an IP
sudo fail2ban-client set sshd unbanip 192.168.1.100

# Restart after configuration changes
sudo systemctl restart fail2ban

SELinux/AppArmor

Mandatory Access Control (MAC) systems like SELinux and AppArmor provide additional security beyond traditional Unix permissions.

AppArmor (Ubuntu/Debian default):


# Check AppArmor status
sudo aa-status

# Set profile to enforce
sudo aa-enforce /etc/apparmor.d/usr.sbin.apache2

# Set profile to complain (log only)
sudo aa-complain /etc/apparmor.d/usr.sbin.apache2

# Disable a profile
sudo ln -s /etc/apparmor.d/usr.sbin.apache2 /etc/apparmor.d/disable/

SELinux (RHEL/CentOS default):


# Check SELinux status
sestatus

# Set SELinux to enforcing
sudo setenforce 1

# Set SELinux to permissive
sudo setenforce 0

# List file contexts
ls -Z /var/www/html/

# Change file context
sudo chcon -t httpd_sys_content_t /var/www/custom/

Access Control

Proper access control ensures that only authorized users and processes can access system resources.

Sudo Configuration

Sudo allows controlled administrative access without sharing the root password.

Basic sudo configuration:


# Edit sudoers file safely
sudo visudo

# Add user to sudo group
sudo usermod -aG sudo username

# Grant specific commands
username ALL=(ALL) /usr/bin/systemctl restart apache2

# Run as specific user
username ALL=(www-data) /usr/bin/php

Security restrictions:


# Require password for sudo
Defaults    timestamp_timeout=0

# Limit password attempts
Defaults    passwd_tries=3

# Log sudo commands
Defaults    logfile="/var/log/sudo.log"

# Require tty
Defaults    requiretty

ACLs (Access Control Lists)

ACLs provide fine-grained permission control beyond traditional Unix permissions.


# View ACLs
getfacl /path/to/file

# Set ACL for user
sudo setfacl -m u:username:rwx /path/to/file

# Set ACL for group
sudo setfacl -m g:groupname:rx /path/to/file

# Set default ACL (for new files)
sudo setfacl -d -m u:username:rwx /path/to/directory

# Remove ACL
sudo setfacl -x u:username /path/to/file

# Remove all ACLs
sudo setfacl -b /path/to/file

PAM (Pluggable Authentication Modules)

PAM provides flexible authentication mechanisms for Linux systems.

Password policies:


# Install PAM modules
sudo apt install libpam-pwquality

# Edit password quality
sudo nano /etc/security/pwquality.conf

Example PAM configuration:


# /etc/security/pwquality.conf
minlen = 12
dcredit = -1
ucredit = -1
lcredit = -1
ocredit = -1
maxrepeat = 3

Account lockout:


# Edit PAM configuration
sudo nano /etc/pam.d/common-auth

# Add after first line:
auth required pam_tally2.so deny=5 unlock_time=900

User Security

User account security is a critical component of overall system security.

Password Policies

Strong password policies prevent unauthorized access through credential guessing or theft.


# Set password expiration
sudo chage -M 90 username    # Max age 90 days
sudo chage -m 7 username     # Min age 7 days
sudo chage -W 14 username    # Warn 14 days before expiry

# View password policy
sudo chage -l username

# Force password change on next login
sudo chage -d 0 username

SSH Hardening

SSH is often the primary remote access method and requires careful hardening.

Configuration (/etc/ssh/sshd_config):


# Disable root login
PermitRootLogin no

# Use SSH protocol 2
Protocol 2

# Limit authentication attempts
MaxAuthTries 3

# Disable empty passwords
PermitEmptyPasswords no

# Use key-based authentication only
PubkeyAuthentication yes
PasswordAuthentication no

# Restrict to specific users
AllowUsers username1 username2

# Change default port
Port 2222

# Set idle timeout
ClientAliveInterval 300
ClientAliveCountMax 2

Key-based authentication:


# Generate SSH key pair
ssh-keygen -t ed25519 -C "your_email@example.com"

# Copy public key to server
ssh-copy-id username@server

# Disable password authentication
sudo nano /etc/ssh/sshd_config
# Set: PasswordAuthentication no
sudo systemctl restart sshd

User Monitoring

Monitor user activity to detect suspicious behavior.


# Show who is currently logged in
who

# Show last login attempts
last

# Show failed login attempts
sudo lastb

# Monitor user activity
sudo ausearch -m USER_LOGIN --interpret

Encryption

Encryption protects data both at rest (stored data) and in transit (data being transmitted).

Disk Encryption

Disk encryption protects your data if the physical media is stolen or accessed.

LUKS (Linux Unified Key Setup):


# Create encrypted partition
sudo cryptsetup luksFormat /dev/sdb1

# Open encrypted partition
sudo cryptsetup luksOpen /dev/sdb1 secure_data

# Create filesystem
sudo mkfs.ext4 /dev/mapper/secure_data

# Mount encrypted partition
sudo mount /dev/mapper/secure_data /mnt/encrypted

# Close encrypted partition
sudo cryptsetup luksClose secure_data

Automount encrypted partition:


# Add to /etc/crypttab
secure_data /dev/sdb1 none luks

# Add to /etc/fstab
/dev/mapper/secure_data /mnt/encrypted ext4 defaults 0 2

Filesystem Encryption

For encrypting individual directories or files without full disk encryption.

eCryptfs for home directories:


# Install eCryptfs
sudo apt install ecryptfs-utils

# Migrate home directory
sudo ecryptfs-migrate-home -u username

# Follow prompts and save passphrase securely

GPG (GNU Privacy Guard)

GPG provides asymmetric encryption for files and communications.


# Generate key pair
gpg --full-generate-key

# List keys
gpg --list-keys

# Encrypt file
gpg -e -r recipient@email.com file.txt

# Decrypt file
gpg -d file.txt.gpg

# Sign file
gpg --sign file.txt

# Verify signature
gpg --verify file.txt.sig

SSL/TLS Configuration

Secure web communications require proper SSL/TLS configuration.

Generate self-signed certificate:


# Generate private key
openssl genrsa -out private.key 2048

# Generate CSR
openssl req -new -key private.key -out request.csr

# Generate self-signed certificate
openssl x509 -req -days 365 -in request.csr -signkey private.key -out certificate.crt

Let's Encrypt (recommended for production):


# Install Certbot
sudo apt install certbot python3-certbot-nginx

# Obtain certificate
sudo certbot --nginx -d example.com

# Auto-renewal
sudo certbot renew --dry-run

Security Monitoring

Continuous monitoring is essential for detecting and responding to security incidents.

Log Analysis

System logs contain valuable information about security events.

Important log files:


┌─ Log File Structure (/var/log/) ───────────────────────────┐
│ /var/log/                                                    │
│ ├── auth.log      → Authentication & SSH events              │
│ ├── syslog        → Central system messages & daemons        │
│ ├── kern.log      → Kernel messages & hardware events        │
│ ├── secure        → Security events (RHEL/CentOS)            │
│ └── audit/        → Audit logs & SELinux events              │
└──────────────────────────────────────────────────────────────┘

Log analysis commands:


# Real-time authentication monitoring
sudo tail -f /var/log/auth.log

# Search for failed login attempts
sudo grep "Failed password" /var/log/auth.log

# Check for root login attempts
sudo grep "root" /var/log/auth.log

# Analyze SSH activity
sudo grep "sshd" /var/log/auth.log | tail -50

# Count failed attempts by IP
sudo grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -n

System Auditing

System auditing provides detailed tracking of security-relevant events.

Auditd (Linux Audit Daemon):


# Install auditd
sudo apt install auditd

# Add audit rule
sudo auditctl -w /etc/passwd -p wa -k password_changes

# Search audit logs
sudo ausearch -k password_changes

# Generate audit report
sudo aureport --auth
sudo aureport --login
sudo aureport --failed

Security Scanning

Regular security scans help identify vulnerabilities before attackers can exploit them.

Lynis (security auditing tool):


# Install Lynis
sudo apt install lynis

# Run security audit
sudo lynis audit system

# Check specific profile
sudo lynis audit system --profile /etc/lynis/custom.prf

ClamAV (antivirus):


# Install ClamAV
sudo apt install clamav clamav-daemon

# Update virus definitions
sudo freshclam

# Scan specific directory
sudo clamscan -r /home/

# Scan entire system
sudo clamscan -r --bell -i /

Intrusion Detection

Intrusion detection systems monitor for unauthorized access or suspicious activity.

AIDE (Advanced Intrusion Detection Environment):


# Install AIDE
sudo apt install aide

# Initialize database
sudo aideinit

# Check for changes
sudo aide --check

# Update database after legitimate changes
sudo aide --update
sudo cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db

Security Hardening

Systematic hardening reduces attack vectors and improves overall security posture.

Disable Unnecessary Services


# List enabled services
systemctl list-unit-files --type=service | grep enabled

# Stop and disable a service
sudo systemctl stop cups
sudo systemctl disable cups

# Mask a service (prevent manual start)
sudo systemctl mask cups

Kernel Hardening

Sysctl parameters can enhance kernel security.


# Edit sysctl.conf
sudo nano /etc/sysctl.conf

Recommended security parameters:


# Ignore ICMP broadcast requests
net.ipv4.icmp_echo_ignore_broadcasts = 1

# Disable source packet routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0

# Enable SYN flood protection
net.ipv4.tcp_syncookies = 1

# Disable IP forwarding (unless router)
net.ipv4.ip_forward = 0

# Log Martian packets
net.ipv4.conf.all.log_martians = 1

# Ignore redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0


# Apply changes
sudo sysctl -p

File Permission Hardening


# Find world-writable files
sudo find / -type f -perm -0002 -ls

# Find SUID files
sudo find / -type f -perm -4000 -ls

# Find SGID files
sudo find / -type f -perm -2000 -ls

# Remove world-writable permissions
sudo chmod o-w /path/to/file

# Remove SUID bit
sudo chmod u-s /path/to/file

SSH Directory Permissions


# Set correct permissions
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
chmod 644 ~/.ssh/known_hosts

Best Practices Checklist

Use these checklists as a reference for maintaining system security.

Daily Tasks


┌─ Daily Security Tasks ──────────────────────────────────────┐
│ - Check auth.log for failed login attempts                  │
│ - Review system resource usage                              │
│ - Verify backup completion                                  │
├─ Weekly Tasks ──────────────────────────────────────────────┤
│ - Install security updates                                  │
│ - Review IDS reports                                        │
│ - Audit user accounts                                       │
├─ Monthly Tasks ─────────────────────────────────────────────┤
│ - Run comprehensive security audit                          │
│ - Review firewall rules                                     │
│ - Test disaster recovery procedures                         │
└─────────────────────────────────────────────────────────────┘

Security Audit Commands


# Run comprehensive security audit
sudo lynis audit system

# Check for open ports
sudo netstat -tulpn

# Check listening services
sudo ss -tulpn

# Review recent login activity
last -20

# Check for failed logins
sudo lastb | head -20

# Verify system file integrity
sudo debsums -c

Exercise

Web Server Security Audit

Scenario: You've been hired as a security consultant for a small company. The company runs a web server that has shown suspicious activity recently. Your task is to perform a comprehensive security audit and implement hardening measures.

Tasks:

Configure UFW to allow only SSH (port 22), HTTP (port 80), and HTTPS (port 443).
Install and configure fail2ban to protect SSH with a maximum of 3 failed attempts.
Review all system logs for suspicious activity in the last 24 hours.
Check for unnecessary services and disable them.
Implement proper SSH hardening (disable root login, disable password authentication).
Create a monitoring script that checks for failed login attempts every hour.

Solution:

Configure UFW:


sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 22/tcp
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable

Configure fail2ban:


sudo apt install fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local


[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600

Check logs for suspicious activity:


sudo grep "Failed password" /var/log/auth.log | grep "$(date -d '24 hours ago' +'%b %d')"
sudo grep "Accepted" /var/log/auth.log | grep "$(date -d '24 hours ago' +'%b %d')"

SSH hardening (/etc/ssh/sshd_config):


PermitRootLogin no
PasswordAuthentication no
MaxAuthTries 3

Monitoring script:


#!/bin/bash
THRESHOLD=100
COUNT=$(grep "Failed password" /var/log/auth.log | grep "$(date +'%b %d %H')" | wc -l)
if [ $COUNT -gt $THRESHOLD ]; then
	echo "Warning: More than $THRESHOLD failed SSH login attempts in the last hour!" | mail -s "Security Alert" admin@example.com
fi

Save as /usr/local/bin/security_monitor.sh, make executable, and add to crontab:


chmod +x /usr/local/bin/security_monitor.sh


0 * * * * /usr/local/bin/security_monitor.sh

Command Reference (Cheatsheet)

Command	Category	Description
`sudo ufw enable`	Firewall	Enables the Uncomplicated Firewall
`sudo ufw allow 22/tcp`	Firewall	Opens TCP port 22 for SSH connections
`sudo ufw status verbose`	Firewall	Shows detailed firewall rule status
`sudo fail2ban-client status sshd`	Intrusion Prevention	Shows SSH jail status and banned IPs
`sudo lynis audit system`	Security Audit	Performs comprehensive local system security audit
`sudo getfacl /path/file`	Access Control	Reads extended POSIX Access Control Lists
`sudo setfacl -m u:user:rwx /path`	Access Control	Grants specific user extended ACL permissions
`sudo visudo`	Privileges	Safely edits the `/etc/sudoers` configuration file
`sudo chage -l user`	Password Policy	Shows password expiration and aging information
`sudo cryptsetup luksOpen /dev/sdX data`	Encryption	Opens a LUKS-encrypted block device
`sudo auditctl -l`	Audit & Logging	Lists all active kernel audit rules

Further Resources

Resource	Description
Ubuntu Security Guide	Official security documentation and best practices for Ubuntu Server
Debian Securing Manual	Official guide for hardening and securing Debian systems
CIS Linux Benchmarks	Industry-standard benchmarks for secure system configurations
Lynis Documentation	Documentation and hardening guide for the Lynis security auditing tool
SELinux Project Wiki	Official documentation and guidelines for Security-Enhanced Linux

Conclusion

The security of a Linux system is not a one-time state but a continuous process of hardening, restrictive privilege assignment, and proactive monitoring. In this module, you learned how to secure your systems reliably through firewalls (UFW), intrusion prevention (Fail2ban), least-privilege sudo rules, ACLs, and modern encryption methods.

👉 Next up: Linux Administration #8: Virtualization and VM Management

👉 Course Overview: All Linux Administration Articles & Modules