Linux server hardening: SSH with FIDO2/YubiKey and CrowdSec

Linux server hardening with SSH FIDO2/YubiKey for passwordless authentication and CrowdSec as a collaborative intrusion-prevention system on Ubuntu 24.04/26.04.

Reading time: 40 min

SSH brute-force attacks run continuously against every public server. Most Linux servers still default to passwords or static SSH keys — both have known weaknesses. Passwords fall to credential stuffing and GPU clusters in minutes; static keys stay unchanged until someone rotates them by hand.

On Ubuntu 24.04 LTS and Ubuntu 26.04 LTS the server is locked down with two concrete measures: SSH with FIDO2 and YubiKey for hardware-based, passwordless authentication, and CrowdSec as a collaborative intrusion-prevention system. Together they close the most common attack vectors without password rotation or key-management busywork.

The configuration works on a fresh server the same as on an existing setup — no assumptions about the current state.

💡 Prerequisites: Basic knowledge of SSH and systemd. An Ubuntu 24.04 or 26.04 server with root or sudo access. For FIDO2 you need a YubiKey (or another FIDO2 device). For CrowdSec a TCP port for SSH is enough. The audience is Linux administrators, DevOps engineers and homelab operators who want to harden existing servers or stand up new ones to a current standard.

⚠️ Note: Hardening here is at operating-system level. Application-specific hardening (web servers, databases) stays out of scope, because each application has its own requirements.

SSH with FIDO2 and YubiKey

Why FIDO2 for SSH?

Passwords are the weakest link in the SSH chain. Every server with a public IP is permanently worked by brute-force scanners — fail2ban logs show that daily. Static SSH keys (Ed25519, RSA) are better, but they do not rotate. Anyone holding a compromised private key stays current unnoticed.

FIDO2 solves both problems:

The private key is stored on a physical device (YubiKey) and never leaves it. Without the YubiKey on the local machine, login is impossible — no matter how good the password or key is. FIDO2 additionally enforces user verification (PIN or biometrics) on every login.

What changes:

Property Password Static SSH key FIDO2 + YubiKey
Physical component No No Yes
Brute-force resistant Conditional Yes Yes
Phishing resistant No Conditional Yes
Rotation Manual Manual Automatic (key on device)
Multi-factor Optional No Standard

The combination of physical token and PIN makes SSH logins nearly unattackable. An attacker would need both the YubiKey and the PIN — and both at the same physical station.

💡 FIDO2 vs. U2F: FIDO2 is the successor to U2F and supports WebAuthn (browser) and CTAP2 (native apps). For SSH, CTAP2 is what matters — the YubiKey is addressed directly by the SSH client, with no browser detour.

FIDO2 and WebAuthn: how it works

FIDO2 is challenge-response based. The server sends a random challenge, the YubiKey signs it with a private key stored on the device, and the server verifies the signature. The private key never leaves the YubiKey.


┌─────────────────────────────────────────────────────────────┐
│                  FIDO2 challenge-response                   │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│   ┌──────────────┐       ┌──────────┐      ┌──────────────┐ │
│   │  SSH client  │       │  YubiKey │      │  SSH server  │ │
│   └──────┬───────┘       └────┬─────┘      └──────┬───────┘ │
│          │                    │                   │         │
│          │ Challenge (random) │                   │         │
│          │ ──────────────────>│                   │         │
│          │                    │  PIN / Touch      │         │
│          │                    │ ────────── (User) │         │
│          │                    │                   │         │
│          │ Signature(Chall.)  │                   │         │
│          │ <───────────────── │                   │         │
│          │                    │                   │         │
│          │ Challenge + sig.   │                   │         │
│          │ ──────────────────────────────────────>│         │
│          │                    │                   │         │
│          │                    │ Verify(PubKey,Sig)│         │
│          │                    │ ───────────────── │         │
│          │                    │                   │         │
│          │ Access granted     │                   │         │
│          │ <──────────────────────────────────────│         │
│                                                             │
└─────────────────────────────────────────────────────────────┘

What happens in detail:

  1. The SSH client detects that a FIDO2 key is configured
  2. It asks the YubiKey to sign the challenge
  3. The user enters their PIN or touches the sensor
  4. The YubiKey signs the challenge with the ed25519-sk key pair
  5. The server verifies the signature against the stored public key
  6. Login is granted

The decisive difference from static keys: the challenge is different on every login attempt. Even if someone intercepts the signature, they cannot reuse it (replay protection).

YubiKey hardware setup

For SSH with FIDO2, every YubiKey 5-series with USB-C or USB-A is suitable. What matters is that the key supports FIDO2 and not only U2F — you see that from the product name "YubiKey 5" (not "YubiKey Security Key", which only does U2F).

Initialise the YubiKey:


# Install YubiKey Manager (Debian/Ubuntu)
sudo apt install yubikey-manager -y

# Set the FIDO2 PIN (default PIN is empty)
ykman fido access change-pin

The PIN protects the YubiKey against unauthorised use. Choose a PIN with at least 6 characters. Without a PIN, anyone with physical access to the YubiKey can read and use the FIDO2 keys.

Verification:


# Check whether the YubiKey is detected
ykman fido info

Expected output:


Applications: FIDO2
    slots: 2
     pin: true
     pin retries: 3

⚠️ Backup YubiKey and break-glass emergency anchor: Always create at least a second YubiKey as a physical backup in a safe place. If you lose your only YubiKey, you are locked out — there is no reset mechanism for FIDO2 keys. For root and cloud servers, also protect the hoster's out-of-band web console (for example Hetzner Robot, Proxmox NoVNC or IPMI) with a separate 2FA as an emergency anchor.

Generate SSH keys with FIDO2

OpenSSH 8.2+ supports FIDO2 keys natively. The ssh-keygen command with -t ed25519-sk creates a key that is stored on the YubiKey.

Generate the key:


# Generate a FIDO2 SSH key
ssh-keygen -t ed25519-sk -C "admin@server"

During generation you are asked to touch the YubiKey and enter your FIDO2 PIN. The private key is stored on the YubiKey, not on disk.

What happens during generation:

  1. ssh-keygen creates an ed25519-sk key pair
  2. The private key is placed in the YubiKey's secure element
  3. The public key is stored in ~/.ssh/id_ed25519-sk.pub
  4. Every login attempt requires the YubiKey

Options:


# Without touch (PIN only)
ssh-keygen -t ed25519-sk -O no-touch-required

# With a custom filename
ssh-keygen -t ed25519-sk -f ~/.ssh/id_yubikey -C "admin@server"

💡 Resident vs. non-resident keys: With -O resident the private key is stored fully on the YubiKey and can be exported from there. Without that option only a handle is stored. For SSH, non-resident (the default) is recommended, because the key is then not exportable.

Copy the public key to the server:


# Transfer the public key to the server
ssh-copy-id -i ~/.ssh/id_ed25519-sk.pub user@server

If ssh-copy-id has trouble with FIDO2 (sometimes the case on older OpenSSH versions), copy the key manually:


# Manual installation
cat ~/.ssh/id_ed25519-sk.pub | ssh user@server "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

Check on the server:


# Check the installed key
cat ~/.ssh/authorized_keys | grep "sk@"

The line should start with ssh-ed25519-cert-v01@openssh.com or contain sk@ — that confirms it is a FIDO2 key.

Configure the SSH server for FIDO2

The SSH server must know that it accepts FIDO2 keys. That is already true in the default configuration, but we want to be explicit and set additional security settings.

Server configuration:


sudo nano /etc/ssh/sshd_config

Relevant settings:


# Allow FIDO2 keys (default: yes)
PubkeyAuthentication yes

# Allow SSH keys only (disable passwords)
PasswordAuthentication no

# Forbid root login
PermitRootLogin no

# Maximum of 3 authentication attempts
MaxAuthTries 3

# Cap grace time at 30 seconds
LoginGraceTime 30

# Minimise information disclosure
DebianBanner no
Banner none

# Keepalive for long-running sessions
ClientAliveInterval 300
ClientAliveCountMax 2

Optional: allow only FIDO2 keys (no more static keys):

If you want to use FIDO2 keys exclusively, you can restrict PubkeyAuthOptions:


# Accept only verified FIDO2 keys
PubkeyAuthOptions verify-required

⚠️ Caution with verify-required: This option accepts only FIDO2 keys with user verification (PIN or biometrics). Static Ed25519 and RSA keys are rejected. Test the change first in an additional session before you close the existing one.

Restart the SSH server:


sudo systemctl restart sshd

⚠️ Never close the active session before you have tested! Open a second terminal and test login with the new FIDO2 key. Only when that login works do you close the old session.

Test:


# Login with the FIDO2 key
ssh -i ~/.ssh/id_ed25519-sk user@server

# Check which method was used
ssh -v user@server 2>&1 | grep "Offering"

The verbose output should show Offering public key: .../id_ed25519-sk and Accepted publickey as the result.

Certificate-based authentication

For larger environments an SSH certificate authority (CA) pays off. Instead of distributing individual keys on every server, a CA signs public keys and the server trusts only the CA.

Create the CA on the signing host:


# Generate the CA key pair
sudo mkdir -p /etc/ssh/ca
sudo ssh-keygen -t ed25519 -f /etc/ssh/ca/server-ca -C "SSH CA"

Distribute the public CA key to all servers:


# On every server: register the CA public key as a TrustedKey
echo "cert-authority $(cat /etc/ssh/ca/server-ca.pub)" >> /etc/ssh/authorized_keys.d/trusted-ca

Sign a user key with the CA:


# Sign the FIDO2 key (creates a certificate)
sudo ssh-keygen -s /etc/ssh/ca/server-ca \
    -I "admin-fido2" \
    -n admin \
    -V +52w \
    ~/.ssh/id_ed25519-sk.pub

Options:

Option Meaning
-s CA private key for signing
-I Certificate ID (for logs)
-n Allowed usernames (comma-separated)
-V Validity (+52w = 52 weeks)

The generated certificate (id_ed25519-sk-cert.pub) is recognised automatically by the SSH client and sent along at login.

💡 Advantage of the CA: On every server you only need to place the public CA key. Individual user keys are managed centrally and can be revoked through the CA without editing authorized_keys on every server.

SSH config and ssh-agent

For daily use a clean SSH configuration on the client pays off.

~/.ssh/config:


# Server definition with FIDO2 key
Host server
    HostName 192.168.1.100
    User admin
    IdentityFile ~/.ssh/id_ed25519-sk
    IdentitiesOnly yes

# Jump host for internal servers
Host internal-server
    HostName 10.0.0.50
    User admin
    ProxyJump server
    IdentityFile ~/.ssh/id_ed25519-sk
    IdentitiesOnly yes

IdentitiesOnly yes forces the SSH client to use only the specified key. That prevents another key from the agent being offered by accident.

ssh-agent with FIDO2:

The ssh-agent can support FIDO2 keys, but because the private key stays on the YubiKey, only the handle is stored in the agent. For FIDO2 the agent makes little sense — the YubiKey has to be physically attached anyway.


# Start ssh-agent (optional for FIDO2)
eval $(ssh-agent -s)

# Add the key (touch is requested on every login)
ssh-add ~/.ssh/id_ed25519-sk

💡 Recommendation: Skip ssh-agent for FIDO2 keys. The agent brings no advantage, because the YubiKey must be touched on every login. Use IdentityFile in ~/.ssh/config instead.

Troubleshooting: common SSH-FIDO2 problems

Problem: "Key rejected by server"


# Check whether FIDO2 is supported on the SSH server
ssh -V  # OpenSSH_8.9 or higher

# Check the server configuration
sudo sshd -T | grep pubkeyauth

⚠️ Cause: OpenSSH version too old (below 8.2). FIDO2 support arrived with OpenSSH 8.2. Ubuntu 24.04 has OpenSSH 9.6, Ubuntu 26.04 has OpenSSH 9.7 — both support FIDO2.

Problem: "Sign_and_send_pubkey: signing failed: agent refused operation"


# ssh-agent is running and trying to sign the key
# Fix: specify the key explicitly or disable the agent
ssh -i ~/.ssh/id_ed25519-sk user@server

Cause: ssh-agent tries to sign the key with an old method. IdentitiesOnly yes in the SSH config solves the problem.

Problem: "Touch your authenticator" does not appear


# Check whether the YubiKey is detected
lsusb | grep Yubico

# Check whether the FIDO2 module is loaded
ls /dev/hidraw*

Cause: YubiKey not detected or drivers missing. On Ubuntu all required drivers are present by default.

Problem: "PIN auth required" but the PIN is not prompted


# Set or reset the FIDO2 PIN manually
ykman fido access change-pin

⚠️ Cause: The YubiKey was initialised without a PIN, or the PIN is outdated.

Verification on the server:


# Check logs
journalctl -u sshd -f

# Recognise successful FIDO2 logins
journalctl -u sshd | grep "Accepted publickey"

The logs show which key was used and whether it was a FIDO2 key (sk@).

CrowdSec: collaborative intrusion prevention

Fail2Ban vs. CrowdSec

Fail2Ban reads local log files and bans IPs through iptables/nftables rules. Each server acts in isolation — an attacker scanning from 1,000 IPs has to be banned on every server individually.

CrowdSec works differently: the agent analyses logs and decisions are shared centrally in a community database. When an attacker is banned on one community server, everyone else benefits. CrowdSec also separates analysis (agent) from enforcement (bouncer), which makes the architecture more flexible.

What changes:

Property Fail2Ban CrowdSec
Log analysis Local Local + community
Enforcement iptables directly Bouncer (external)
Performance Good with few jails Scalable for many servers
Community None Shared decisions
Dashboard None Web dashboard + metrics
Architecture Monolith Agent + bouncer split

💡 Core idea: CrowdSec collects signatures of attack patterns (scenarios) and compares them with your logs. Hits produce decisions (bans) that bouncers enforce. Community intelligence supplements local detection.

Installation and first-time setup

Install the CrowdSec agent:


# Add the repository (Debian/Ubuntu)
curl -s https://packagecloud.io/install/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash

# Install CrowdSec
sudo apt install crowdsec -y

Install the firewall bouncer:


# Bouncer for nftables/iptables
sudo apt install crowdsec-firewall-bouncer-nftables -y

⚠️ Watch the order: Install the agent first, then the bouncer. The bouncer connects to the agent through the local API. Do not start the bouncer before the agent is running.

Check status:


sudo systemctl status crowdsec
sudo systemctl status crowdsec-firewall-bouncer

Expected output for the agent:


● crowdsec.service - CrowdSec SIEM
     Loaded: loaded (/lib/systemd/system/crowdsec.service; enabled)
     Active: active (running) since Mon 2026-08-26 12:00:00 UTC

Check first decisions:


sudo cscli decisions list

In the initial state the list is empty. As soon as the agent analyses logs and recognises attack patterns, decisions appear here.

Scenarios and decisions

CrowdSec works with scenarios — signatures that describe particular attack patterns. The agent compares log entries with these scenarios and creates decisions on hits.

Scenarios enabled by default:


sudo cscli scenarios list
Name Service Type
ssh-bf ssh leak
ssh-slow-bf ssh leak
http-bad-user-agent http leak
http-dos-bf-path http leak

Scenarios relevant for SSH:

Scenario Detection Action
ssh-bf 5 failed logins in 10 min Ban for 1h
ssh-slow-bf Slow brute force over hours Ban for 24h
ssh-crawl Behavioural analysis of login attempts Ban for 4h

Create your own scenarios:

If the default scenarios do not fit, you can define your own:


sudo nano /etc/crowdsec/scenarios/custom-ssh-aggressive.yaml

type: leak
name: ssh-aggressive-scan
description: "Aggressive SSH scans with many failed logins"
labels:
  service: ssh
  remediation: true
filter:
  LABELS.service == "ssh" && MSG.ContainerName == "sshd"
groupby: LABELS.source_ip
capacity: 5
leakspeed: 1m
blackhole: 1m
certainty: 75
for: 10m

Verification:


# Reload scenarios
sudo cscli scenarios list | grep custom

# Simulate a test scenario
sudo cscli scenarios install --local /etc/crowdsec/scenarios/custom-ssh-aggressive.yaml

Bouncers (firewall integration)

Bouncers enforce the agent's decisions. There are different bouncer types for different use cases.

nftables bouncer (default):


# Check whether the bouncer is active
sudo cscli bouncers list

Name       │ IP Address │ Valid Until │ Type
───────────┼────────────┼─────────────┼──────
firewall   │ 127.0.0.1  │ 2027-01-01  │ nftables

Bouncer types:

Bouncer Use Advantage
nftables Server firewall Fast, close to the kernel
nginx Web server inline No extra port
caddy Caddy integration Simple configuration
crowdsec-openapi External APIs Flexible

Check firewall rules:


# Check the CrowdSec chain in nftables
sudo nft list chain inet crowdsec-crowdsec-block

table inet crowdsec-crowdsec {
    chain crowdsec-block {
        type filter hook input priority filter; policy accept;
        ip saddr @crowdsec-blacklist-jump drop
        ip6 saddr @crowdsec-blacklist6-jump drop
    }
}

Whitelisting

Not every IP should be banned. Local servers, monitoring systems or trusted networks need whitelisting.

Whitelist file:


sudo nano /etc/crowdsec/parsers/s02-enrich/whitelist.yaml

filter: "Meta.IP != ''"
whitelist:
  reason: "Local address"
  expression: "Ip('127.0.0.1') || Ip('::1')"
  expression: "Ip('192.168.0.0/16')"
  expression: "Ip('10.0.0.0/8')"

Alternatively: whitelist via cscli:


# Whitelist a single IP
sudo cscli decisions add --ip 192.168.1.100 --duration 0 --type whitelist --reason "Monitoring"

# Whitelist an IP range
sudo cscli decisions add --range 192.168.1.0/24 --duration 0 --type whitelist --reason "Local network"

⚠️ Check the whitelist before bans: Always check whether an IP might come from a local monitoring system or a trusted network before you analyse bans. Otherwise CrowdSec accidentally bans your own systems.

Dashboards and alerting

CrowdSec provides a web dashboard and metrics for monitoring systems.

Web dashboard:


# Show dashboard credentials
sudo cscli console status

The dashboard is reachable at http://localhost:6060 (default port). Adjust the firewall for external access.

Metrics for Prometheus:


# Check the metrics endpoint
curl -s http://localhost:6060/metrics | head -20

# Important metrics
crowdsec_decisions_total{source_ip="...",scenario="..."}
crowdsec_alerts_total{scenario="..."}
crowdsec_bouncers_total{type="..."}

Alerting via webhook:


sudo nano /etc/crowdsec/notifications/email.yaml

type: email
name: email_default
log_level: info
 # Email configuration
smtp_host: "smtp.example.com"
smtp_port: 587
smtp_username: "alert@example.com"
smtp_password: "YOUR_PASSWORD"
smtp_from: "crowdsec@example.com"
smtp_to:
  - "admin@example.com"
smtp_subject: "CrowdSec Alert: {scenario}"
format: |
  CrowdSec detected an attack.
  Scenario: {scenario}
  Source IP: {source_ip}
  Decision: {decision}
  Time: {created_at}

Verification:


# Check notifications
sudo cscli notifications list

# Send a test notification
sudo cscli alert inspect --id 1

Performance and scaling

For larger environments (more than 10 servers) a central CrowdSec instance pays off.

Architecture for multiple servers:


┌─────────────────────────────────────────────────────────────┐
│              CrowdSec multi-server architecture             │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│   ┌─────────┐        ┌─────────┐        ┌─────────┐         │
│   │ Server1 │        │ Server2 │        │ Server3 │         │
│   │ Agent   │        │ Agent   │        │ Agent   │         │
│   └────┬────┘        └────┬────┘        └────┬────┘         │
│        │                  │                  │              │
│        ▼                  ▼                  ▼              │
│   ┌─────────────────────────────────────────────────────┐   │
│   │                Local API (central)                  │   │
│   │                http://central:8080                  │   │
│   └──────────────────────────┬──────────────────────────┘   │
│                              │                              │
│                              ▼                              │
│   ┌─────────────────────────────────────────────────────┐   │
│   │                 Community database                  │   │
│   │                 (shared decisions)                  │   │
│   └─────────────────────────────────────────────────────┘   │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Point the local agent at the central API:


sudo nano /etc/crowdsec/config.yaml

api:
  client:
    insecure_skip_verify: false
  server:
    listen_uri: http://127.0.0.1:8080

💡 Performance: A single CrowdSec agent can process 10,000+ log entries per second without trouble. For most setups a central agent on a dedicated server is enough.

Troubleshooting

Problem: "crowdsec service not running"


# Check logs
sudo journalctl -u crowdsec -f

# Common cause: port 8080 occupied
sudo ss -tulnp | grep 8080

Problem: "Bouncer not connected"


# Check the bouncer API token
sudo cscli bouncers list

# Regenerate the token
sudo cscli bouncers add my-bouncer --auto

Problem: own IPs are being banned


# Check the whitelist
sudo cscli decisions list --type whitelist

# Add the local IP to the whitelist
sudo cscli decisions add --ip 192.168.1.100 --duration 0 --type whitelist --reason "Own server"

Verification:


# Check active decisions
sudo cscli decisions list

# Check the agent logs
sudo journalctl -u crowdsec | grep "decision"

System hardening

Kernel parameters (sysctl)

The Linux kernel exposes dozens of parameters through sysctl that shrink the attack surface. The defaults are tuned for compatibility, not security.

Create the file:


sudo nano /etc/sysctl.d/99-hardening.conf

Network security:


# Prevent IP spoofing
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Disable ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0

# Disable source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.default.accept_source_route = 0

# Enable SYN flood protection
net.ipv4.tcp_syncookies = 1

# Log martian packets
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1

Memory protection:


# Address space layout randomisation
kernel.randomize_va_space = 2

# Restrict dmesg (root only)
kernel.dmesg_restrict = 1

# Hide kernel pointers
kernel.kptr_restrict = 2

# Restrict BPF
kernel.unprivileged_bpf_disabled = 1

# Restrict user namespaces (for containers)
kernel.unprivileged_userns_clone = 0

Prevent kernel loading:


# Prevent loading further modules (after boot)
kernel.modules_disabled = 1

⚠️ kernel.modules_disabled = 1 prevents loading new kernel modules after boot. That can break NVIDIA drivers or VirtualBox modules, for example. Only enable it when you are sure every required module is loaded at boot.

Load the parameters:


sudo sysctl -p /etc/sysctl.d/99-hardening.conf

Verification:


# Check whether parameters are set
sysctl net.ipv4.conf.all.rp_filter
sysctl kernel.randomize_va_space

Expected output:


net.ipv4.conf.all.rp_filter = 1
kernel.randomize_va_space = 2

Filesystem hardening

By default many files and directories have permissions that are too open. /tmp, /var/tmp and /dev/shm in particular are attack surface.

Mount options for /tmp:


# /tmp is often already a tmpfs; check with:
mount | grep tmp

If /tmp sits on the main disk, a dedicated partition with safe mount options is recommended:


# Put this in /etc/fstab (with a separate /tmp partition)
# UUID=... /tmp ext4 defaults,noexec,nosuid,nodev 0 2

Set directory permissions strictly:


# Lock down critical directories
sudo chmod 700 /root
sudo chmod 600 /etc/shadow
sudo chmod 600 /etc/gshadow
sudo chmod 644 /etc/passwd
sudo chmod 644 /etc/group

# SSH directory
sudo chmod 700 /root/.ssh
sudo chmod 600 /root/.ssh/authorized_keys

Reduce SUID/SGID bits:


# Check SUID bits on standard tools
find / -perm -4000 -type f 2>/dev/null

Remove unnecessary SUID bits (examples):


# Only if the programs do not need to be SUID
sudo chmod u-s /usr/bin/newgrp
sudo chmod u-s /usr/bin/chsh
sudo chmod u-s /usr/bin/chfn

💡 Tip: find / -perm -4000 -type f shows every program with the SUID bit. Do not remove blindly — some programs need the bit for their function (for example sudo, passwd).

Service minimisation

Every running service is potential attack surface. On a fresh Ubuntu server dozens of services run that you may not need.

Check active services:


# Show every running service
systemctl list-units --type=service --state=running

# Services often unnecessary on a server
systemctl list-units --type=service --state=running | grep -E "cups|bluetooth|avahi|modem"

Disable unnecessary services:


# Printer service (not needed on a server)
sudo systemctl stop cups
sudo systemctl disable cups
sudo systemctl mask cups

# Bluetooth (not needed on a server)
sudo systemctl stop bluetooth
sudo systemctl disable bluetooth
sudo systemctl mask bluetooth

# mDNS/Avahi (service discovery — unnecessary on a server)
sudo systemctl stop avahi-daemon
sudo systemctl disable avahi-daemon
sudo systemctl mask avahi-daemon

mask vs. disable:

Action Effect
disable Service does not start automatically, but can still be started manually
mask Service cannot be started at all (symlink to /dev/null)

⚠️ Only mask what you truly do not need. If you later need a service after all, you first have to run systemctl unmask.

Check the systemd security score:


# Show the security score for a service
systemd-analyze security sshd.service

→ Overall exposure level for sshd.service: 9.6 OK

The lower the value, the better. sshd normally has a good score when PasswordAuthentication no and PermitRootLogin no are set.

Audit logging

auditd records security-relevant events and stores them in a tamper-resistant log file. That is the foundation for forensics and compliance.

Installation:


sudo apt install auditd -y
sudo systemctl enable auditd

Rules for SSH and sudo:


# Watch the SSH configuration
sudo auditctl -w /etc/ssh/sshd_config -p wa -k sshd_config

# Watch authorized_keys
sudo auditctl -w /root/.ssh/authorized_keys -p wa -k ssh_keys

# Watch the sudo configuration
sudo auditctl -w /etc/sudoers -p wa -k sudoers
sudo auditctl -w /etc/sudoers.d/ -p wa -k sudoers

# Watch crontab
sudo auditctl -w /etc/crontab -p wa -k cron
sudo auditctl -w /var/spool/cron/ -p wa -k cron

Persist the rules:


sudo nano /etc/audit/rules.d/hardening.rules

# SSH
-w /etc/ssh/sshd_config -p wa -k sshd_config
-w /root/.ssh/authorized_keys -p wa -k ssh_keys

# Sudo
-w /etc/sudoers -p wa -k sudoers
-w /etc/sudoers.d/ -p wa -k sudoers

# Cron
-w /etc/crontab -p wa -k cron
-w /var/spool/cron/ -p wa -k cron

# User changes
-w /etc/passwd -p wa -k user_change
-w /etc/shadow -p wa -k user_change
-w /etc/group -p wa -k group_change

Verification:


# List rules
sudo auditctl -l

# Search audits for a key
sudo ausearch -k sshd_config --interpret

# Create an audit report
sudo aureport --auth

File-integrity monitoring (AIDE)

AIDE (Advanced Intrusion Detection Environment) compares files against a known state and reports changes.

Installation:


sudo apt install aide -y

Initial scan:


# Initialise the database
sudo aideinit

⚠️ The initial scan can take time. On a large server the first scan can take 10-30 minutes. Do not run it during production operation.

Activate the database:


# Move the new database into place
sudo cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db

Cron-based check:


sudo nano /etc/cron.d/aide-check

# Daily check at 03:00
0 3 * * * root /usr/bin/aide --check | mail -s "AIDE Report" admin@example.com

Verification:


# Run a manual check
sudo aide --check

# Expected output on a clean system:
# AIDE found NO differences between database and filesystem.

Automatic security patches (unattended upgrades)

A purely manual patch routine leaves zero-day holes and known CVEs unprotected for days. Automatic security patches close critical vulnerabilities promptly, while needrestart restarts stale processes after library updates.

Installation and activation:


# Install automatic updates and the restart checker
sudo apt install unattended-upgrades needrestart -y

# Activate the configuration
sudo dpkg-reconfigure -plow unattended-upgrades

Adjust the configuration (/etc/apt/apt.conf.d/50unattended-upgrades):


// Install security updates automatically only
Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
};

// Automatically remove orphaned dependencies
Unattended-Upgrade::Remove-Unused-Dependencies "true";

// Automatic reboot on kernel updates at 04:00
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "04:00";

Verification:


# Dry run without changes
sudo unattended-upgrade --dry-run --debug

Mandatory access control (AppArmor)

AppArmor restricts applications (such as Nginx, Unbound or PHP-FPM) to the absolutely necessary filesystem and network access. Even if a service has a remote-code-execution vulnerability, the AppArmor profile prevents breakout into the system.

Check status and active profiles:


sudo aa-status

Expected output:


apparmor module is loaded.
XX profiles are in enforce mode.
0 processes are unconfined but have a profile defined.

Switch profiles to enforce mode:


# Activate profiles and enforce them strictly
sudo apt install apparmor-utils -y
sudo aa-enforce /etc/apparmor.d/*

💡 Tip: You can record new or custom profiles with aa-genprof <program> in interactive learning mode and then put them live with aa-enforce.

Network hardening

nftables: the modern firewall

nftables has replaced iptables as the default firewall manager. On Ubuntu 24.04 and 26.04, nftables is the kernel default. Anyone still using iptables is working through a compatibility layer — that works, but nftables is more efficient and has clearer syntax.

Basic structure:


# Show current tables
sudo nft list ruleset

table inet filter {
    chain input {
        type filter hook input priority filter; policy drop;
        # Rules here
    }
    chain forward {
        type filter hook forward priority filter; policy drop;
    }
    chain output {
        type filter hook output priority filter; policy accept;
    }
}

Create the firewall baseline:


sudo nano /etc/nftables.conf

#!/usr/sbin/nft -f

flush ruleset

table inet filter {
    chain input {
        type filter hook input priority filter; policy drop;

        # Allow established connections
        ct state established,related accept

        # Allow loopback
        iif lo accept

        # Allow ICMP (ping)
        ip protocol icmp accept
        ip6 nexthdr icmpv6 accept

        # Allow SSH (port 22)
        tcp dport 22 accept

        # Allow HTTP/HTTPS (if a web server)
        tcp dport { 80, 443 } accept

        # Logging for dropped traffic
        log prefix "nft-drop: " counter drop
    }

    chain forward {
        type filter hook forward priority filter; policy drop;
    }

    chain output {
        type filter hook output priority filter; policy accept;
    }
}

Start the service:


sudo systemctl enable nftables
sudo systemctl start nftables

⚠️ Watch the order: On remote servers always allow SSH (port 22) first, before you set the default policy to drop. Otherwise you lock yourself out.

Rate limiting for SSH:


# SSH rate limit: at most 5 connections per minute
sudo nft add rule inet filter input tcp dport 22 meter ssh-rate { limit rate 5/minute burst 5 packets } accept

Verification:


# Check active rules
sudo nft list ruleset

# Check connections
sudo ss -tulnp | grep -E "22|80|443"

# Check drops in the logs
sudo journalctl -k | grep "nft-drop"

CrowdSec + nftables integration

CrowdSec works with nftables through the firewall bouncer. The agent analyses logs and the bouncer enforces bans through nftables rules.

Check bouncer rules:


# Show CrowdSec-specific chains
sudo nft list chain inet crowdsec-crowdsec-block

table inet crowdsec-crowdsec {
    chain crowdsec-block {
        type filter hook input priority filter; policy accept;
        ip saddr @crowdsec-blacklist-jump drop
        ip6 saddr @crowdsec-blacklist6-jump drop
    }
}

Traffic flow:


┌─────────────────────────────────────────────────────────────┐
│                    Firewall traffic flow                    │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│   [Inbound traffic]                                         │
│          │                                                  │
│          ▼                                                  │
│   ┌──────────────────┐                                      │
│   │  CrowdSec check  │  ◀── Whitelist? ──▶ ACCEPT           │
│   │  (blacklist)     │  ◀── Ban? ────────▶ DROP + LOG       │
│   └────────┬─────────┘                                      │
│            │                                                │
│            ▼                                                │
│   ┌──────────────────┐                                      │
│   │  Firewall rules  │  ◀── SSH/HTTP ────▶ ACCEPT           │
│   │  (nftables)      │  ◀── Rest ────────▶ DROP             │
│   └────────┬─────────┘                                      │
│            │                                                │
│            ▼                                                │
│   ┌──────────────────┐                                      │
│   │  CrowdSec agent  │  ◀── Analyse logs                    │
│   │  (log analysis)  │  ◀── Create decisions                │
│   └──────────────────┘                                      │
│                                                             │
└─────────────────────────────────────────────────────────────┘

TLS configuration

For web servers, TLS 1.3 is the current standard. TLS 1.2 is still acceptable, but older versions (1.0, 1.1) should be disabled.

Let's Encrypt with Certbot:


# Install Certbot
sudo apt install certbot -y

# Request a certificate (Nginx)
sudo certbot --nginx -d example.com

# Request a certificate (Apache)
sudo certbot --apache -d example.com

TLS configuration for Nginx:


sudo nano /etc/nginx/snippets/ssl-params.conf

ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;
ssl_session_timeout 1d;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;

# OCSP stapling
ssl_stapling on;
ssl_stapling_verify on;

# Security headers
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains" always;
add_header X-Content-Type-Options nosniff always;
add_header X-Frame-Options DENY always;

Certificate auto-renewal:


# Cron job for automatic renewal
sudo crontab -e

# Daily at 02:30, renew the certificate if needed
30 2 * * * certbot renew --quiet --post-hook "systemctl reload nginx"

Verification:


# Test the TLS configuration
openssl s_client -connect example.com:443 -tls1_3

# Check certificate details
echo | openssl s_client -connect example.com:443 2>/dev/null | openssl x509 -noout -dates

DNS-over-HTTPS

By default the server sends DNS queries unencrypted over UDP port 53. Anyone on the path can see which domains you resolve. DNS-over-HTTPS (DoH) encrypts those queries.

Unbound as a local resolver:


sudo apt install unbound -y

Configuration:


sudo nano /etc/unbound/unbound.conf.d/hardening.conf

server:
    interface: 127.0.0.1
    access-control: 127.0.0.0/8 allow

    # Enable DNSSEC
    auto-trust-anchor-file: /var/lib/unbound/root.key

    # Privacy
    hide-identity: yes
    hide-version: yes

    # Performance
    msg-cache-slabs: 8
    rrset-cache-slabs: 8
    infra-cache-slabs: 8
    key-cache-slabs: 8

Forward upstream with Cloudflare DoH:


forward-zone:
    name: "."
    forward-addr: 1.1.1.1@853#cloudflare-dns.com
    forward-addr: 9.9.9.9@853#dns.quad9.net
    forward-tls-upstream: yes

Start the service:


sudo systemctl enable unbound
sudo systemctl start unbound

Verification:


# Test DNS resolution
dig example.com @127.0.0.1

# DNSSEC test
dig dnssec-failed.org @127.0.0.1
# Expectation: SERVFAIL (DNSSEC breaker recognised correctly)

# Test DNS-over-HTTPS
curl -v https://cloudflare-dns.com/dns-query?name=example.com 2>&1 | grep "HTTP/2"

💡 Recommendation: For servers with many clients (for example a homelab), Unbound as a local resolver is the best choice. You configure clients with 127.0.0.1 as DNS server; Unbound resolves upstream over DoH/DoT.

Automation

Ansible playbook for server hardening

The previous sections showed which configurations are needed. Running individual commands on every server is labour-intensive and error-prone. Ansible automates that — you describe the desired state, and Ansible applies it on all servers.

Project structure:


mkdir -p ~/ansible-hardening/{roles,inventory,group_vars}

Inventory:


nano ~/ansible-hardening/inventory/hosts.ini

[production]
server1 ansible_host=192.168.1.100
server2 ansible_host=192.168.1.101

[homelab]
server3 ansible_host=192.168.10.50

[all:vars]
ansible_user=admin
ansible_python_interpreter=/usr/bin/python3

Playbook:


nano ~/ansible-hardening/hardening.yml

---
- name: Linux server hardening
  hosts: all
  become: yes
  vars:
    ssh_port: 22
    allowed_users:
      - admin
    enable_crowdsec: true
    enable_aide: true

  tasks:
    # System updates
    - name: Update the system
      apt:
        update_cache: yes
        upgrade: dist

    # SSH hardening
    - name: Harden SSH configuration
      lineinfile:
        path: /etc/ssh/sshd_config
        regexp: "{{ item.regexp }}"
        line: "{{ item.line }}"
      loop:
        - { regexp: '^#?PermitRootLogin', line: 'PermitRootLogin no' }
        - { regexp: '^#?PasswordAuthentication', line: 'PasswordAuthentication no' }
        - { regexp: '^#?MaxAuthTries', line: 'MaxAuthTries 3' }
        - { regexp: '^#?LoginGraceTime', line: 'LoginGraceTime 30' }
      notify: restart sshd

    # Sysctl hardening
    - name: Set sysctl parameters
      sysctl:
        name: "{{ item.name }}"
        value: "{{ item.value }}"
        state: present
        reload: yes
      loop:
        - { name: 'net.ipv4.conf.all.rp_filter', value: '1' }
        - { name: 'net.ipv4.conf.all.accept_redirects', value: '0' }
        - { name: 'kernel.randomize_va_space', value: '2' }
        - { name: 'kernel.dmesg_restrict', value: '1' }

    # Unnecessary services
    - name: Disable unnecessary services
      systemd:
        name: "{{ item }}"
        state: stopped
        enabled: no
        masked: yes
      loop:
        - cups
        - bluetooth
        - avahi-daemon

    # Audit logging
    - name: Install auditd
      apt:
        name: auditd
        state: present

    - name: Deploy audit rules
      copy:
        src: files/audit-rules.rules
        dest: /etc/audit/rules.d/hardening.rules
      notify: restart auditd

  handlers:
    - name: restart sshd
      systemd:
        name: sshd
        state: restarted

    - name: restart auditd
      systemd:
        name: auditd
        state: restarted

Run the playbook:


cd ~/ansible-hardening
ansible-playbook -i inventory/hosts.ini hardening.yml

💡 Idempotence: Ansible is idempotent — you can run the playbook multiple times without the system changing unnecessarily. Only divergent states are corrected.

Verification:


# Check whether every server is reachable
ansible all -i inventory/hosts.ini -m ping

# Check SSH configuration
ansible all -i inventory/hosts.ini -m shell -a "sshd -T | grep permitrootlogin"

Expected output:


server1 | CHANGED | rc=0 >>
permitrootlogin no

server2 | CHANGED | rc=0 >>
permitrootlogin no

CIS scoring with OpenSCAP

CIS (Center for Internet Security) benchmarks define security standards for various systems. OpenSCAP checks your system against those benchmarks and produces a score.

Installation:


sudo apt install libopenscap8 scap-security-guide -y

Run the Ubuntu 24.04/26.04 benchmark:


# Show available profiles
oscap info /usr/share/xml/scap/ssg/content/ssg-ubuntu2404-ds.xml | grep "Profile"

# CIS Level 1 scan
sudo oscap xccdf eval \
    --profile cis_level1_server \
    --results cis-results.xml \
    --report cis-report.html \
    /usr/share/xml/scap/ssg/content/ssg-ubuntu2404-ds.xml

Open the report in the browser:


# Copy the report to the local machine and open it
scp admin@server:/tmp/cis-report.html ~/Desktop/

Interpret the score:

Score Meaning
100% All CIS recommendations met
80-99% Good state, smaller adjustments needed
60-79% Baseline configuration, some hardening missing
< 60% Critical gaps, urgent action required

Automated remediation (caution):


# Only if you understand the impact
sudo oscap xccdf generate fix \
    --profile cis_level1_server \
    --fix-type ansible \
    --output cis-remediation.yml \
    /usr/share/xml/scap/ssg/content/ssg-ubuntu2404-ds.xml

⚠️ Remediation only with intent: Automatic remediation can overwrite working configurations. Check every change before running it. When in doubt: manual and with judgement.

Verification:


# Check the score
oscap xccdf generate report cis-results.xml > /dev/null
grep -E "score|pass|fail" cis-results.xml | tail -10

Hardening checklist

Step Measure Effect
1 SSH: PasswordAuthentication no, PermitRootLogin no Brute force blocked
2 Set up FIDO2/YubiKey Passwordless, hardware-based auth
3 Install CrowdSec agent + bouncer Collaborative intrusion protection
4 Apply sysctl hardening Lock down kernel parameters
5 Mask unnecessary services Shrink attack surface
6 Enable auditd Log security events
7 Initialise AIDE File-integrity monitoring
8 Set up nftables firewall Filter network traffic
9 TLS 1.3 + security headers Harden the web server
10 Configure Unbound + DoH Encrypt DNS queries

Each step builds on the previous one. The order is deliberate: first the most critical weaknesses (SSH, CrowdSec), then system, then network layer.

Further Resources

Official documentation

OpenSSH FIDO2 CrowdSec documentation nftables wiki CIS Benchmarks OpenSCAP documentation

Tools and packages

YubiKey Manager CrowdSec Hub Certbot Unbound

Conclusion

Server hardening is not a one-off project, but an ongoing process. The measures described here — SSH with FIDO2, CrowdSec, kernel hardening, nftables and TLS — form the foundation every production Linux system should build on.

What matters is the starting point: not everything at once, but step by step. The hardening checklist above sets the order. With each step the attack surface shrinks, and the combination of several measures creates a robust security profile that does not give individual weaknesses the same damage potential.

For teams, automation with Ansible pays off — what is configured once can be transferred to any number of servers. CIS benchmarks with OpenSCAP set the score and show where there is still room to improve.

A hardened server is not a server that is secured against everything. It is a server where you know what is secured and what is not — and why.

Share & export

Export as Markdown

Related posts