Fail2ban Setup to Stop Brute Force Logins on a Small Server

Open the auth log on any public Linux server and you will find thousands of failed SSH logins from addresses you have never seen. A fail2ban setup will not make that noise disappear, but it turns each persistent scanner into a short-lived firewall rule, and it takes about twenty minutes on a small Debian or Ubuntu box.

This guide walks through a fail2ban setup that a two or three person team can actually maintain: one config file, sensible ban times, a whitelist so you never lock yourself out, and a few checks to prove it works. I run this exact layout on a handful of 2 GB VPS instances that host mail relays and small web apps.

What a fail2ban setup actually does

Fail2ban is a log watcher. It reads files like /var/log/auth.log or the systemd journal, matches lines against regular expressions called filters, and counts failures per IP address. When an address crosses a threshold inside a time window, fail2ban runs an action, which is usually a firewall rule that drops traffic from that address for a set period.

Three numbers control almost everything:

maxretry: how many failures trigger a ban. The default is 5.
findtime: the window in which those failures must happen. The default is 10 minutes.
bantime: how long the ban lasts. The default is 10 minutes, which is far too short for most servers.

A combination of jail (the rule set for one service), filter and action is what you are configuring. You can run one jail for SSH and stop there, or add jails for Postfix, Dovecot, Nginx and anything else that writes useful failure lines.

Installing fail2ban on Debian or Ubuntu

On Debian 12 or Ubuntu 24.04, the package from the main repository is fine. Version 1.0 or newer handles the systemd journal properly, which matters because recent Debian images no longer write auth.log by default.

Install it: sudo apt install fail2ban. The package is under 1 MB and pulls in Python if it is missing.
Check which firewall backend you use. Run sudo nft list ruleset. If you see tables, you are on nftables. If you use ufw, fail2ban can call ufw instead.
Never edit jail.conf. Package upgrades overwrite it. Create /etc/fail2ban/jail.local instead, and fail2ban reads it after the defaults.

Here is a starting jail.local that I would hand to any small team:

[DEFAULT] section: bantime = 1h, findtime = 10m, maxretry = 5, backend = systemd, banaction = nftables-multiport
ignoreip = 127.0.0.1/8 ::1 plus the static IP of your office or bastion host
bantime.increment = true, with bantime.maxtime = 1w
[sshd] section: enabled = true, port = 22 or whatever port you actually use

Restart with sudo systemctl restart fail2ban, then run sudo fail2ban-client status sshd. You should see the jail listed with zero or more currently banned addresses. On a server that has been online for a day, it is common to see 20 to 60 bans within the first hour.

Picking ban times that match your risk

The incremental ban option is the single most useful setting in a fail2ban setup. With bantime.increment turned on, an address that gets banned a second time stays out for twice as long, then four times, up to the cap you set. A one hour first ban that grows to a week catches the scanners that come back every night without punishing a colleague who fat-fingers a password twice.

Service	maxretry	findtime	First bantime
SSH with keys only	3	10m	1h
Postfix SASL auth	5	30m	2h
Dovecot IMAP	5	10m	1h
Nginx basic auth	6	10m	30m

Mail login jails deserve a longer window because credential stuffing against SMTP is slow and patient. Bots often try one password every 5 to 8 minutes per address to stay under default thresholds. A 30 minute findtime catches that pattern.

The whitelist is not optional

The most common fail2ban support ticket is "I locked myself out." Put your fixed office IP, your VPN exit address and any monitoring service in ignoreip before the first restart. If your team works from home on dynamic addresses, a cheap WireGuard tunnel to a bastion host with a static IP is a better answer than a loose whitelist.

Why fail2ban is not your SSH defense

A lot of hardening guides treat fail2ban as the main protection for SSH. I disagree. If your server accepts password logins, fail2ban only slows an attacker down. A botnet with 10,000 addresses can try 50,000 passwords before every address is banned, and that is plenty for a weak password.

The real fix is to turn off password authentication entirely and use keys, which we covered step by step in our guide to SSH key authentication on a small Linux server. Once passwords are off, failed attempts against SSH cannot succeed at all. Fail2ban then becomes a log hygiene and resource tool: it keeps auth logs readable, reduces CPU spent on handshakes, and gives you a quick list of who is knocking.

Where fail2ban earns its keep is on services that must accept passwords, such as SMTP submission and IMAP for staff mailboxes. Those are the jails worth tuning carefully.

Testing that your jails really work

A jail that loads but never matches anything is worse than no jail, because it gives you false confidence. Test each one.

Run sudo fail2ban-regex systemd-journal sshd for journal backends, or point it at a log file. It prints how many lines matched the filter. Zero matches on a busy server means the filter or backend is wrong.
From a phone hotspot (not a whitelisted network), try to log in with a wrong user four times. Then check sudo fail2ban-client status sshd from your normal connection. The hotspot IP should be listed.
Unban it with sudo fail2ban-client set sshd unbanip ADDRESS.
Confirm the firewall rule exists with sudo nft list set inet f2b-table addr-set-sshd.

Add a simple alert as well. If your uptime monitor already watches the box, the setup described in uptime monitoring with Uptime Kuma can check that the fail2ban service is active, so you hear about it if a config error stops the daemon after an upgrade.

Keeping it maintained without effort

Small teams forget about fail2ban once it runs, and that is mostly fine. A few habits keep it healthy:

Store jail.local in the same repository as your other server config so changes are reviewed.
After every distribution upgrade, run fail2ban-client status and confirm each jail still loads.
Check the ban database size once a quarter. It lives at /var/lib/fail2ban/fail2ban.sqlite3 and should stay well under 50 MB with a one week maxtime.
Back it up with the rest of /etc. If you already run restic backups on a self hosted server, include /etc/fail2ban in the paths.

That is a complete fail2ban setup for a small server. It will not stop a determined attacker on its own, but combined with key-only SSH and strong mail passwords it removes most of the background noise and gives you a clear record of what is hitting your machines.

Frequently asked questions

Is fail2ban still useful if I only use SSH keys?

Yes, though for different reasons. With keys only, attackers cannot guess their way in, but each attempt still costs a handshake and fills your logs. Fail2ban cuts that traffic and keeps the logs readable when you need to investigate something real.

Does fail2ban work with IPv6?

Version 0.10 and later supports IPv6 bans, and the nftables actions handle both address families. Make sure your ignoreip line includes ::1 and any IPv6 address you connect from, or you can ban yourself over v6 while v4 still works.

Should I use fail2ban or CrowdSec?

Fail2ban is simpler, local only and well documented, which suits a team of two or three. CrowdSec shares threat data across users and has a nicer dashboard, but it adds more moving parts. Start with fail2ban and switch only if you need shared blocklists across many servers.