It’s been a long time since we systematically updated our security measures at Earlham CS. I spent some time on that this week. I wanted to share some of the changes we made so that if you’re running a small-to-midsize network you might implement similar fixes.
The bare minimum
We’ve been using two critical and often unmentioned security measures already:
- physically locking down the data center
- running a network firewall
These two things alone do a lot to secure the system.
Securing services
Of course, we also provide a lot of services over the network, everything from web servers to shells. We have to secure access to all of those tools, plus our data. We want the necessary cracks in our firewall have as low a risk as possible of being exploited.
What remained, then, was the installation and configuration of server tools to harden security above and beyond physical locks and firewalls – in a word, “DevSecOps”.
First, on those machines that didn’t already have it, we installed unattended-upgrades (Debian/Ubuntu)/yum-cron (CentOS 7)/dnf-automatic (CentOS 8). We use these to automatically apply security patches to package-managed software. We’re still free to install larger updates each semester manually to minimize disruptions. It’s a good balance of stability and security vigilance.
Next we installed fail2ban on the small number of servers to which our firewall allows SSH access. It detects and blocks possibly-malicious IP addresses trying to connect to the servers. We enabled two “jails” in fail2ban: sshd, which catches likely bad actors attempting ssh connections and bans them for a short time; and recidive, which checks the log records from sshd (and potentially other jails), detects repeat offenders, and imposes longer-lasting bans against them.
(This is the digital equivalent of locking up your house so that the lazy would-be burglar going door-to-door checking knobs can’t get in.)
We then ran trufflehog on our public GitLab repos. It gave us a few warnings but none that actually contained compromising system or user information. I consider this good luck more than anything, and we’re taking steps now proactively to prevent such mistakes.
Still to come
Our next security steps will focus on improved monitoring and notification. This has been an issue in the past for stability, but fixing it will also contribute to security. We are also constantly reevaluating security approaches at a department policy level.
Thanks to this post for pointing me to some of the tools mentioned here.