Linux Server Hardening

With all you rely on your Linux server for, you need your system to be as secure as possible. That means taking steps to keep the kernel up to date, enhance user security, and control daemons and other processes. Here are some essential steps to take to harden your Linux servers.

The default installation of any modern Linux distribution is relatively safe and secure already. Make sure you choose strong passwords for your administrator and user accounts, and update the system software as soon as it is connected to the Internet so you can run the latest versions with all security patches in place.

You should also encrypt your disk partitions. Even if your server is in a secure datacenter today, you never know what may happen to its hard drives in an year or two. Remember the story about NASA selling hard drivers with sensitive information on them? Major Linux distributions have options for encrypting partitions without installing third-party software: Check the Red Hat howto, which also covers distributions such as CentOS, or learn more about the package ecryptfs-utils in Debian.

Make sure you mount disk partitions securely. Remove execute and write options where possible. Partitions like /tmp, for instance, don't need to be executable. If you have configured /tmp to be a separate partition, you can remount it as non-executable with the command:

mount -o remount,noexec /tmp

If you're running a web server and your scripts don't change often, you can make it more secure by configuring the partition it's on as read-only. If the webroot (/var/www/html) has been set as a separate partition, you can mount it as read-only with the command:

mount -o remount,ro /var/www/html

If you mount the webroot as read-only, ensure that all sessions and cache files are saved to a different directory, such as /tmp, or your web applications will not work correctly.

Once your partitions are set, it's time to think about your system services. Start with the minimal possible installation, then add just the services you really need. The fewer packages you install, the fewer vulnerabilities to which you are exposed. Security professionals recommend leaving out X.org-related packages from server installations, so forget about fancy graphical tools – you can run them from client desktops. Also avoid installing development libraries, compilers, or kernel headers, which could be used for malicious purposes later.

Linux Kernel Hardening

Once you have a secure base installation, you can proceed with hardening the Linux kernel to protect your system from zero-day vulnerabilities, which cannot always be patched in time. Hackers commonly exploit such vulnerabilities to give regular users root privileges.

The most popular solution for hardening your kernel is the Grsecurity kernel patch, a collection of security enhancements that includes other projects, such as PaX. Grsecurity provides address space layout randomization (ASLR), role-based access control (RBAC), chroot options, advanced logging, and many other security-oriented options.

No popular Linux distributions let you install Grsecurity via binary package. You may find some third-party ready-to-use packages, but you're probably better off compiling and installing it from source.

When you install Grsecurity from source you are just patching the source of a default kernel with Grsecurity enhancements. Once you reboot, you'll see all Grsecurity configurations in the menu of the new kernel under Security options in the kernel configuration menu. The first option, Security Level, allows you to choose a predefined model – high, medium, low, or custom. Once you've chosen a security level the rest of Grsecurity options are adjusted. If you choose the low level of protection, your system will be less restrictive, with fewer linking and FIFO restrictions; the high protection level is very restrictive and may cause problems for your applications, as it adds additional /proc and ptrace restrictions and kernel stack randomization. If you decide to go with the custom option, ensure that you are well acquainted with all of Grsecurity (and PaX) options from the security level link above.

No matter which security level you choose, some options are not preconfigured. For instance, you can enable Sysctl in the Sysctl menu in the kernel configuraion to let you to change Grsecurity-related parameters in real time, which is helpful for fine-tuning your system and debugging problems.

Advanced Security Through Access Control

Probably the most important and complex task in configuring Grsecurity is managing RBAC. When you browse through the Grsecurity options you can enable or disable RBAC. Only after you recompile and boot the new kernel will you be able to configure and work with RBAC. Grsecurity's main configuration tool, gradm, lets you manage Grsecurity's RBAC system.

One of gradm's most useful features is its learning mode, which generates ready-to-use rules for RBAC based on the normal work of your system. In the most simple and straightforward use case for Grsecurity's RBAC, you:

1. Enable full learning mode (system-wide) with gradm -F -L /etc/grsec/learning.logs. At this time no restrictions are enforced.

1. Allow some time to ensure that the system has gone through all usual and normal operations. Then stop the learning mode with gradm -D.

1. Set the newly learned rules as the defaults with gradm -F -L /etc/grsec/learning.logs -O /etc/grsec/policy.

1. Start enforcing RBAC with gradm -E.

As an alternative to RBAC, you can implement mandatory access control (MAC) mechanisms. In RBAC, permissions are impersonal and are granted to user roles. By contrast, in MAC, administrators must give individual users permission to access certain resources. MAC is more robust and widely accepted than RBAC, in part because with MAC it's more practical and straightforward to define a security policy for a service user, such as Apache's user nobody.

The two best-known MAC implementations are SELinux (for Red Hat-based distributions) and AppArmor (for Ubuntu). Each ships with its respective distribution, so you don't have to worry about manual installations and updates.

SELinux is the best supported tool for advanced access control. It is backed by US National Security Agency, Network Associates, Secure Computing Corp., and a few other impressive corporations, as well as a strong community. It enforces MAC through Linux Security Modules (LSM) in the kernel.

Be careful when you're enabling SELinux lest you make your server unusable. If you have problems, you can disable SELinux before the kernel has booted if you add a parameter to the line for your operating system in GRUB's boot menu: enforcing=0.

To enable SELinux for the first time, set SELINUX=enforcing in /etc/selinux/config, then run fixfiles relabel to relabel the filesystem.

Once you've booted into a system running SELinux, you will at times run into various permission issues. For example, try executing touch /var/www/html/test.html, which tries to create a file test.html in /var/www/html/. If you have relabeled your system you will most probably get an "access denied" error. SELinux prints its logs by default in /var/log/audit/audit.log. For this event you will see an entry that tells you that SELinux will not allow you to create files in the requested directory:

type=AVC msg=audit(1316550115.905:90): avc: denied { associate } for pid=2161 comm="touch" name="test.html" scontext=root:object_r:unlabeled_t:s0 tcontext=system_u:object_r:fs_t:s0 tclass=filesystem

You can use the audit2allow tool to generate policies based on denied executions from the audit log. In this case, to be able to create a file test.html in /var/www/html, you can use the command grep 'test.html' /var/log/audit/audit.log | audit2allow -M test, then apply the new policy with semodule -i test.pp. You should then be able to create the file /var/www/html/test.html without problems.

This example shows a basic scenario for using SELinux, but there's more you can do. SELinux has a similar feature to Grsecurity's RBAC, and its learning mode is called permissive mode. To use it, set SELINUX=permissive in /etc/selinux/config. After you save the file and reboot, SELinux will only generate logs instead of taking any actions. Later, you can use the logs as shown above to generate policies.

Further Hardening

All of the above helps you secure Linux itself. You're in for more potential problems when you start exposing essential system services, such as SSH, HTTP, and mail, to the public. While hardening each service has specific steps, here are a few general principles:

• Encrypt traffic – the stronger the encryption, the better. If a service does not support encryption by default, try using stunnel to tunnel its traffic.

• Restrict administrative access for administrators. For this purpose change services' default ports, limit remote IP addresses, and block default users. For instance, for sshd, you can set PermitRootLogin to no and Port to 11111 in /etc/ssh/sshd_config. It is good practice to pick ports above 10000 for sensitive services because most vulnerability scanners scan only for the first 10000 ports by default.

• Deploy application firewalls, such as mod_security for Apache and GreenSQL for MySQL.

• Enforce strong user authentication policies. Specify minimum password strength, expiration, and logout time. Passwords are still among the weakest points in security. Recent security breaches at kernel.org and linux.com are suspected to have been caused by brute-forced passwords.

• Deploy intrusion prevention and detection system (IDS/IPS). Snort is the most widely used one; one emerging alternative is Suricata. Snort can be distributed over numerous systems and be managed centrally, which gives you an enhanced view of suspicious activities in larger deployments.

• If IDS/IPS is too complex or too elaborate for your needs, you can at least set up a honeypot and log suspicious remote IP addresses, which you can then redirect away from your sensitive services. For example, on the default SSH port you can use the Kojoney honeypot instead of genuine sshd and log attackers trying to connect to SSH.

All of the above can help make you safer – but at some cost. Excessive Linux hardening can cause you problems with both the operating system and applications. Make sure your security policies are applied correctly and are not too aggressive.

Hardening a Linux server is not a simple task, but it is an important one. Not every administrator has to go through the process, but if you work in industries such as financial services or government – and even if you don't – consider implementing at least some of these policies.

Do you know something worth sharing with Wazi readers? You should write about it. We pay for technical articles and articles on open source policy and governance.

Follow @openlogic
Follow @CloudSwing

This work is licensed under a Creative Commons Attribution 3.0 Unported License
.Follow @openlogic
Follow @OSCloudServices

This work is licensed under a Creative Commons Attribution 3.0 Unported License
.