Compiling and Installing ModSecurity for NGINX Open Source

[blockquote author=”Ivan Ristić, creator of ModSecurity”]Web applications – yours, mine, everyone’s – are terribly insecure on average. We struggle to keep up with the security issues and need any help we can get to secure them.[/blockquote]

We all want to create secure applications that will never be breached. But the almost weekly news of a high‑profile company being hacked is a stark reminder of how challenging security really is. And with the prevalence of scanners, rootkits, and other malicious tools, it’s easier than ever for anyone with even minimal technical knowledge to begin hacking websites. Though getting breached may feel like an inevitability, we should still take all the precautions we can to protect our apps and data.

A great tool for securing applications is ModSecurity, used by over a million sites around the world. It protects against a broad range of Layer 7 attacks, such as SQL injection (SQLi), local file inclusion (LFI), and cross‑site scripting (XSS), which together accounted for 95% of known Layer 7 attacks in Q1 2017, according to Akamai. Best of all, ModSecurity is open source.

The latest version, ModSecurity 3.0, breaks new ground with a modular architecture that runs natively in NGINX. Previous versions worked only with the Apache HTTP Server. We recently released ModSecurity 3.0 as a dynamic module for NGINX Plus, but as of this writing there is no prebuilt ModSecurity dynamic module for NGINX Open Source. In this blog we show how to create a ModSecurity 3.0 dynamic module for use with NGINX Open Source.

Learn even more about ModSecurity and NGINX in our ebook: ModSecurity 3.0 and NGINX: Quick Start Guide

Installation Overview

In NGINX 1.11.5 and later, you can compile individual dynamic modules without compiling the complete NGINX binary. After covering the compilation process step by step, we’ll explain how to load the ModSecurity dynamic module into NGINX and run a basic test to make sure it’s working.

1 – Install NGINX from Our Official Repository

If you haven’t already, the first step is to install NGINX. There are multiple ways to install NGINX, as is the case with most open source software. We generally recommend you install NGINX from the mainline branch in our official repository. For more details on how to properly install NGINX from our official repository, see our on‑demand webinar NGINX: Basics and Best Practices.

The instructions in this blog assume that you have installed NGINX from our official repository. They might work with NGINX as obtained from other sources, but that has not been tested.

Note: NGINX 1.11.5 or later is required.

2 – Install Prerequisite Packages

The first step is to install the packages required to complete the remaining steps in this tutorial. Run the following command, which is appropriate for a freshly installed Ubuntu/Debian system. The required packages might be different for RHEL/CentOS/Oracle Linux.

$ apt-get install -y apt-utils autoconf automake build-essential git libcurl4-openssl-dev libgeoip-dev liblmdb-dev libpcre++-dev libtool libxml2-dev libyajl-dev pkgconf wget zlib1g-dev

3 – Download and Compile the ModSecurity 3.0 Source Code

With the required prerequisite packages installed, the next step is to compile ModSecurity as an NGINX dynamic module. In ModSecurity 3.0’s new modular architecture, libmodsecurity is the core component which includes all rules and functionality. The second main component in the architecture is a connector that links libmodsecurity to the web server it is running with. There are separate connectors for NGINX, Apache HTTP Server, and IIS. We cover the NGINX connector in the next section.

To compile libmodsecurity:

Clone the GitHub repository:

$ git clone –depth 1 -b v3/master –single-branch https://github.com/SpiderLabs/ModSecurity

Change to the ModSecurity directory and compile the source code:

$ cd ModSecurity

$ git submodule init

$ git submodule update

$ ./build.sh

$ ./configure

$ make

$ make install

$ cd ..

The compilation takes about 15 minutes, depending on the processing power of your system.

Note: It’s safe to ignore messages like the following during the build process. Even when they appear, the compilation completes and creates a working object.

fatal: No names found, cannot describe anything.

4 – Download the NGINX Connector for ModSecurity and Compile It as a Dynamic Module

Compile the ModSecurity connector for NGINX as a dynamic module for NGINX.

Clone the GitHub repository:

$ git clone –depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git

Determine which version of NGINX is running on the host where the ModSecurity module will be loaded:
```
$ nginx -v

nginx version: nginx/1.13.1
```
Download the source code corresponding to the installed version of NGINX (the complete sources are required even though only the dynamic module is being compiled):
```
$ wget http://nginx.org/download/nginx-1.13.1.tar.gz

$ tar zxvf nginx-1.13.1.tar.gz
```

Compile the dynamic module and copy it to the standard directory for modules:

$ cd nginx-1.13.1

$ ./configure –with-compat –add-dynamic-module=../ModSecurity-nginx

$ make modules

$ cp objs/ngx_http_modsecurity_module.so /etc/nginx/modules

$ cd ..

5 – Load the NGINX ModSecurity Connector Dynamic Module

Add the following load_module directive to the main (top‑level) context in /etc/nginx/nginx.conf. It instructs NGINX to load the ModSecurity dynamic module when it processes the configuration:

load_module modules/ngx_http_modsecurity_module.so;

6 – Configure, Enable, and Test ModSecurity

The final step is to enable and test ModSecurity.

Set up the appropriate ModSecurity configuration file. Here we’re using the recommended ModSecurity configuration provided by TrustWave Spiderlabs, the corporate sponsors of ModSecurity.

$ mkdir /etc/nginx/modsec

$ wget -P /etc/nginx/modsec/ https://raw.githubusercontent.com/SpiderLabs/ModSecurity/v3/master/modsecurity.conf-recommended

$ mv /etc/nginx/modsec/modsecurity.conf-recommended /etc/nginx/modsec/modsecurity.conf

To guarantee that ModSecurity can find the unicode.mapping file (distributed in the top‑level ModSecurity directory of the GitHub repo), copy it to /etc/nginx/modsec.
```
$ cp ModSecurity/unicode.mapping /etc/nginx/modsec
```
Change the SecRuleEngine directive in the configuration to change from the default “detection only” mode to actively dropping malicious traffic.
```
$ sed -i ‘s/SecRuleEngine DetectionOnly/SecRuleEngine On/’ /etc/nginx/modsec/modsecurity.conf
```
Configure one or more rules. For the purposes of this blog we’re creating a single simple rule that drops a request in which the URL argument called testparam includes the string test in its value. Put the following text in /etc/nginx/modsec/main.conf:
```
# From https://github.com/SpiderLabs/ModSecurity/blob/master/

# modsecurity.conf-recommended

#

# Edit to set SecRuleEngine On

Include “/etc/nginx/modsec/modsecurity.conf”
# Basic test rule

SecRule ARGS:testparam “@contains test” “id:1234,deny,status:403”
```
In a production environment, you presumably would use rules that actually protect against malicious traffic, such as the free OWASP core rule set.

Add the modsecurity and modsecurity_rules_file directives to the NGINX configuration to enable ModSecurity:

server {

    # …

    modsecurity on;

    modsecurity_rules_file /etc/nginx/modsec/main.conf;

}

Issue the following curl command. The 403 status code confirms that the rule is working.

$ curl localhost?testparam=test

<html>

<head><title>403 Forbidden</title></head>

<body bgcolor=”white”>

<center><h1>403 Forbidden</h1></center>

<hr><center>nginx/1.13.1</center>

</body>

</html>

Conclusion

ModSecurity is one of the most trusted and well‑known names in application security. The steps outlined in this blog cover how to compile ModSecurity from source and load it into NGINX Open Source.

For large‑scale production deployments we recommend the NGINX ModSecurity WAF module for NGINX Plus, which we recently released as a prebuilt dynamic module. This provides a number of benefits:

You don’t need to compile the ModSecurity dynamic module yourself.
We have extensively tested our dynamic module, so you know it’s suitable for production usage.
ModSecurity 3.0, though stable, is new and still being actively developed. We continually track changes and update the module for every important change and security vulnerability.
Each new release of NGINX Plus includes a version of the dynamic module built against the corresponding NGINX Open Source version, so you can upgrade without having to compile anything yourself.

If you’re interested in a free trial of NGINX ModSecurity WAF, please contact our sales team.

[Editor – NGINX ModSecurity WAF officially went End-of-Sale as of April 1, 2022 and is transitioning to End-of-Life effective March 31, 2024. For more details, see F5 NGINX ModSecurity WAF Is Transitioning to End-of-Life on our blog.]