NGINX Community Blog

Using NGINX as an Object Storage Gateway

—

by

Elijah Zupancic
in

Have you ever accidentally posted a link to file on an object store like Amazon S3 instead of your CDN and seen it go viral? Were you surprised by how high your cloud services bill shot up or did your cloud provider block access? If so, this blog is for you. We describe how you can front a private S3 bucket with NGINX as a caching gateway to put a layer of protection between your private bucket and the public Internet. There are two benefits: NGINX caches requests to your object store and prevents public discovery of its contents.

About Amazon S3 and Compatible Storage Products

Although proprietary to AWS, the Amazon S3 API has become the de facto standard interface for object storage systems, with many vendors offering compatible storage products – Backblaze, Digital Ocean, Google Storage, Minio, NetApp, Nutanix, Tencent Cloud, VMware, and Wasabi among them. Despite the broad support for the S3 API, specific S3 API implementations are often not well suited to non‑storage operations such as serving high‑volume public Internet traffic. As such, there is a common need for a layer in front of the S3 API to bridge the storage interface to another application or user‑specific interface. NGINX can act as this bridge and provide additional value because of its module ecosystem and configurability.

In this blog we show how to configure NGINX Open Source and NGINX Plus as a read‑only gateway to an S3‑compatible object store by exploring a fully functioning Docker‑based implementation hosted in GitHub (an example of a stand‑alone Ubuntu installation is also available). Although we are only covering NGINX as a gateway for an S3‑compatible service, you can extend the approach to any service that uses AWS API Request Signing.

Use Cases

You might be wondering, “My object store already serves files at massive scale. Why would I want to put another layer in front of it?”

The answer is simple: object stores like S3 do one thing well – store and retrieve objects. NGINX, on the other hand, is a highly configurable reverse proxy which provides additional functionality on top of the basic object storage gateway. As such, it can be a good solution for a diverse set of use cases, including the following.

Caching

Object storage systems are designed for the storage of massive amounts of data and aren’t optimized for the repeated retrieval of the same objects (such as files or binary data). It is generally considered a best practice to have a caching layer in front of an object store to prevent unnecessary read operations. Using NGINX as a cache lowers latency of object delivery, protects against outages, and reduces usage costs.

Security Control

Although S3 has its own authentication and policy‑based access system, it doesn’t necessarily accommodate your specific authentication needs. NGINX supports many different security controls, so as a gateway it can provide authentication and access control that meets those needs. You can even create signed links with the Secure Link module!

Access for Internal Applications

Implementing AWS signature‑based authentication is tricky, and many applications and platforms do not have client libraries that allow for easy communication with S3. With NGINX as a gateway for internal services accessing S3, those applications become able to access S3 resources easily, without having to generate AWS signatures.

Compression and Content Optimization

You may want to compress the data in your object store to save on storage costs, but still serve content to clients that do not support compression. In this case, you can configure NGINX to decompress data in real time before sending it.

Alternatively, you might store content on the object store uncompressed and want to reduce delivery time by compressing data before transmission. In that case, configure the built‑in Gzip module or Brotli dynamic module to reduce transfer time to end users.

NGINX also supports other types of real‑time modifications to content, like the Image-Filter dynamic module for transforming GIF, JPEG, and PNG images, and tools like PageSpeed for reducing page load time by automatically applying web performance best practices to pages and associated assets. (At the time of writing, the Pagespeed site says that Pagespeed is “undergoing incubation at The Apache Software Foundation”. It is in fact the same mature code developed by Google and referenced in Optimizing Your Website with the Google PageSpeed Dynamic Module for NGINX Plus.)

Security

With NGINX you can build another layer of security in front of the S3 API, for example to rate limit users who are making excessive download requests and to limit which object paths are accessible within a bucket. You can further protect the S3 API with an integrated web application firewall (WAF) like F5 NGINX App Protect WAF.

Application Gateway

It is common when serving content from a site to use the same domain name for all resources to satisfy security, branding, consistency, and portability requirements. As such, serving static content and dynamic content together from the same host is a common need. While serving static files from an object store in its role as an S3 API gateway, NGINX can also proxy and load balance requests for dynamic content originating in application servers.

Features and Limitations

The NGINX S3 gateway implementation discussed in this blog has the following features and limitations:

  • Supports AWS Signature Version 2 and Version 4
  • Supports GET and HEAD requests only
  • Supports NGINX Open Source and NGINX Plus
  • Supports a single bucket
  • Caches content for 1 hour
  • Not configured with SSL/TLS
  • No WAF is installed by default
  • Compression (for example, Gzip or Brotli) is not enabled by default
  • Proxy buffering settings are set to default values (you may want to modify these settings based on your workload and object size)

Configuring and Running the Gateway in Docker

You can use both NGINX Open Source and NGINX Plus as the gateway to S3 or a compatible object store.

To use NGINX Open Source, start with the official NGINX image from Docker Hub and the Dockerfile we provide on GitHub. Proceed to Creating the NGINX Configuration File.

Using NGINX Plus as the S3 API gateway has some compelling benefits:

  • Dynamic DNS resolution of upstreams using the NGINX Plus API – Whenever your S3 API domain changes its backing IP addresses, NGINX reconfigures automatically to use the new addresses. This is an important feature because Amazon S3 and other S3‑compatible APIs sometimes add or remove load balancers to better scale their system. When the NGINX configuration for upstreams is not reloaded after DNS changes, a service outage can result if all the backing IP address records are no longer valid.
  • Reduced time and resource consumption using the key‑value store – When generating AWS Version 4 authentication signatures, a portion of the cryptographic signature (signing key) is valid and cacheable for up to 24 hours. By storing that part of the signature in the key‑value store, the overall time and CPU resources needed to compute an authentication request is reduced. This is a useful performance enhancement for use cases with a high volume of requests. For sample configuration of a related use case, see Configuring the NGINX Plus Key‑Value Store for SSL Storage on our blog.

We do not distribute a Docker image for NGINX Plus, because the image needs to incorporate your NGINX Plus credentials. Follow the instructions in the next section to build an NGINX Plus Docker image with the Dockerfile we provide, which includes settings for downloading the NGINX Plus binary.

Building the NGINX Plus Gateway Docker Image

To build the NGINX Plus Docker image, perform these steps:

  1. Clone the NGINX S3 Gateway project from our GitHub repo:
    git clone https://github.com/nginxinc/nginx-s3-gateway.git
  2. Change directory to nginx-s3-gateway:
    cd nginx-s3-gateway
  3. Copy your NGINX Plus certificate and key (nginx-repo.crt and nginx-repo.key) to the plus/etc/ssl/nginx subdirectory. NGINX Plus customers can find them at the F5 customer portal; if you are doing a free trial of NGINX Plus, they were provided with your trial package.
  4. Build the container image. We recommend using Docker BuildKit, as it enables you to pass your NGINX Plus license files to the Docker build context without risk of exposing the data or having the data persist between Docker build layers.
    • To build using BuildKit, run this command:
      DOCKER_BUILDKIT=1 docker build -f Dockerfile.buildkit.plus -t nginx-plus-s3-gateway –secret id=nginx-crt,src=plus/etc/ssl/nginx/nginx-repo.crt –secret id=nginx-key,src=plus/etc/ssl/nginx/nginx-repo.key –squash .
    • To build without BuildKit, run this command:
      docker build -f Dockerfile.plus -t nginx-plus-s3-gateway .
  5. Continue to Creating the NGINX Configuration File.

Creating the NGINX Configuration File

To get started with running the gateway (either NGINX Open Source or NGINX Plus), you first need to create a configuration file with the settings for connecting NGINX with an S3 API upstream. You can find a sample configuration file in our GitHub repository. Let’s look at some sample configurations for different S3 API backends.

Amazon S3

Here is the configuration for using Amazon S3 as the backend. We’re using AWS Signature Version 4 because AWS is deprecating Version 2.

S3_BUCKET_NAME=my-bucket-name

S3_ACCESS_KEY_ID=PUBLICACCESSKEY4TEXT

S3_SECRET_KEY=ProtectThisPrivateKeyBecauseItIsASecret!

S3_SERVER=s3-us-west-2.amazonaws.com

S3_SERVER_PORT=443

S3_SERVER_PROTO=https

S3_REGION=us-west-2

AWS_SIGS_VERSION=4

ALLOW_DIRECTORY_LIST=true

Wasabi

Here is the configuration for using Wasabi as the S3‑compliant backend. Here we are using AWS Signature Version 2 because Wasabi maintains support for that authentication method, and it generally performs better.

S3_BUCKET_NAME=my-bucket-name

S3_ACCESS_KEY_ID=PUBLICACCESSKEY4TEXT

S3_SECRET_KEY=ProtectThisPrivateKeyBecauseItIsASecret!

S3_SERVER=s3.us-west-1.wasabisys.com

S3_SERVER_PORT=443

S3_SERVER_PROTO=https

S3_REGION=us-west-1

AWS_SIGS_VERSION=2

ALLOW_DIRECTORY_LIST=true

Starting Up the Gateway

Once you have saved your configuration to a file (here it is s3-settings.env in the current directory), you are ready to start the gateway using Docker.

  • To start the NGINX Open Source gateway in the foreground, run:
    docker run -it –env-file ./s3-settings.env -p8080:80 nginxinc/nginx-s3-gateway
  • To start the NGINX Plus gateway in the foreground, run:
    docker run -it –env-file ./s3-settings.env -p8080:80 nginx-plus-s3-gateway

Once you have verified that the image is working as expected, you may want to push it to your organization’s private Docker repository so that it can be used more widely. But:

Never upload your NGINX Plus images to a public repository such as Docker Hub. Doing so violates your license agreement.

After NGINX starts up, you can access the gateway at http://localhost:8080.

Customizing the Gateway

After becoming comfortable with running the NGINX S3 gateway in the default configuration, you can explore extending its configuration for your specific needs. The base NGINX Docker image is configured to automatically load any *.conf files found in the /etc/nginx/conf.d directory, making it relatively easy to add configuration for additional features.

For example, you can store your configuration for the Gzip compression module in a local directory under the path etc/nginx/conf.d/gzip_compression.conf. We provide a working example of customizing the NGINX S3 Gateway to work with Gzip compression in our GitHub repository.

To add configuration of additional features:

  1. Create a configuration file for the feature you want to support, for example gzip_compression.conf in the local etc/nginx/conf.d/ directory.
  2. Add these lines to your Dockerfile, to automatically load any files with the .conf extension from the local etc/nginx/conf.d directory.
    FROM nginxinc/nginx-s3-gateway
    
COPY etc/nginx/conf.d /etc/nginx/conf.d

Summary

We have shown how to use NGINX as a caching gateway in front of a private bucket in Amazon S3 or another compatible object store. This not only reduces traffic to your object store, but also puts a layer of protection between it and the public Internet, preventing unwanted public discovery of its contents.

To get started with NGINX Open Source, download the official NGINX image from Docker Hub and the Dockerfile we provide on GitHub.