This post is part of our series about container technology:
- What Are Namespaces and cgroups, and How Do They Work?
- Building Smaller Container Images (this post)
Containers are everywhere in the modern application landscape. Developers are using them in multiple ways – building them, pushing them to registries, and generally making applications work within containers.
In this blog, I explore the world of container images – specifically, how to make your container images smaller and why that’s important. Along the way, I show the code and commands for building a very small container image that can be used for testing.
What Are Container Images?
The best definition that I’ve seen of a container image is this one:
Images can exist without containers, but a container needs to run an image to exist.
This is somewhat circular, but still accurate. A container image is a computing object that contains application code and is “run” by a container runtime (such as Docker, rkt, and podman). Kubernetes is the most popular container orchestration system, but the three other tools mentioned can be used for orchestration if you’re doing local development.
The image defines how the application is deployed – for example, which ports to expose, the application runtime startup (or entry point), and so on.
Why Is a Smaller Image Better?
A smaller container image has three main advantages:
- Reduced build time for the application. The build time includes the container build, but also the time to push the container into a registry.
- A much smaller memory footprint. The smaller the image, the less memory you end up using. This may not be a concern when operating in a public cloud provider, but definitely is when working on a laptop for development purposes.
- A much smaller attack surface and fewer dependencies (especially if the container doesn’t use a base image). This makes for a reduced security surface and a smaller footprint in terms of extraneous libraries, dependencies, and other things inside your image.
A smaller container image typically has fewer components inside it. This means there is a reduced amount of non‑application code inside the image. The largest amount of “non application” code that you usually find inside a container image consists of shared libraries. Shared libraries are pieces of software that implement functionality you know is used (or likely to be used) by multiple applications. Using a shared library means the same functionality doesn’t have to be coded over and over in each new application. Under normal circumstances, shared libraries are a good thing because they externalize shared code and so enable application binaries to be smaller.
When running a single application in a container, shared libraries aren’t needed. After all, there’s no other application to share the code with!
Shared libraries take up space, and their life cycles need to be managed independently as part of the build process. Shared libraries are usually shipped with an operating system, and maintained quite separately from the applications that use them.
When operating within a container, there is no need to have this usual separation. Not separating out shared libraries means that, as a developer, you include only the components that your application needs to run – and nothing more. In a container context, this means no shared libraries.
Tools for Reducing Image Size
The traditional approach to building container images is to include and use a prebuilt operating system. If we use one of the more commonly used base images, we see something like this listing for Ubuntu (here divided across multiple lines for readability):
# podman images
REPOSITORY TAG IMAGE ID …
docker.io/library/ubuntu latest 9873176a8ff5 …
docker.io/library/fedora latest 055b2e5ebc94 …
registry.fedoraproject.org/f33/fedora-toolbox latest af1f279fed20 …
… CREATED SIZE
… 3 weeks ago 75.1 MB
… 7 weeks ago 184 MB
… 6 months ago 351 MB
As you can see, there’s quite a wide variation in image size: from just 75 MB for Ubuntu to a whopping 351 MB for the complete fedora-toolbox image. Each time one of these images is run, it takes time to start and load it, not counting the build time if you recompile an application and push the image into a registry.
There are two common choices for reducing image size: Alpine Linux and the Red Hat Universal Base Image (UBI).
Alpine Linux is based on the musl implementation of the C standard library (libc) and on BusyBox, a minimal kernel that nonetheless includes a plethora of tools. The use of musl libc means that you must recompile the application code to work with Alpine Linux, which is problematic if you don’t have access to the application’s source code.
UBI from Red Hat is another angle here. UBI is a standardized set of container images that have a set of runtimes for developers to use.
Using Alpine Linux usually results in a much smaller image, just 5.87 MB in this listing:
# podman images
REPOSITORY TAG IMAGE ID …
registry.access.redhat.com/ubi8/ubi latest 8215cb84fa58 …
registry.access.redhat.com/ubi8/ubi-minimal latest 3f32499d4f3a …
docker.io/library/alpine latest d4ff818577bc …
… CREATED SIZE
… 2 weeks ago 234 MB
… 2 weeks ago 105 MB
… 3 weeks ago 5.87 MB
Alpine and UBI address the same question (how do I build a smaller image?) but approach it from different starting points. Alpine starts with a very small codebase, and only adds the tools that are required. UBI, on the other hand, starts with a larger operating system and strips it down to bare essentials.
Building Using a Minimal Image
Before you can build a minimal image, you of course need an application. For the purposes of this blog, I’ve written the following very simple application in C. (Actually, I first wrote this app for a colleague who wanted to spin up thousands of containers for Istio testing and so needed a tiny container image.)
Perhaps the most notable thing about it is that it doesn’t actually do anything! It just calls the pause() function and waits for a signal.
# more pausle.c
#include <unistd.h>
int main(void) {return pause(); }This may seem like an odd application to use as an example, but it’s good for illustrating my point about image size. Being so small, the application has very little effect on the container size.
Under normal circumstances, I run this gcc command to compile the app with several optimizations:
# gcc -Os -fdata-sections -ffunction-sections -fipa-pta -W1,–gc-sections -W1,-O1 -W1,–as-needed -W1,–strip-all pausle.c -o pausle-dynamicThe result is a very small binary (just 15 KB):
# ls -lh pausle-dynamic
-rwxr-xr-x. 1 root root 15K Jul 22 22:00 pausle-dynamicThis application is dynamically linked. That is, it requires shared libraries on the operating system in order for it to run. I can check this by running the ldd command.
# ldd pausle-dynamic
linux-vdso.so.1 (0x00007fffafbe3000)
libc.so.6 => /lib64/libc.so.6 (0x00007fb193983000)
/lib64/ld-linux-x86-64.so.2 (0x00007fb193b5d000)In order to build this into a container, I need to use a base image that contains the shared libraries. I’m using podman, and Dockerfile for builds as it’s somewhat ubiquitous.
# more Dockerfile
FROM registry.access.redhat.com/ubi8/ubi-minimal
ADD pausle-dynamic /
CMD [“/pausle-dynamic”]
I use the RedHat UBI minimal image (in STEP 1) and add my precompiled application with the command to run it (in STEP 3) when the container starts.
# podman build –tag=pausle-dynamic .
STEP 1: FROM registry.access.redhat.com/ubi8/ubi-minimal
STEP 2: ADD pausle-dynamic /
–> 344589591c7
STEP 3: CMD [“/pausle-dynamic”]
STEP 4: COMMIT pausle-dynamic
–> 1f72538cf84
1f72538cf84c10ae525e545fb5596840f09d277eccaffae46f6b6a3815339c8bThe podman images command shows that the new image is no bigger (105 MB) than the ubi-minimal base image, because my application only adds 15 KB.
# podman images
REPOSITORY TAG IMAGE ID …
localhost/pausle-dynamic latest 1f72538cf84c …
registry.access.redhat.com/ubi8/ubi-minimal latest 3f32499d4f3a …
docker.io/library/alpine latest d4ff818577bc …
… CREATED SIZE
… About a minute ago 105 MB
… 4 weeks ago 105 MB
… 5 weeks ago 5.87 MB
When I run this image, I can see that it is available and running.
# podman run -d pausle-dynamic
5905d1ae4dc00b37f47ae4dlef4c7d99d8d5e1bd781da3b1decc436b10f5663b
# podman ps -a
CONTAINER ID IMAGE COMMAND …
5905d1ae4dc0 localhost/pausle-dynamic:latest /pausle-dynamic …
… CREATED STATUS
… 4 seconds ago Up 4 seconds agoI can execute a shell inside the new container and look at my binary:
# podman exec -it 5905d1ae4dc0 /bin/bash
# ldd pausle-dynamic
linux-vdso.so.1 (0x00007ffd89762000)
libc.so.6 => /lib64/libc.so.6 (0x00007fa65a658000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa65aa1d000)The application is the same one that I compiled and built into the container. Because it’s dynamically linked, it requires shared libraries in the operating system to run. This means that my application needs a base image containing these libraries in order to run. This increases the size of my container, so it’s still not as minimal as I want.
Building Without an Image
I can do two things to make this container image a lot smaller. I can statically link my code, meaning that my application “bundles in” the shared libraries into the binary, for want of a better description.
I run this command to statically link:
# gcc -Os -s static -ffunction-sections -fipa-pta -W1,–gc-sections pausle.c -o pausle-dynamic
# strip pausle-static
# ls-lh pausle-static
-rwxr-xr-x. 1 root root 697K Jul 22 22:31 pausle-staticThe ls command shows that the resulting application binary is 697 KB – considerably larger than the dynamically linked application – because the libraries are bundled into the application.
Now when I run the ldd command to display the shared libraries, I get a message telling me that the executable is not dynamic.
# ldd pausle-static
not a dynamic executableIn the Dockerfile, I’m using the special no‑op keyword scratch to indicate I’m not using a base image.
# more Dockerfile
FROM scratch
ADD pausle-static /
CMD [“/pausle-static”]
I now build an image in the same way as before by running the podman build command:
# podman build –tag=pausle-static .
STEP 1: FROM scratch
STEP 2: ADD pausle-static /
–> 7fb16e85314
STEP 3: CMD [“/pausle-static”]
STEP 4: COMMIT pausle-static
–> f7f7c833975
f7f7c83397545aef51e0ad665def03040d5a06adf50651133184864bd1adaed4The container builds in exactly the same way as the dynamic executable, but the resulting image is much smaller than the dynamic image – only 716 KB, which is barely bigger than the statically compiled binary itself (697 KB).
# podman images
REPOSITORY TAG IMAGE ID …
localhost/pausle-static latest f7f7c8339754 …
localhost/pausle-dynamic latest 1f72538cf84c …
… CREATED SIZE
… 2 minutes ago 716 kB
… 20 minutes ago 105 MB
I initialize the container and confirm it’s running:
# podman run -d pausle-static
1748d59be199a797aa01f569b10445787d3f40439fb0b404b22e4226f4f44e09
# podman ps -a
CONTAINER ID IMAGE COMMAND …
1748d59be199 localhost/pausle-static:latest /pausle-static …
… CREATED STATUS
… 5 seconds ago Up 6 seconds agoIf I try to exec a shell, or run the ls command inside the container, I get error messages indicating that there are no other applications in the container. This is because the container doesn’t include an operating system or base image.
# podman exec -it 1748d59be199 /bin/bash
Error: executable file `/bin/bash` not found in $PATH: No such file or directory: OCI not found
# podman exec -it 1748d59be199 /bin/ls
Error: executable file `/bin/ls` not found in $PATH: No such file or directory: OCI not foundConclusion
Building small container images is useful in all sorts of scenarios – development and testing, for example. It significantly cuts down the build time of new images, including the time that it takes to push an image to a remote registry.
As mentioned before, smaller container images (especially if they do not use a base image) also have a much smaller attack surface and fewer dependencies, reducing their footprint in terms of extraneous libraries, dependencies, and other things inside your image. Mostly though, creating small images carries a sense of neatness and symmetry, making it an uber cool thing to do!