Building ARM64 Docker Images on x86_64 Machines Using QEMU and Docker Buildx

Introduction

In the world of software development, the ability to build and deploy applications across different architectures is invaluable. This capability becomes particularly essential when dealing with ARM-based applications such as those for embedded systems or newer ARM-based servers. In this blog post, we will explore how to build ARM64 Docker container images on an x86_64 machine using QEMU emulation and Docker’s buildx tool.

Understanding the Challenge

The main challenge in building Docker images for a different architecture than your host machine lies in the architecture-specific binaries and dependencies. Directly running ARM binaries on an x86_64 machine is not possible without emulation due to differences in architecture instruction sets.

Introducing QEMU and binfmt_misc

To address this challenge, we use QEMU, a generic and open source machine emulator and virtualizer. When used as an emulator, QEMU can run operating systems and applications made for one machine (e.g., an ARM64 server) on a different machine (e.g., your x86_64 PC). By leveraging QEMU, developers can emulate ARM64 architectures on their x86 machines, facilitating cross-compilation and testing of ARM-specific applications.

What is binfmt_misc?

binfmt_misc is a capability of the Linux kernel that allows the kernel to recognize and parse various binary formats transparently to the user. It’s used to tell the kernel which program (in our case, QEMU) to invoke when executing binaries from foreign architectures. Essentially, binfmt_misc allows users to run executables for any architecture, as long as the appropriate emulator is installed on the system.

Step-by-Step Guide to Building ARM64 Docker Images

Step 1: Install Docker and Docker Buildx

Ensure Docker is installed on your machine along with the buildx plugin, which supports building multi-architecture images:

docker buildx version

If not present, set up a new buildx builder:

docker buildx create --name mybuilder
docker buildx use mybuilder
docker buildx inspect --bootstrap

Step 2: Set Up QEMU Emulation

Install the QEMU packages to add emulation capabilities for different architectures:

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

This command prepares Docker to emulate different architectures including ARM64.

Step 3: Prepare Your Dockerfile

Create a Dockerfile that will form the basis of your ARM64 container. Here’s a sample Dockerfile:

# Use an official ARM64 Python base image
FROM arm64v8/python:3.9-slim

# Install make, g++ and other dependencies
RUN apt-get update && apt-get install -y \
    make \
    g++ \
 && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY . /app

CMD ["python3", "your_script.py"]

Step 4: Build the Docker Image with Buildx

With everything set up, build your Docker image for ARM64:

docker buildx build --platform linux/arm64 -t yourname/yourimage:tag .

To check whether binfmt_misc support is enabled on your Linux system and to verify the associated entries in the /proc filesystem, you can follow these steps. The /proc filesystem is a special filesystem in Unix-like operating systems that presents information about processes and other system information in a hierarchical file-like structure.

Check the /proc filesystem

After ensuring that the binfmt_misc module is loaded, proceed to check the /proc/sys/fs/binfmt_misc/ directory. This is where the binfmt_misc entries are located:

ls /proc/sys/fs/binfmt_misc/

You should see several files including status, and possibly other entries if any specific binary formats have been registered. The status file indicates whether binfmt_misc is enabled. You can check its content with:

cat /proc/sys/fs/binfmt_misc/status

Step 3: Check registered binary formats

To see detailed information about each registered binary format, you can cat the specific files in the binfmt_misc directory. Each registered binary format has its own file, named after the identifier of the binary format. For example:

cat /proc/sys/fs/binfmt_misc/qemu-arm

This command would show the details for the ARM binary format if QEMU has registered it under that name. The output will include information like the type of binaries it supports, the interpreter (QEMU in this case), and any flags set.

Step 4: Test a binary execution (optional)

If you want to test whether the binary execution through binfmt_misc is working, you can try running a simple binary from the foreign architecture that you have set up through QEMU. For instance, running an ARM binary on an x86_64 machine where QEMU and binfmt_misc are configured to handle ARM binaries.

Challenges

Cross-compiling using qemu can be slow, it can take a huge amount of time to build other platform images on a different architecture host machine. So this might not be a practical solution, but if you use caching with docker images, several steps can be cached, and it can reduce the build time significantly.

To read mode about docker buildx, check official documentation : docker buildx | Docker Docs

Conclusion

Using QEMU and Docker buildx to build ARM64 images on an x86_64 host is a powerful solution for developers looking to ensure their applications are cross-platform compatible. This approach not only facilitates development and testing but also ensures that your deployment process is streamlined and efficient. By understanding and utilizing technologies like QEMU and binfmt_misc, developers can seamlessly develop for and deploy to a variety of architectures from a single machine.