How Containers Work: Deep Dive into Internals

Introduction

Containers have revolutionized modern software development and deployment, offering a lightweight, portable, and consistent environment for applications. From small microservices to large-scale enterprise applications, containers, exemplified by technologies like Docker, have become the de facto standard for packaging and running software. While many engineers use containers daily, a deep understanding of their underlying mechanisms is crucial for debugging complex issues, optimizing performance, and building robust, secure systems.

This guide aims to demystify containers by peeling back the layers and explaining how they function at a fundamental level. We’ll explore the core Linux kernel features that power containerization, trace the lifecycle of a container, and dissect its key components. By the end of this explanation, you will have a comprehensive understanding of how containers achieve their remarkable isolation and resource efficiency.

The Problem It Solves

Before containers gained widespread adoption, software deployment faced significant challenges, often summarized by the infamous phrase “it works on my machine.” Developers would build applications in their local environments, only for them to fail or behave differently when deployed to testing, staging, or production servers. This inconsistency stemmed from a myriad of factors: differing operating system versions, library dependencies, environmental configurations, and even subtle variations in system utilities.

Traditional solutions included:

• Manual Configuration: Tediously setting up each server to match the development environment, a process prone to human error and difficult to scale.
• Virtual Machines (VMs): VMs provided strong isolation by encapsulating an entire operating system (guest OS) on top of a hypervisor. While effective, VMs are resource-intensive, slow to start, and carry significant overhead due to running multiple kernels.

The core problem was the lack of a standardized, lightweight, and truly portable unit of software deployment that could reliably package an application and all its dependencies, ensuring consistent execution across any infrastructure. This is precisely the challenge containers address by providing a consistent execution environment without the heavy overhead of full virtualization.

High-Level Architecture

Containers fundamentally leverage the Linux kernel’s capabilities to isolate processes and resources. Unlike virtual machines that virtualize hardware and run entire guest operating systems, containers share the host operating system’s kernel. This shared kernel is the cornerstone of their efficiency and speed.

The primary components that enable containerization are:

• Namespaces: Provide process isolation, giving each container its own view of the system’s resources (e.g., process IDs, network interfaces, mount points).
• Control Groups (cgroups): Govern resource allocation and limits, ensuring containers don’t starve the host or other containers of CPU, memory, or I/O.
• Union Filesystems (e.g., OverlayFS): Enable efficient storage by layering filesystem changes, allowing multiple containers to share common base images and reducing disk space.

A container runtime, such as containerd (which Docker uses), orchestrates these kernel features. The Docker daemon interacts with containerd to manage container lifecycles, image management, networking, and volumes.

In this architecture, the Docker Daemon acts as a high-level API for users. It translates user commands (like docker run) into calls to the underlying container runtime. The runtime then uses Linux kernel features to create and manage the isolated environments for each container. Each container runs its processes directly on the host’s kernel but within its dedicated namespaces and cgroups, and with its own view of the filesystem provided by a union filesystem.

How It Works: Step-by-Step Breakdown

Let’s trace the execution flow when you run a command like docker run -p 8080:80 myapp:1.0.

Step 1: Image Pull and Layering

When you execute docker run, the Docker client first checks if the myapp:1.0 image exists locally. If not, it communicates with a Docker registry (like Docker Hub) to pull the image.

An image is not a single, monolithic file; it’s a collection of read-only layers. Each layer represents a change to the filesystem, like installing a package or adding a file. These layers are stacked on top of each other. Docker pulls these layers individually and caches them. This layering is crucial for efficiency: if multiple images share a base layer (e.g., an Ubuntu base image), that layer is only downloaded once.

# Example: Pulling an image
docker pull myapp:1.0
# Output might show layers being downloaded
# 1. Pulling from library/myapp
# 2. Digest: sha256:abcdef...
# 3. Status: Downloaded newer image for myapp:1.0

Step 2: Container Creation - Namespaces

Once the image is available, the container runtime starts the process of creating a container instance. This involves invoking the Linux kernel’s namespace features. The runtime uses syscalls like clone() with specific CLONE_NEW* flags to create new namespaces for the new process.

• PID Namespace (CLONE_NEWPID): The container’s processes get their own process ID (PID) numbering system, starting from 1. A process with PID 1 inside the container will have a different, higher PID on the host. This prevents processes inside the container from seeing or signaling processes outside their namespace.
• Network Namespace (CLONE_NEWNET): The container gets its own network stack, including network interfaces, IP addresses, routing tables, and firewall rules, isolated from the host’s network.
• Mount Namespace (CLONE_NEWNS): The container has its own view of the filesystem hierarchy. It sees only the mounts configured for it, typically the union filesystem provided by the image and any mounted volumes.
• UTS Namespace (CLONE_NEWUTS): The container can have its own hostname and domain name, independent of the host.
• IPC Namespace (CLONE_NEWIPC): The container gets its own Inter-Process Communication (IPC) resources (e.g., message queues, semaphores), preventing interference with host or other containers’ IPC.
• User Namespace (CLONE_NEWUSER): (Advanced) Allows mapping a user ID inside the container to a different user ID on the host, enhancing security.
• Cgroup Namespace (CLONE_NEWCGROUP): Isolates the view of cgroup hierarchies for processes inside the container.

# Conceptual shell command to illustrate namespace creation (not what Docker directly runs)
# This uses 'unshare' utility to create new namespaces for a command
# unshare --pid --mount --uts --ipc --net --fork bash
# Inside the new bash shell, 'ps aux' would show different PIDs, 'hostname' would be different

Step 3: Resource Management - Cgroups

After namespaces provide isolation, cgroups (Control Groups) are used to allocate and limit resources for the container. The container runtime creates entries in the /sys/fs/cgroup hierarchy for the new container.

• CPU: Limits the percentage of CPU time the container can use.
• Memory: Sets the maximum amount of RAM the container can consume.
• Block I/O: Controls access to block devices (disks).
• Network I/O: (Often managed by network drivers, but cgroups can also contribute).

These limits prevent a runaway process in one container from consuming all host resources and impacting other containers or the host itself.

# Example: Inspecting cgroups for a running Docker container
# First, find the container's ID
CONTAINER_ID=$(docker ps -lq)
# Then, look into the cgroup filesystem on the host
# The path often includes 'docker' and the full container ID
ls /sys/fs/cgroup/cpu/docker/$CONTAINER_ID/
cat /sys/fs/cgroup/cpu/docker/$CONTAINER_ID/cpu.cfs_quota_us # Example CPU quota

Step 4: Filesystem Setup - Union Filesystem

With namespaces and cgroups in place, the container’s filesystem is prepared. This is where Union Filesystems, most commonly OverlayFS on Linux, come into play.

Docker takes the read-only layers of the image and stacks them. On top of these, it adds a new, writable layer specific to the container. Any changes made by the container (e.g., creating files, modifying existing ones) are written only to this top-most writable layer. The original image layers remain untouched.

• Read-only layers: These are the immutable layers from the image.
• Writable layer (container layer): This is where all runtime changes occur.
• Copy-on-Write (CoW): When a container modifies a file that exists in a lower read-only layer, the file is first copied to the writable layer, and then the modification is applied to the copy. The original file in the lower layer remains unchanged. This mechanism saves disk space and allows efficient sharing of base image layers.

# Conceptual view of OverlayFS mount
# mount -t overlay overlay -o lowerdir=/var/lib/docker/overlay2/l1:/var/lib/docker/overlay2/l2,upperdir=/var/lib/docker/overlay2/upper,workdir=/var/lib/docker/overlay2/work /var/lib/docker/overlay2/merged

Step 5: Process Execution

Finally, the container runtime executes the specified command (e.g., /usr/local/bin/python app.py) inside the newly created and configured environment. This command runs as PID 1 within the container’s PID namespace. From its perspective, it’s the only process running and has full control over its isolated resources.

# Example Dockerfile entrypoint
# ENTRYPOINT ["python", "app.py"]
# CMD ["--port", "80"]

Step 6: Networking and Port Mapping

For networking, Docker typically creates a virtual bridge network on the host. Each container gets a virtual network interface (e.g., eth0) connected to this bridge. This allows containers to communicate with each other and with the outside world.

When you specify -p 8080:80, Docker configures network address translation (NAT) rules using iptables on the host. This rule forwards traffic from port 8080 on the host to port 80 inside the container’s network namespace.

# Conceptual iptables rule for port mapping
# iptables -t nat -A DOCKER -p tcp --dport 8080 -j DNAT --to-destination 172.17.0.2:80

Deep Dive: Internal Mechanisms

Mechanism 1: Linux Namespaces

Namespaces are the bedrock of container isolation. They segment global system resources into isolated subsets, making them appear unique to processes within that namespace. This is achieved via specific flags passed to the clone() system call when a new process is created, or by using unshare() on an existing process.

• PID Namespace: Each PID namespace has its own set of PIDs. The first process in a new PID namespace gets PID 1 and becomes the “init” process for that namespace. This process is responsible for reaping zombies. Processes in a child PID namespace are visible in the parent namespace but with different PIDs.
• Network Namespace: Provides a completely isolated network stack. This includes network devices, IP addresses, IP routing tables, /proc/net entries, and netfilter (iptables) rules. This means a container can have an IP address and open ports without conflicting with the host or other containers.
• Mount Namespace: Each mount namespace has an independent list of mount points. Changes to the filesystem hierarchy (mounting/unmounting) within a namespace are not visible outside it. This is fundamental for isolating the container’s filesystem view.
• UTS Namespace: Isolates the hostname and NIS domain name. A container can report a different hostname than the host.
• IPC Namespace: Isolates System V IPC objects (message queues, semaphores, shared memory segments) and POSIX message queues.
• User Namespace: Allows a process to have root privileges inside the namespace while being mapped to an unprivileged user on the host. This significantly enhances security by preventing container root users from having root privileges on the host. It’s often combined with other namespaces.
• Cgroup Namespace: Isolates the view of the cgroup hierarchy. Processes in a new cgroup namespace see a simpler, potentially empty, cgroup hierarchy, making it harder for them to escape or manipulate host cgroups.

// Simplified C code demonstrating clone() with namespaces
#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

#define STACK_SIZE (1024 * 1024)
static char child_stack[STACK_SIZE];

int child_function(void *arg) {
    printf("Inside child process (PID: %d)\n", getpid());
    // Change hostname in UTS namespace
    sethostname("my-container", 12);
    printf("Child hostname: %s\n", "my-container");

    // Try to mount a filesystem (will only affect this namespace)
    // For a real container, this would involve pivoting the root filesystem
    // For this simple example, we'll just demonstrate PID and UTS

    // Simulate some work
    sleep(2);
    printf("Child process exiting.\n");
    return 0;
}

int main() {
    printf("Host process (PID: %d)\n", getpid());
    char host_hostname[256];
    gethostname(host_hostname, sizeof(host_hostname));
    printf("Host hostname: %s\n", host_hostname);

    int flags = CLONE_NEWPID | CLONE_NEWUTS | CLONE_NEWNS | SIGCHLD;
    pid_t child_pid = clone(child_function, child_stack + STACK_SIZE, flags, NULL);

    if (child_pid == -1) {
        perror("clone");
        exit(EXIT_FAILURE);
    }

    printf("Child created with PID: %d (from host's perspective)\n", child_pid);
    waitpid(child_pid, NULL, 0); // Wait for child to finish

    printf("Host process exiting.\n");
    return 0;
}

This simplified example demonstrates how clone() with CLONE_NEWPID and CLONE_NEWUTS creates a child process that has its own PID (starting from 1 within its namespace) and can set its own hostname without affecting the host.

Mechanism 2: Control Groups (cgroups)

Cgroups are a Linux kernel feature that allows for hierarchical organization of processes and allocation of system resources among them. They are essential for preventing resource starvation and ensuring fair resource distribution.

The cgroup filesystem is typically mounted at /sys/fs/cgroup. Within this hierarchy, different controllers (subsystems) manage specific resources:

• cpu: Controls CPU access (e.g., cpu.cfs_quota_us, cpu.cfs_period_us for CPU time limits, cpu.shares for relative CPU allocation).
• memory: Limits memory usage (e.g., memory.limit_in_bytes, memory.memsw.limit_in_bytes for swap).
• blkio: Controls I/O access to block devices.
• pids: Limits the number of processes a cgroup can create.
• net_cls / net_prio: Tags network packets with a class identifier for traffic shaping, or sets priority.

When a container is launched, the container runtime creates a new cgroup directory (e.g., /sys/fs/cgroup/memory/docker/<container_id>) and writes the desired resource limits into the control files within that directory. The container’s main process, and all its child processes, are then moved into this cgroup. The kernel then enforces these limits.

# Example: Creating a simple cgroup and limiting memory on the host
# (Requires root)

# Create a new cgroup for memory
sudo mkdir /sys/fs/cgroup/memory/my_test_cgroup

# Set a memory limit (e.g., 100MB)
echo 100M | sudo tee /sys/fs/cgroup/memory/my_test_cgroup/memory.limit_in_bytes

# Get the PID of a process you want to limit (e.g., a simple sleep process)
sleep 1000 &
TEST_PID=$!
echo "Process PID: $TEST_PID"

# Add the process to the cgroup
echo $TEST_PID | sudo tee /sys/fs/cgroup/memory/my_test_cgroup/cgroup.procs

# Verify the process is in the cgroup
cat /sys/fs/cgroup/memory/my_test_cgroup/cgroup.procs

# Observe memory usage (e.g., by trying to allocate more than 100M in a simple program
# or by checking /sys/fs/cgroup/memory/my_test_cgroup/memory.usage_in_bytes)

# Clean up
sudo kill $TEST_PID
sudo rmdir /sys/fs/cgroup/memory/my_test_cgroup

Mechanism 3: Union Filesystems (OverlayFS)

OverlayFS is a union mount filesystem implementation for Linux. It allows you to overlay one filesystem on top of another. In the context of containers, it’s used to combine multiple read-only image layers with a single writable layer.

• Lowerdir: The read-only layers of the image. Multiple lower directories can be specified, representing the stacked image layers.
• Upperdir: The writable layer for the container. All new files, modified files (after copy-on-write), and deleted files are recorded here.
• Workdir: An empty directory used internally by OverlayFS for atomic operations.
• Merged directory: The combined view of lowerdir and upperdir. This is what the container sees as its root filesystem.

When a container starts, OverlayFS creates a “merged” view. If a file exists in both a lower layer and the upper layer, the version in the upper layer is shown (this is how changes “override” base files). If a file is deleted in the upper layer, it effectively hides the file from the lower layers, making it appear deleted to the container. This copy-on-write strategy is extremely efficient, as it avoids duplicating entire filesystems for each container and allows quick creation of new containers.

# Conceptual: How OverlayFS is mounted
# Assuming:
# lower1 = /var/lib/docker/overlay2/<layer_id_1>/diff (base image)
# lower2 = /var/lib/docker/overlay2/<layer_id_2>/diff (next layer)
# upper = /var/lib/docker/overlay2/<container_id>/diff (writable layer)
# work = /var/lib/docker/overlay2/<container_id>/work (working directory for overlayfs)
# merged = /var/lib/docker/overlay2/<container_id>/merged (what the container sees as /)

# The actual mount command is complex and handled by the container runtime.
# It would look something like:
# mount -t overlay overlay -o lowerdir=<lower1>:<lower2>,upperdir=<upper>,workdir=<work> <merged>

Hands-On Example: Building a Mini Version

We can simulate the core isolation principles of a container using basic Linux utilities like unshare and chroot. This won’t be a full container runtime, but it will demonstrate namespaces and filesystem isolation.

#!/bin/bash

# 1. Create a root filesystem for our "container"
mkdir -p mini_root/bin mini_root/usr/bin mini_root/etc mini_root/proc mini_root/sys
cp /bin/bash mini_root/bin/
cp /bin/ls mini_root/bin/
cp /usr/bin/id mini_root/usr/bin/
cp /usr/bin/whoami mini_root/usr/bin/
# Copy necessary libraries for bash, ls, etc. (more complex in real life)
# For simplicity, we'll assume a minimal statically linked 'bash' or rely on host libs if not strictly isolated.
# A more robust example would use 'ldd' to find and copy all dynamic libraries.
# For this demo, let's just make sure bash runs.

# Create a simple /etc/passwd and /etc/hostname for isolation
echo "root:x:0:0:root:/root:/bin/bash" > mini_root/etc/passwd
echo "container-host" > mini_root/etc/hostname

echo "--- Entering mini-container ---"

# 2. Use unshare to create new namespaces and chroot for filesystem isolation
# --pid: New PID namespace
# --mount: New mount namespace (essential for chroot)
# --uts: New UTS namespace (for hostname)
# --fork: Fork a child process to run the command in the new namespaces
# --root: Change root directory for the new process
sudo unshare --pid --mount --uts --fork --root=mini_root /bin/bash -c "
    echo 'Inside container: PID $$'
    echo 'Inside container: Hostname is $(hostname)'
    echo 'Inside container: Whoami is $(whoami)'
    echo 'Inside container: Listing /'
    ls /
    echo 'Inside container: Mounting /proc and /sys'
    mount -t proc proc /proc
    mount -t sysfs sysfs /sys
    echo 'Inside container: Listing /proc'
    ls /proc
    echo 'Inside container: Running another process'
    sleep 5 &
    echo 'Inside container: ps aux output:'
    ps aux
    echo 'Inside container: Exiting'
"

echo "--- Exited mini-container ---"
echo "Back on host: Hostname is $(hostname)"
echo "Back on host: Listing mini_root (original files unchanged)"
ls mini_root/

# Clean up
sudo rm -rf mini_root

Explanation:

1. mini_root setup: We create a directory structure mimicking a minimal Linux root filesystem and copy essential binaries (bash, ls, id, whoami) into it. We also create a basic /etc/passwd and /etc/hostname to show isolation.
2. unshare command:
   • sudo unshare: We need sudo because creating namespaces and using chroot requires elevated privileges.
   • --pid: Creates a new PID namespace. The bash process inside will see itself as PID 1.
   • --mount: Creates a new mount namespace, allowing us to perform chroot and later mount /proc and /sys specific to this container.
   • --uts: Creates a new UTS namespace, so hostname inside will be independent.
   • --fork: Forks a child process to run the command, ensuring the unshare utility itself doesn’t stay in the new namespaces.
   • --root=mini_root: This performs a chroot operation, changing the root directory for the new process to mini_root.
   • /bin/bash -c "...": Executes a series of commands within the new isolated environment.
3. Inside the mini-container:
   • PID $$: Shows the process’s PID within its new namespace (it will be 1).
   • hostname: Shows “container-host”.
   • whoami: Shows “root” (based on mini_root/etc/passwd).
   • ls /: Lists the contents of mini_root.
   • mount -t proc proc /proc and mount -t sysfs sysfs /sys: These are crucial. Without a new mount namespace, these mounts would affect the host. With --mount, they are isolated. Mounting proc allows ps aux to show processes within the container’s PID namespace.
   • ps aux: Only shows processes running within this container’s PID namespace.
4. Outside the mini-container: After the unshare command finishes, the host’s environment is unaffected. The host’s hostname remains the same, and the mini_root directory is just a regular directory.

This example highlights how namespaces provide isolation for PIDs, hostnames, and mount points, while chroot provides the basic filesystem isolation. It’s a simplified demonstration of the fundamental building blocks Docker uses.

Real-World Project Example

Let’s use a simple Python Flask web application to demonstrate how Docker leverages these concepts.

Project Structure:

.
├── app.py
├── requirements.txt
└── Dockerfile

app.py:

from flask import Flask
import os

app = Flask(__name__)

@app.route('/')
def hello():
    hostname = os.uname().nodename
    pid = os.getpid()
    return f"Hello from container! My hostname is {hostname} and my PID is {pid}.\n"

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=80)

requirements.txt:

Flask==2.3.3

Dockerfile:

# Use an official Python runtime as a parent image
FROM python:3.9-slim-buster

# Set the working directory in the container
WORKDIR /app

# Copy the requirements file into the container at /app
COPY requirements.txt .

# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

# Copy the rest of the application code into the container at /app
COPY app.py .

# Make port 80 available to the world outside this container
EXPOSE 80

# Run app.py when the container launches
CMD ["python", "app.py"]

Setup and Run:

1. Save the files: Create app.py, requirements.txt, and Dockerfile in the same directory.
2. Build the Docker image:

   docker build -t my-flask-app:1.0 .
   

   • During docker build, each RUN command creates a new read-only layer. Docker caches these layers.
   • FROM python:3.9-slim-buster: Pulls the base image, which itself is a stack of layers.
   • WORKDIR /app: Sets the default directory.
   • COPY commands add new files to new layers.
   • RUN pip install: Installs dependencies into a new layer.
3. Run the Docker container:

   docker run -d -p 8080:80 --name flask-container my-flask-app:1.0
   

   • -d: Runs the container in detached mode (background).
   • -p 8080:80: Maps host port 8080 to container port 80. This triggers iptables NAT rules on the host.
   • --name flask-container: Assigns a human-readable name.
   • my-flask-app:1.0: The image to run.
   • When docker run executes, the container runtime:
     • Creates new namespaces (PID, NET, UTS, etc.) for the container’s processes.
     • Sets up cgroups for resource limits (default limits if none specified).
     • Combines the image layers with a new writable layer using OverlayFS.
     • Starts python app.py as PID 1 inside the container’s PID namespace.
     • Configures a virtual network interface for the container and iptables rules for port forwarding.
4. Observe Isolation:
   • Access the application:

     curl http://localhost:8080
     

     Output: Hello from container! My hostname is container-host and my PID is 1. (The hostname will be the container ID or a derivative, and the PID is 1, even if the host’s PID for this process is very different).
   • Inspect container processes from host:

     ps aux | grep "python app.py"
     

     You’ll see python app.py running, but with a high PID assigned by the host kernel, not PID 1. This shows the PID namespace isolation.
   • Inspect container’s network from host:

     docker inspect flask-container | grep -i "ipaddress"
     

     This will show the container’s internal IP address, which is on a Docker-managed bridge network, separate from the host’s primary IP.
   • Enter the container and inspect:

     docker exec -it flask-container bash
     

     Inside the container:
     • hostname: Will show the container’s ID or the name flask-container.
     • ps aux: Will show python app.py as PID 1.
     • ip a: Will show the container’s virtual network interface and IP.
     • ls /: Shows the merged filesystem view. Try touch /test_file and then exit. The file will only exist in the container’s writable layer, not on the host’s base image layers.

This example clearly demonstrates how Docker uses kernel features to provide a truly isolated environment for our Flask application.

Performance & Optimization

Containers offer significant performance advantages over traditional virtual machines due to their shared kernel architecture.

• Low Overhead: Since containers don’t run a full guest OS, they consume far less CPU and RAM for the operating system itself. Startup times are typically in milliseconds, compared to minutes for VMs.
• Efficient Resource Utilization: Cgroups ensure that resources are allocated and limited precisely, preventing resource hogging and allowing for higher density (more containers per host) than VMs.
• Copy-on-Write Filesystems: Union filesystems like OverlayFS are highly optimized. Sharing base image layers across multiple containers saves disk space. The copy-on-write mechanism only duplicates data when it’s modified, making container creation and updates very fast.
• Layer Caching: Docker’s image layering allows for efficient caching during builds. If a layer hasn’t changed, it’s reused, speeding up build times and reducing network traffic.
• Direct Kernel Access: Containerized applications interact directly with the host kernel, avoiding the virtualization overhead of hypervisors in VMs.

Trade-offs and Considerations:

• Kernel Compatibility: All containers on a host must be compatible with the host’s Linux kernel. This means you can’t run a Windows container on a Linux host (without specific virtualization layers like WSL2).
• Security Boundary: While strong, the isolation provided by namespaces and cgroups is not as absolute as hardware virtualization. A vulnerability in the host kernel could potentially affect all containers. This is why user namespaces and security mechanisms like Seccomp and AppArmor are important.
• Disk I/O: While CoW is efficient, heavily write-intensive applications can introduce performance penalties if not properly managed (e.g., by using volumes).

Common Misconceptions

1. Containers are lightweight Virtual Machines: This is the most common misconception. VMs virtualize hardware and run a complete guest OS, including its own kernel. Containers, however, share the host’s kernel and only virtualize the OS environment at the process level. They are process-level isolation, not machine-level.
2. Containers provide perfect security isolation: While containers offer significant isolation, it’s not foolproof. Because they share the host kernel, a kernel vulnerability could potentially allow a container to break out. VMs, with their hardware virtualization, generally offer a stronger security boundary. Best practices like running containers as non-root users, using user namespaces, and enabling security profiles (Seccomp, AppArmor) are crucial.
3. Containers don’t run on the host machine: The processes inside a container are still processes running on the host machine’s kernel. They simply operate within their own isolated namespaces and resource limits. You can see container processes in ps aux on the host, just with different PIDs than they report internally.
4. Container images are executables: An image is a template – a snapshot of a filesystem and metadata (like entrypoint, exposed ports). It’s not an executable itself. When an image is run, it becomes a container, which is a running instance of that image, complete with its own writable layer and isolated environment.
5. Containers encapsulate the entire OS: They only encapsulate the application and its direct dependencies, providing the necessary user-space environment. The kernel is always provided by the host.

Advanced Topics

• Container Runtimes (runc, containerd, CRI-O): Docker itself doesn’t directly implement all the low-level kernel interactions. It delegates this to a container runtime. containerd is a robust runtime that sits between Docker (or Kubernetes) and the underlying operating system. runc is the actual low-level component that interfaces with the kernel to create and run containers according to the OCI (Open Container Initiative) specification. CRI-O is another OCI-compliant runtime specifically designed for Kubernetes.
• Volumes and Bind Mounts: While the union filesystem provides a writable layer, this layer is ephemeral. When a container is removed, its writable layer is lost. Volumes and bind mounts provide persistent storage by mounting a directory from the host filesystem (or a dedicated volume) directly into the container’s mount namespace. This allows data to persist beyond the container’s lifecycle and be shared between containers.
• Networking Modes: Beyond the default bridge network, Docker supports various networking modes: host (container shares host’s network stack, no isolation), none (no network access), overlay (for multi-host networking), and custom user-defined bridge networks.
• Security Boundaries (Seccomp, AppArmor/SELinux): These Linux security modules enhance container isolation.
  • Seccomp (Secure Computing mode): Filters system calls a process can make, limiting the attack surface. Docker applies a default Seccomp profile.
  • AppArmor/SELinux: Mandatory Access Control (MAC) systems that restrict what programs can do (e.g., file access, network access) based on profiles. Docker can integrate with these.
• Rootless Containers: Running containers without root privileges on the host. This significantly enhances security by preventing a container breakout from granting root access to the host. It leverages user namespaces to map a non-root user on the host to root inside the container.

Comparison with Alternatives

Virtual Machines (VMs)

Chroot Jails

Debugging & Inspection Tools

• docker ps: Lists running containers.
• docker logs <container_id_or_name>: Fetches logs from a container.
• docker exec -it <container_id_or_name> bash: Opens an interactive shell inside a running container to inspect its environment, run commands (ps aux, ip a, ls -l /).
• docker inspect <container_id_or_name>: Provides detailed low-level information about a container, including its configuration, network settings, and mounted volumes. This is where you can see the actual Mounts and CgroupParent paths.
• docker stats <container_id_or_name>: Shows live resource usage (CPU, memory, network I/O, block I/O) for running containers, drawing data from cgroups.
• nsenter: A powerful Linux utility to enter the namespace of an existing process. You can use it to “jump into” a container’s namespaces from the host to debug kernel-level interactions.

  # Find the PID of a container's main process on the host
  CONTAINER_PID=$(docker inspect -f '{{.State.Pid}}' flask-container)
  # Enter its network namespace to see its network config
  sudo nsenter -t $CONTAINER_PID -n ip a
  # Enter its PID namespace to see its internal PIDs
  sudo nsenter -t $CONTAINER_PID -p ps aux
  

• lsof -p <container_pid>: On the host, shows files opened by the container’s process, including those from the union filesystem.
• strace -p <container_pid>: Traces system calls made by a container’s process (can be very verbose).
• /sys/fs/cgroup/: Directly inspect the cgroup hierarchy on the host to see how resource limits are applied.

Key Takeaways

• Containers are built on Linux Kernel features: Namespaces, Cgroups, and Union Filesystems are the fundamental technologies.
• Namespaces provide isolation: Each container gets its own view of PIDs, network interfaces, mount points, hostnames, and IPC.
• Cgroups provide resource limits: They prevent containers from consuming excessive CPU, memory, or I/O, ensuring host stability.
• Union Filesystems (OverlayFS) enable efficiency: Layered images and copy-on-write optimize storage and allow fast container startup.
• Containers share the host kernel: This is why they are lightweight and fast, but also why their isolation isn’t as absolute as VMs.
• Docker is an orchestration tool: It uses containerd and runc to manage images, build processes, and interact with the kernel features.
• Understanding internals is key to debugging and security: Knowing how containers work helps solve complex issues and implement robust security practices.

References

1. Docker Official Documentation
2. Linux Kernel Documentation on Namespaces
3. Linux Kernel Documentation on Cgroups
4. Linux Kernel Documentation on OverlayFS
5. Open Container Initiative (OCI) Specifications

Transparency Note

This document was created by an AI Expert to provide an in-depth technical explanation of how containers work, based on publicly available information and common industry understanding as of December 2025. While every effort has been made to ensure accuracy and detail, specific implementation details may vary slightly across different Linux distributions, kernel versions, or container runtimes.