Introduction

In previous chapters, we learned how to build and run individual Docker containers. While this is powerful for isolated services, real-world applications often consist of multiple interconnected services—a web server, a database, a cache, a message queue, etc. Managing these services individually with docker run can quickly become cumbersome and error-prone. This is where Docker Compose comes into play.

Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services, networks, and volumes. Then, with a single command, you can create and start all the services from your configuration. This chapter will delve into the core concepts of Docker Compose, its benefits, and how to use it effectively to orchestrate complex applications.

Main Explanation

What is Docker Compose?

Docker Compose is a tool that allows you to define and run multi-container Docker applications. It simplifies the process of managing multiple containers that form a single application by allowing you to define your entire application stack in a single docker-compose.yml file. This file describes the services, networks, and volumes needed for your application.

Why Use Docker Compose?

Docker Compose offers several compelling advantages for developing and deploying multi-container applications:

  • Simplified Application Definition: Define your entire application stack (services, networks, volumes, environment variables) in a single, human-readable YAML file.
  • Reproducibility: Ensure that your application environment is consistent across different machines (development, testing, production) as it’s all defined in the docker-compose.yml file.
  • Easier Orchestration: Start, stop, and rebuild all services with a single command, rather than managing each container individually.
  • Isolation: Each service runs in its own container, providing isolation and making it easier to manage dependencies and scale individual components.
  • Networking: Compose automatically sets up a default network, allowing services to communicate with each other using their service names as hostnames.
  • Portability: Your Compose file can be easily shared and used by other developers or CI/CD pipelines.

The docker-compose.yml File Structure

The docker-compose.yml file is the heart of any Docker Compose application. It uses a YAML format to define the services, networks, and volumes.

A typical docker-compose.yml file includes:

  • version: Specifies the Compose file format version. (e.g., 3.8)
  • services: Defines the individual containers that make up your application. Each service typically includes:
    • image: The Docker image to use (e.g., nginx:latest, postgres:13).
    • build: Path to a Dockerfile if you need to build a custom image.
    • ports: Port mappings from the host to the container (e.g., "80:80").
    • volumes: Mount points for data persistence or configuration (e.g., ./app:/app).
    • environment: Environment variables to pass to the container.
    • depends_on: Specifies service dependencies to control startup order (though it doesn’t wait for services to be ready).
    • networks: Attaches services to specific networks.
  • networks: Defines custom networks for services to communicate over, providing better isolation and organization than the default bridge network.
  • volumes: Defines named volumes for persistent data storage, independent of container lifecycles.

Key Docker Compose Commands

Once you have a docker-compose.yml file, you can use the docker compose (or docker-compose for older versions) commands to manage your application.

  • docker compose up: Builds, (re)creates, starts, and attaches to containers for all services defined in the docker-compose.yml file.
    • docker compose up -d: Runs containers in detached mode (in the background).
    • docker compose up --build: Forces rebuilding images before starting containers.
  • docker compose down: Stops and removes containers, networks, and volumes defined in the docker-compose.yml file.
    • docker compose down --volumes: Also removes named volumes.
  • docker compose ps: Lists all services and their status.
  • docker compose logs: Displays log output from services.
    • docker compose logs -f: Follows log output.
  • docker compose build: Builds or rebuilds services.
  • docker compose exec [service_name] [command]: Executes an arbitrary command inside a service container.
    • docker compose exec web bash: Opens a bash shell in the web service container.
  • docker compose restart [service_name]: Restarts one or more services.
  • docker compose stop [service_name]: Stops one or more services.
  • docker compose start [service_name]: Starts one or more services.

Networking in Compose

By default, Docker Compose sets up a single bridge network for your application. All services in your docker-compose.yml file are connected to this default network and can discover each other using their service names as hostnames.

You can also define custom networks under the networks section to create more isolated environments or connect to existing Docker networks.

Volumes in Compose

Volumes are crucial for data persistence. Docker Compose allows you to define two types of volumes:

  • Named Volumes: Managed by Docker, ideal for persistent data that needs to outlive containers. Defined under the top-level volumes key and then referenced by services.
  • Bind Mounts: Mounts a file or directory from the host machine into the container. Useful for development to hot-reload code changes or inject configuration files.

Examples

Let’s create a simple multi-container application: a Python Flask web application that connects to a PostgreSQL database.

Project Structure

First, create a directory for your project:

mkdir my-flask-app
cd my-flask-app

Inside my-flask-app, create the following files:

  1. app.py (Flask application)
  2. Dockerfile (for the Flask app)
  3. requirements.txt (Flask app dependencies)
  4. docker-compose.yml (Compose configuration)

1. app.py

This simple Flask app will connect to a PostgreSQL database and display a “Hello World” message along with a database connection status.

from flask import Flask
import os
import psycopg2

app = Flask(__name__)

@app.route('/')
def hello():
    db_status = "Disconnected"
    try:
        conn = psycopg2.connect(
            host=os.environ.get("DB_HOST", "db"),
            database=os.environ.get("DB_NAME", "mydatabase"),
            user=os.environ.get("DB_USER", "user"),
            password=os.environ.get("DB_PASSWORD", "password")
        )
        cursor = conn.cursor()
        cursor.execute("SELECT 1")
        db_status = "Connected to PostgreSQL!"
        cursor.close()
        conn.close()
    except Exception as e:
        db_status = f"Failed to connect to PostgreSQL: {e}"

    return f"<h1>Hello from Flask!</h1><p>Database Status: {db_status}</p>"

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

2. requirements.txt 

Flask==2.3.2
psycopg2-binary==2.9.9

3. Dockerfile (for the Flask app) 

# Use an official Python runtime as a parent image
FROM python:3.9-slim-buster

# Set the working directory in the container
WORKDIR /app

# Copy the current directory contents into the container at /app
COPY requirements.txt .
COPY app.py .

# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

# Make port 5000 available to the world outside this container
EXPOSE 5000

# Run app.py when the container launches
CMD ["python", "app.py"]

4. docker-compose.yml

This file defines two services: web (our Flask app) and db (PostgreSQL).

version: '3.8'

services:
  web:
    build: .
    ports:
      - "5000:5000"
    environment:
      # These environment variables are used by app.py to connect to the database
      DB_HOST: db
      DB_NAME: mydatabase
      DB_USER: user
      DB_PASSWORD: password
    depends_on:
      - db # Ensures 'db' service starts before 'web', but doesn't wait for DB to be ready
    volumes:
      - .:/app # Mounts the current directory to /app in the container for live changes

  db:
    image: postgres:13
    environment:
      POSTGRES_DB: mydatabase
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    volumes:
      - db_data:/var/lib/postgresql/data # Persistent storage for PostgreSQL data

volumes:
  db_data: # Define a named volume for database persistence

Running the Application

  1. Navigate to your project directory:

    cd my-flask-app

  2. Start the application:

    docker compose up -d

    This command will:

    • Build the web service image using the Dockerfile.
    • Pull the postgres:13 image for the db service.
    • Create a default network for both services.
    • Start both containers in detached mode.

  3. Check the status:

    docker compose ps

    You should see both web and db services running.

  4. View logs (optional):

    docker compose logs -f

    You can see the logs from both services, or specify a service:

    docker compose logs web

  5. Access the application: Open your web browser and go to http://localhost:5000. You should see “Hello from Flask! Database Status: Connected to PostgreSQL!”.

  6. Stop and remove the application: When you’re done, you can stop and remove all services, networks, and the named volume:

    docker compose down --volumes

Mini Challenge

Extend the Flask application example by adding a Redis cache service.

  1. Modify docker-compose.yml: Add a redis service using the redis:latest image.
  2. Modify app.py:
    • Install the redis Python client (pip install redis).
    • In app.py, try to connect to the redis service (using redis as the hostname) and store/retrieve a simple key-value pair.
    • Display the Redis connection status and the retrieved value on the web page.
  3. Rebuild and run: Use docker compose up --build -d to bring up the updated application.
  4. Verify: Check http://localhost:5000 to ensure both PostgreSQL and Redis connections are successful.

Summary

Docker Compose is an indispensable tool for managing multi-container Docker applications. It allows you to define complex service architectures in a single docker-compose.yml file, simplifying development workflows, ensuring reproducibility, and streamlining the deployment of interconnected services. By leveraging services, networks, and volumes, you can build robust and scalable applications with ease. Mastering Docker Compose is a significant step towards becoming proficient in modern containerized application development.