Docker Architecture and Docker Compose Files

Hello all,

In my previous article, we discussed on Dockerfile and how we can create our own images. So now that we know the basics of images and containers, lets take a deeper dive to see how Docker works.

Docker Architecture

Docker Architecture

There are 3 main components involved:

The Client — The User interacts with Docker Daemon, which is the controller for Docker. We can interact to the CLI , where the Docker host is installed or through Remote API calls. All the commands are run and the commands are received through API calls to the Daemon.

Docker Daemon — The Daemon is the controller for performing the services, once the API call is recieved , it performs actions accordingly. It is responsible for creating and managing containers, images, network and Volumes. The volumes are created in the Filesystem of the Docker Host. It can also pull images for creation of containers.

Docker Registry — The registry is a storage system for Docker Images. With the right accesses, we can pull/push images from registry. To perform actions from CLI , we first need to authenticate docker credentials and then it will be able to push/pull images from Docker. The Registry is public with each account having only 1 private repository , but infinite number of public repositories, for the community edition.

Now that we know the agents involved, its obvious that the Daemon is the brain of the functioning of Docker.

Docker Compose

Now that we know the basic Architecture for Docker, lets look into Docker compose.

Why did the need for Docker compose come up?

Imagine that we have not just 1 , but multiple containers to provide services- for example a container that runs nginx server, another container for the python application having flask framework and a container for the backend, mysql. All these containers are for deploying a simple python application.

Obviously, we could bring up each container one at a time.

But what happens if one of them is down? And what if we want to set up an internal network so that each containers are connected with each other?

All of these issues are addressed using docker compose. A docker compose is a YAML file that integrates containers with each other. It is also beneficial to bring up all containers in one go. And, if a container goes down, it only recreates containers that have changed.

Let me introduce you to awesome compose, a github repository, where they have defined various files for different services-

Repo Link : https://github.com/docker/awesome-compose

This is a great link for beginners who start with Docker.

Note : Docker compose utility has to be installed.

Here, lets take an example from this project with Go application, Nginx Server and MariaDB

awesome-compose/nginx-golang-mysql at master · docker/awesome-compose

Here, lets investigate more on the compose.yml file.

services:
  backend:
    build:
      context: backend
      target: builder
    secrets:
      - db-password
    depends_on:
      db:
        condition: service_healthy

  db:
    # We use a mariadb image which supports both amd64 & arm64 architecture
    image: mariadb:10-focal
    # If you really want to use MySQL, uncomment the following line
    #image: mysql:8
    command: '--default-authentication-plugin=mysql_native_password'
    restart: always
    healthcheck:
      test: ['CMD-SHELL', 'mysqladmin ping -h 127.0.0.1 --password="$$(cat /run/secrets/db-password)" --silent']
      interval: 3s
      retries: 5
      start_period: 30s
    secrets:
      - db-password
    volumes:
      - db-data:/var/lib/mysql
    environment:
      - MYSQL_DATABASE=example
      - MYSQL_ROOT_PASSWORD_FILE=/run/secrets/db-password
    expose:
      - 3306

  proxy:
    image: nginx
    volumes:
      - type: bind
        source: ./proxy/nginx.conf
        target: /etc/nginx/conf.d/default.conf
        read_only: true
    ports:
      - 80:80
    depends_on: 
      - backend

volumes:
  db-data:

secrets:
  db-password:
    file: db/password.txt

The first line is always version( except for version 1).

Version 1 didn’t support networking and hence , version 2 came up as an extended feature with version 1 features. Currently, Version 3 is the latest.

The second line is services. Under services, we provide the services that we need, here, there are 3 services(backend, db, proxy). Note that the name of the service can be anything , its not based on any keywords.

Here build is the keyword for docker compose , with context being the folder name where the Dockerfile is present. The target keyword is to build the specified stage as defined inside the Dockerfile. It is useful for multi stage build which is out of scope for now.

We have added a secrets block so that it can fetch secrets that is mapped to the db-secrets.

The depends_on is an important keyword , when means that it would bring up the container ONLY IF the container that is specified under depends_on block is brought up first , and here , it also makes sure that the container brought up should also be healthy. This is useful when we want to bring up a DB first and then the application.

Make sure to know YAML scripting as a pre-requisite because indentations are important for YAML Files.

For the next service, here db, notice that we use the keyword image, to specify the image that we need to pull from the Docker Repo. As you can see, there are multiple arguments that we can provide for the service, say volumes, port numbers , environment variables etc.

For the last service, proxy, we pull an image from the Docker repo and copy the configuration files from local to a path inside the nginx container.

After the 3 services are defined as per our requirement, we have specified a volumes block to create a volume.

Note that networks can also be specified using the networks block, just like we did for volumes. By default, Docker creates a network of the format — — { <folder_name>_default }

Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name.

Implementation

There are 2 main commands that we use to bring up/down docker compose file:

The docker-compose down command helps to Stop and remove containers, networks, images, and volumes.

docker compose up -d #To bring up containers and the associated resources

docker compose down  #To stop and destroy the containers,volumes and networks

Example 1 : Apache Server with PHP

Bringing the containers up- docker compose up

Notice that Network name — apache-php_default is created.

(1) — Notice that Network apache-php_default is created

(2)

(3)

Output onto localhost on port 80

Bringing the containers down- docker compose down

Example 2 : Nginx Server with Flask and Mysql

Bringing the containers up — docker compose up

(1)

(2)

Output

Bringing the containers down — docker compose down

So now, we know the Docker Architecture and how it implements Docker engine. We also know the basic framework of how to write a docker compose file. However, this just covers some of the many attributes available , so do explore more on it! Awesome compose is a great way to start. Next stop, Docker network on the learning train :)