Docker Compose Wait for Dependencies

Home Docker Compose Wait for Dependencies
Docker Compose Wait for Dependencies

Docker Compose Wait for Dependencies

6 Lessons

18 mins

Free
666246707151584446  
514
Shares

This article describes how to configure the docker-compose.yml file to make docker compose wait for dependencies to be ready before starting a given application container.

We’ll introduce two different tools (dockerize and docker-compose-wait tool) to make docker compose wait for any service or dependencies to start. You can use these two solutions to deploy your application stack, particularly when you want:

  • docker compose wait for mysql,
  • docker compose wait for postgres,
  • docker compose wait for redis or
  • docker compose wait for mongodb

You will learn a simple and complex examples with MongoDB, Postgres, MySQL.



Contents:

Example of docker-compose.yml file

The following simple example includes 2 services: my_super_app, which depends on a mysql database.

version: '3.6'
services:
  mysql:
    image: "mysql:5.7"
    container_name: mysql
    restart: always
    volumes:
      - ./mysql:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=your_password
      - MYSQL_USER=root
      - MYSQL_PASSWORD=your_password
      - MYSQL_DATABASE=wordpress
    ports:
      - "3306:3306"

  my_super_app:
    build: ./my_super_app
    image: "my_super_app:latest"
    container_name: my_supper_app
    depends_on:
      - mysql

You can control the order of service startup and shutdown with the depends_on option. Compose always starts and stops containers in dependency order, where dependencies are determined by depends_on, links, volumes_from, and network_mode: "service:...".

However, for startup Compose does not wait until a container is “ready” (whatever that means for your particular application) - only until it’s running.

The problem

When you run docker-compose up, docker starts the mysql database container first and then the my_super_app. However the MySQL database might not be ready before the execution of the web app.

The web app will try to connect to the database, which may be still initializing, leading to the application crash. And because it’s race, you probably won’t see this every time.

Intermediate solution is to add restart: always to our application config, to retry until the database starts responding. A better solution is to use one of the following tools: dockerize tool, docker-compose-wait tool, wait-for and wait-for-it.

Solution 1: Using the dockerize tool

Description

Dockerize is an utility to simplify running applications in docker containers. It allows you to:

  • generate application configuration files at container startup time from templates and container environment variables
  • Tail multiple log files to stdout and/or stderr
  • Wait for other services to be available using TCP, HTTP(S), unix before starting the main process.

Here, we’ll focus on how to make docker compose wait for other dependencies. Dockerize gives you the ability to wait for services on a specified protocol (file, tcp, tcp4, tcp6, http, https and unix) before starting your application:

dockerize -wait tcp://db:5432 -wait http://web:80 -wait file:///tmp/generated-file

Important arguments:

  • timeout. You can optionally specify how long to wait for the services to become available by using the -timeout # argument (Default: 10 seconds). If the timeout is reached and the service is still not available, the process exits with status code 1.
  • wait-retry-interval. dockerize sleeping time before checking whether the dependencies are ready
dockerize -wait tcp://db:5432 -wait http://web:80 -timeout 10s -wait-retry-interval 3s

Quick start

# Download a template
git clone https://github.com/kassambara/docker-compose-wait-for-container.git

# Build the demo application
cd docker-compose-wait-for-container/ex02-using-dockerize-tool
docker-compose build
# Running your app
docker-compose run my_super_app

# Stopping containers and cleaning
docker-compose down 
rm -rf mysql

Step 0: Download a template

# Download a template
git clone https://github.com/kassambara/docker-compose-wait-for-container.git
cd docker-compose-wait-for-container/ex02-using-dockerize-tool

Project folder structure:

files/docker-compose-wait-for-container/ex02-using-dockerize-tool
├── docker-compose.yml
└── my_super_app
    ├── Dockerfile
    └── sayhello

Essential project contents:

  • docker-compose.yml to run all container services
  • my_super_app scripts: template Dockerfile to build your application. Here, this demo app will ask your name and then congratulate you!

Step 1: Add dockerize to your application Dockerfile

Use this for Ubuntu image:

RUN apt-get update && apt-get install -y wget

ENV DOCKERIZE_VERSION v0.6.1
RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz

Or this for Alpine images:

RUN apk add --no-cache openssl

ENV DOCKERIZE_VERSION v0.6.1
RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && tar -C /usr/local/bin -xzvf dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && rm dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz

Example of Dockerfile using the alpine image:

FROM alpine:latest

# Add hello scripts
ADD sayhello /sayhello
RUN chmod +x /sayhello

# Add dockerize tool -------------------
RUN apk add --no-cache openssl
ENV DOCKERIZE_VERSION v0.6.1
RUN wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && tar -C /usr/local/bin -xzvf dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz \
    && rm dockerize-alpine-linux-amd64-$DOCKERIZE_VERSION.tar.gz

CMD ["/sayhello"]

Step 2: Modify your docker-compose.yml file

version: '3.6'
services:
  mysql:
    image: "mysql:5.7"
    container_name: mysql
    restart: always
    volumes:
      - ./mysql:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=your_password
      - MYSQL_USER=root
      - MYSQL_PASSWORD=your_password
      - MYSQL_DATABASE=wordpress
    ports:
      - "3306:3306"
    expose:
      - 3306

  my_super_app:
    build: ./my_super_app
    image: "my_super_app:latest"
    container_name: my_supper_app
    depends_on:
      - mysql
    command: sh -c "dockerize -wait tcp://mysql:3306 -timeout 300s -wait-retry-interval 30s /sayhello"

In essence, Dockerize is a wrapper. dockerize our_normal_command just calls our command. But optionally, we can add parameters to delay execution, perform file templating or redirect output from files to STDOUT/STDERR. Very common and useful operations in a Docker world.

Examples of optional dockerize configurations:

# redirect files to stdout and stderr
dockerize \
  -stdout info.log \
  -stdout perf.log \
  ...

# wait for 2 services with 10s timeout
dockerize \
  -wait tcp://db:5432 \
  -wait http://web:80 \
  -timeout 10s \
  ...

# template option
dockerize \
  -template nginx.tmpl:nginx.conf \
  ...

Step 3: Building and running your app

# Building your app
cd docker-compose-wait-for-container/ex02-using-dockerize-tool
docker-compose build
# Running your app
docker-compose run my_super_app

No more race! Dockerize delay start of our command until the database is available. Not just started - it’s waiting for connections. That’s a difference between Dockerize and depends_on.

Console log output:

Docker compose wait logs

After typing your name, you will see a congratulation message from my_super_app

Step 4: Stopping containers and cleaning

docker-compose down 
rm -rf mysql

Solution 2: Using the docker-compose-wait tool

Description

The docker-compose-wait tool is a small command line utility to wait for other docker images to be started while using docker-compose. It permits to wait for a fixed amount of seconds and/or to wait until a TCP port is open on a target image.

Like for the dockerize tool, you need to add the docker-compose-wait tool in your application Dockerfile.



Quick start

# Download a template
git clone https://github.com/kassambara/docker-compose-wait-for-container.git

# Build the demo application
cd docker-compose-wait-for-container/ex01-using-wait-tool
docker-compose build
# Running your app
docker-compose run my_super_app

# Stopping containers and cleaning
docker-compose down 
rm -rf mysql

Step 0: Download a template

# Download a template
git clone https://github.com/kassambara/docker-compose-wait-for-container.git
cd docker-compose-wait-for-container/ex02-using-dockerize-tool

Project folder structure:

files/docker-compose-wait-for-container/ex01-using-wait-tool
├── docker-compose.yml
└── my_super_app
    ├── Dockerfile
    └── sayhello

Step 1: Add the docker-compose-wait tool to your application Dockerfile

Example of Dockerfile using the alpine image:

FROM alpine:latest

# Add hello scripts
ADD sayhello /sayhello
RUN chmod +x /sayhello

# Add docker-compose-wait tool -------------------
ENV WAIT_VERSION 2.7.2
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/$WAIT_VERSION/wait /wait
RUN chmod +x /wait

CMD ["/sayhello"]

Step 2: Modify your docker-compose.yml file

version: '3.6'
services:
  mysql:
    image: "mysql:5.7"
    container_name: mysql
    restart: always
    volumes:
      - ./mysql:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=your_password
      - MYSQL_USER=root
      - MYSQL_PASSWORD=your_password
      - MYSQL_DATABASE=wordpress
    ports:
      - "3306:3306"
    expose:
      - 3306

  my_super_app:
    build: ./my_super_app
    image: "my_super_app:latest"
    container_name: my_supper_app
    depends_on:
      - mysql
    command: sh -c "/wait && /sayhello"
    environment:
      - WAIT_HOSTS=mysql:3306
      - WAIT_HOSTS_TIMEOUT=300
      - WAIT_SLEEP_INTERVAL=30
      - WAIT_HOST_CONNECT_TIMEOUT=30
  • The command sh -c “/wait && /sayhello” will run the wait tool and then your application, here /sayhello.
  • When docker-compose is started (or Kubernetes or docker stack or whatever), your application will be started only when all the pairs host:port in the WAIT_HOSTS variable are available. When docker-compose is started (or Kubernetes or docker stack or whatever), your application will be started only when all the pairs host:port in the WAIT_HOSTS variable are available. The WAIT_HOSTS environment variable is not mandatory, if not declared, the script executes without waiting.
  • To make your docker application container wait for multiple hosts, the environment variable can be specified as for example WAIT_HOSTS=mysql:3306, nginx:80

Additional configuration options. The behavior of the wait utility can be configured with the following environment variables:

  • WAIT_HOSTS: comma separated list of pairs host:port for which you want to wait.
  • WAIT_HOSTS_TIMEOUT: max number of seconds to wait for all the hosts to be available before failure. The default is 30 seconds.
  • WAIT_HOST_CONNECT_TIMEOUT: The timeout of a single TCP connection to a remote host before attempting a new connection. The default is 5 seconds.
  • WAIT_BEFORE_HOSTS: number of seconds to wait (sleep) before start checking for the hosts availability
  • WAIT_AFTER_HOSTS: number of seconds to wait (sleep) once all the hosts are available
  • WAIT_SLEEP_INTERVAL: number of seconds to sleep between retries. The default is 1 second.

Step 3: Building and running your app

# Building your app
cd docker-compose-wait-for-container/ex01-using-wait-tool
docker-compose build
# Running your app
docker-compose run my_super_app

Console log output:

Docker compose wait logs

After typing your name, you will see a congratulation message from my_super_app

Step 4: Stopping containers and cleaning

docker-compose down 
rm -rf mysql

Complex example with MongoDB, Postgres, MySQL

version: "3.6"

services:

  mongo:
    image: mongo:3.4
    hostname: mongo
    ports:
      - "27017:27017"

  postgres:
    image: "postgres:9.4"
    hostname: postgres
    ports:
      - "5432:5432"

  mysql:
    image: "mysql:5.7"
    hostname: mysql
    ports:
      - "3306:3306"

  my_super_app:
    build: ./my_super_app
    image: "my_super_app:latest"
    container_name: my_supper_app
    depends_on:
      - mysql
    command: sh -c "/wait && /sayhello"
    environment:
      - WAIT_HOSTS=postgres:5432, mysql:3306, mongo:27017
      - WAIT_HOSTS_TIMEOUT=300
      - WAIT_SLEEP_INTERVAL=30
      - WAIT_HOST_CONNECT_TIMEOUT=30

Summary

This tutorial describes how to make docker-compose wait for container dependencies using dockerize and docker-compose-wait tools.



 
          

Version: Français

Lessons

No Comments

Give a comment

Want to post an issue with R? If yes, please make sure you have read this: How to Include Reproducible R Script Examples in Datanovia Comments

Teachers

Related Courses