Docker Compose Health Check Patterns - Portable Best Practices

Overview

This guide documents portable health check patterns for Docker Compose services. Following these patterns ensures health checks work across different base images, container runtimes, and environments without relying on utilities that may not be present.

Why Portable Health Checks Matter
Common Pitfalls
Portable Patterns by Service Type
Health Check Configuration Guidelines
Testing Health Checks
Troubleshooting

Why Portable Health Checks Matter

The Problem

Many Docker images use minimal base images (Alpine, distroless, scratch) that:

❌ Lack common utilities (curl, wget, nc, telnet)
❌ Don’t include health check tools (grpc_health_probe, httpie)
❌ May not have shell interpreters (/bin/bash, /bin/sh)
❌ Minimize attack surface by removing non-essential binaries

The Solution

Use service-native health check commands that are guaranteed to exist in the container:

✅ Database clients (pg_isready, redis-cli, mongosh)
✅ Service-specific health endpoints (kc.sh show-config)
✅ Built-in health check binaries shipped with the service
✅ TCP port checks as fallback (most portable)

Benefits

Reliability: Health checks don’t fail due to missing utilities
Portability: Works across different image variants (alpine, distroless, etc.)
Security: Doesn’t require installing additional packages
Performance: Uses lightweight native commands
Maintainability: Survives base image updates

Common Pitfalls

❌ Anti-Pattern 1: Assuming `curl` Exists

# WRONG: curl may not exist in minimal images
healthcheck:
  test: ["CMD", "curl", "-f", "http://localhost:8080/health"]

Why it fails:

Alpine-based images: curl not installed by default
Distroless images: No package manager to install utilities
Security-hardened images: Utilities removed to reduce attack surface

Example failures:

qdrant:v1.15.1 - lacks curl, wget, grpc_health_probe
quay.io/keycloak/keycloak:latest - minimal image without curl

❌ Anti-Pattern 2: Using `wget` Without Verification

# WRONG: wget also commonly missing
healthcheck:
  test: ["CMD", "wget", "--spider", "http://localhost:8080/health"]

❌ Anti-Pattern 3: Hardcoded Shell Paths

# WRONG: /bin/bash may not exist (Alpine uses /bin/sh)
healthcheck:
  test: ["CMD-SHELL", "/bin/bash -c 'redis-cli ping'"]

❌ Anti-Pattern 4: External Dependencies

# WRONG: Requires installing grpc_health_probe
healthcheck:
  test: ["CMD", "/usr/local/bin/grpc_health_probe", "-addr=:8081"]
  # What if grpc_health_probe isn't installed?

Portable Patterns by Service Type

PostgreSQL

✅ Recommended: Use pg_isready

healthcheck:
  test: ["CMD-SHELL", "pg_isready -U postgres"]
  interval: 5s
  timeout: 3s
  retries: 10
  start_period: 10s

Why it works:

pg_isready is always included in official PostgreSQL images
Checks database readiness, not just process existence
Returns proper exit codes (0 = healthy, 1/2 = unhealthy)

Alternative (if pg_isready unavailable):

healthcheck:
  test: ["CMD-SHELL", "psql -U postgres -c 'SELECT 1' || exit 1"]

Redis

✅ Recommended: Use redis-cli ping

healthcheck:
  test: ["CMD", "redis-cli", "ping"]
  interval: 3s
  timeout: 2s
  retries: 10
  start_period: 5s

Why it works:

redis-cli always bundled with Redis
ping command is lightweight and fast
Returns PONG on success (exit code 0)

For Redis with AUTH:

healthcheck:
  test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
  # Note: Password visible in docker inspect - use redis.conf ACL for production

Keycloak

✅ Recommended: Use built-in kc.sh command

healthcheck:
  # Keycloak minimal image doesn't include curl
  # Use kc.sh show-config to verify Keycloak is initialized
  test: ["CMD-SHELL", "/opt/keycloak/bin/kc.sh show-config | grep -q 'kc.db' || exit 1"]
  interval: 5s
  timeout: 5s
  retries: 40
  start_period: 45s  # Keycloak slow to start

Why it works:

kc.sh is the native Keycloak management script
show-config verifies configuration loaded
grep -q 'kc.db' confirms database configured
No external utilities required

Alternative (if HTTP endpoint available):

healthcheck:
  # Only if curl/wget installed or using full image variant
  test: ["CMD", "curl", "-f", "http://localhost:8080/health/ready"]

MongoDB

✅ Recommended: Use mongosh or mongo client

healthcheck:
  test: ["CMD", "mongosh", "--eval", "db.adminCommand('ping')"]
  interval: 5s
  timeout: 3s
  retries: 10
  start_period: 10s

For older MongoDB versions (<5.0):

healthcheck:
  test: ["CMD", "mongo", "--eval", "db.adminCommand('ping')"]

OpenFGA (gRPC Services)

✅ Recommended Option 1: Use bundled grpc_health_probe

healthcheck:
  test: ["CMD", "/usr/local/bin/grpc_health_probe", "-addr=:8081"]
  interval: 3s
  timeout: 3s
  retries: 15
  start_period: 5s

Requirements:

Verify grpc_health_probe is in the image
Check with: docker run --rm <image> ls /usr/local/bin/grpc_health_probe

✅ Recommended Option 2: TCP port check (most portable)

healthcheck:
  test: ["CMD-SHELL", "timeout 1 bash -c '</dev/tcp/localhost/8081' || exit 1"]
  interval: 5s
  timeout: 3s
  retries: 10

Why it works:

No external utilities needed (uses Bash built-in /dev/tcp)
Works on any image with Bash
Checks if port is listening

For Alpine/sh-only environments:

healthcheck:
  test: ["CMD-SHELL", "nc -z localhost 8081 || exit 1"]
  # Note: Requires nc (netcat) - usually present in Alpine

Qdrant (Vector Database)

✅ Recommended: TCP port check

healthcheck:
  # qdrant:v1.15.1 image lacks wget, curl, and grpc_health_probe
  # Use TCP port check as most portable option
  test: ["CMD-SHELL", "timeout 1 bash -c '</dev/tcp/localhost/6333' || exit 1"]
  interval: 5s
  timeout: 3s
  retries: 10
  start_period: 10s

Why TCP check:

Qdrant minimal image has no HTTP clients
Installing utilities defeats minimal image purpose
Port listening = service ready for most use cases

Alternative (if Python available):

healthcheck:
  test: ["CMD-SHELL", "python3 -c 'import socket; s=socket.socket(); s.connect((\"localhost\", 6333))' || exit 1"]

Elasticsearch

✅ Recommended: Use curl if available, fallback to TCP

healthcheck:
  test: ["CMD-SHELL", "curl -f http://localhost:9200/_cluster/health || exit 1"]
  interval: 10s
  timeout: 5s
  retries: 12
  start_period: 30s

For minimal images:

healthcheck:
  test: ["CMD-SHELL", "timeout 1 bash -c '</dev/tcp/localhost/9200' || exit 1"]

RabbitMQ

✅ Recommended: Use rabbitmqctl

healthcheck:
  test: ["CMD", "rabbitmqctl", "status"]
  interval: 10s
  timeout: 5s
  retries: 5
  start_period: 30s

Alternative (check cluster health):

healthcheck:
  test: ["CMD", "rabbitmq-diagnostics", "-q", "ping"]

Health Check Configuration Guidelines

Timing Parameters Best Practices

healthcheck:
  interval: 5s      # How often to check (balance between responsiveness and load)
  timeout: 3s       # MUST be <= interval (Codex validation requirement)
  retries: 10       # Number of consecutive failures before unhealthy
  start_period: 10s # Grace period during startup (no failures count)

Rules:

interval >= timeout (required by Docker Compose validation)
start_period should cover typical startup time
retries should account for temporary failures (network blips)

Service-Specific Recommendations:

Service	Interval	Timeout	Retries	Start Period	Rationale
Redis	3s	2s	10	5s	Fast startup, lightweight check
PostgreSQL	5s	3s	10	10s	Moderate startup, db initialization
Keycloak	5s	5s	40	45s	Slow startup, complex initialization
OpenFGA	3s	3s	15	5s	Fast startup, gRPC ready quickly
Qdrant	5s	3s	10	10s	Moderate startup, index loading

Exit Codes

Health check commands must return proper exit codes:

0: Healthy (container ready)
1: Unhealthy (container not ready or failed)

Example with explicit exit codes:

healthcheck:
  test: ["CMD-SHELL", "pg_isready -U postgres && exit 0 || exit 1"]

Shell vs. Exec Form

CMD-SHELL Form (requires shell):

test: ["CMD-SHELL", "pg_isready -U postgres"]

Runs command through /bin/sh -c
Required for: pipes, redirections, variable expansion
Risk: Fails if /bin/sh doesn’t exist (distroless images)

CMD Form (direct exec):

test: ["CMD", "redis-cli", "ping"]

Executes command directly (no shell)
Preferred when possible (more portable)
Works in distroless/minimal images

Testing Health Checks

Verify Health Check Works

1. Start service and monitor health status:

docker-compose up -d postgres
watch -n 1 'docker-compose ps postgres'

2. Check health check logs:

docker inspect --format='{{json .State.Health}}' postgres | jq

3. Manually run health check command:

docker exec postgres pg_isready -U postgres
echo $?  # Should output 0 if healthy

Test in Different Image Variants

# Test with Alpine variant
docker run --rm postgres:17-alpine pg_isready --version

# Test with Debian variant
docker run --rm postgres:17 pg_isready --version

# Test with distroless (if applicable)
docker run --rm gcr.io/distroless/base ls /bin/sh
# Should fail if truly distroless

Validate Timing Parameters

# Add this to your docker-compose file temporarily
healthcheck:
  # Intentionally misconfigured to test validation
  interval: 2s
  timeout: 3s  # timeout > interval - should fail validation

Run validation:

docker-compose config
# Should report: healthcheck interval must be >= timeout

Troubleshooting

Health Check Never Becomes Healthy

Symptom: Container stays in starting or unhealthy state Debug steps:

Check if command exists:

docker exec <container> which pg_isready
docker exec <container> which redis-cli

Run health check manually:

docker exec <container> pg_isready -U postgres
# Note the exit code and output

Check service is actually running:

docker exec <container> ps aux
docker logs <container>

Verify ports are listening:

docker exec <container> netstat -tlnp
# or
docker exec <container> ss -tlnp

Check start_period is sufficient:

healthcheck:
  start_period: 60s  # Increase if service slow to start

Health Check Command Not Found

Symptom: executable file not found or command not found Solutions:

Verify command path:

docker exec <container> which pg_isready
docker exec <container> find / -name pg_isready 2>/dev/null

Check shell availability:

docker exec <container> ls -l /bin/sh
# If missing, use CMD form instead of CMD-SHELL

Use absolute paths:

healthcheck:
  test: ["CMD", "/usr/bin/redis-cli", "ping"]

Health Check Times Out

Symptom: Health checks fail with timeout Solutions:

Increase timeout:

healthcheck:
  timeout: 10s  # Increase for slow services

Use faster health check:

# Instead of full HTTP request
healthcheck:
  test: ["CMD-SHELL", "timeout 1 bash -c '</dev/tcp/localhost/8080'"]

Check network latency:

docker exec <container> time redis-cli ping
# Measure actual health check execution time

Permission Denied Errors

Symptom: Health check fails with permission errors Solutions:

Run as correct user:

healthcheck:
  test: ["CMD-SHELL", "su - postgres -c 'pg_isready'"]

Check file permissions:

docker exec <container> ls -l /opt/keycloak/bin/kc.sh

Use sudo if available:

healthcheck:
  test: ["CMD-SHELL", "sudo -u postgres pg_isready"]

Decision Tree: Choosing the Right Health Check

Complete Examples

Example 1: PostgreSQL with Optimal Settings

postgres:
  image: postgres:17-alpine
  environment:
    POSTGRES_USER: testuser
    POSTGRES_PASSWORD: testpass
    POSTGRES_DB: testdb
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -U testuser -d testdb"]
    interval: 5s
    timeout: 3s
    retries: 10
    start_period: 10s
  ports:
    - "5432:5432"
  networks:
    - app-network

Example 2: Redis with Auth

redis:
  image: redis:7-alpine
  command: redis-server --requirepass ${REDIS_PASSWORD}
  healthcheck:
    test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
    interval: 3s
    timeout: 2s
    retries: 10
    start_period: 5s
  environment:
    - REDIS_PASSWORD=${REDIS_PASSWORD}
  ports:
    - "6379:6379"

Example 3: Keycloak with Slow Startup

keycloak:
  image: quay.io/keycloak/keycloak:latest
  command: start-dev
  environment:
    KC_DB: postgres
    KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak
    KEYCLOAK_ADMIN: admin
    KEYCLOAK_ADMIN_PASSWORD: admin
  healthcheck:
    # Keycloak minimal image lacks curl - use kc.sh
    test: ["CMD-SHELL", "/opt/keycloak/bin/kc.sh show-config | grep -q 'kc.db' || exit 1"]
    interval: 5s
    timeout: 5s
    retries: 40
    start_period: 45s  # Keycloak needs time to initialize
  ports:
    - "8080:8080"
  depends_on:
    postgres:
      condition: service_healthy

Example 4: Multi-Service with Dependencies

version: '3.8'

services:
  postgres:
    image: postgres:17-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 10s

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 3s
      timeout: 2s
      retries: 10
      start_period: 5s

  app:
    build: .
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    healthcheck:
      test: ["CMD-SHELL", "timeout 1 bash -c '</dev/tcp/localhost/8000' || exit 1"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s
    ports:
      - "8000:8000"

References

Docker Compose Health Check Reference
Docker Health Check Best Practices
Minimal Container Images Security Guide
OpenAI Codex Finding: Docker Compose health check utilities warning
Project: docker-compose.test.yml - Production health check patterns

Summary: Quick Reference

Service	Best Health Check	Why
PostgreSQL	`pg_isready -U postgres`	Native, always available
Redis	`redis-cli ping`	Bundled client, fast
Keycloak	`kc.sh show-config \| grep -q 'kc.db'`	Native command, no curl needed
OpenFGA	`grpc_health_probe -addr=:8081`	gRPC standard
Qdrant	`timeout 1 bash -c '</dev/tcp/localhost/6333'`	No utilities in image
MongoDB	`mongosh --eval "db.adminCommand('ping')"`	Native client
RabbitMQ	`rabbitmqctl status`	Service management tool
Generic HTTP	`timeout 1 bash -c '</dev/tcp/localhost/PORT'`	Most portable

Golden Rules:

✅ Use service-native commands when possible
✅ Fallback to TCP checks for minimal images
✅ interval >= timeout (required)
✅ Test in actual container before deploying
❌ Don’t assume curl/wget exist
❌ Don’t use hardcoded shell paths

Last Updated: 2024-11-17 Maintained By: Infrastructure Team Related: docker-compose.test.yml, docker-compose.dev.yml

​Docker Compose Health Check Patterns - Portable Best Practices

​Overview

​Table of Contents

​Why Portable Health Checks Matter

​The Problem

​The Solution

​Benefits

​Common Pitfalls

​❌ Anti-Pattern 1: Assuming curl Exists

​❌ Anti-Pattern 2: Using wget Without Verification

​❌ Anti-Pattern 3: Hardcoded Shell Paths

​❌ Anti-Pattern 4: External Dependencies

​Portable Patterns by Service Type

​PostgreSQL

​Redis

​Keycloak

​MongoDB

​OpenFGA (gRPC Services)

​Qdrant (Vector Database)

​Elasticsearch

​RabbitMQ

​Health Check Configuration Guidelines

​Timing Parameters Best Practices

​Exit Codes

​Shell vs. Exec Form

​Testing Health Checks

​Verify Health Check Works

​Test in Different Image Variants

​Validate Timing Parameters

​Troubleshooting

​Health Check Never Becomes Healthy

​Health Check Command Not Found

​Health Check Times Out

​Permission Denied Errors

​Decision Tree: Choosing the Right Health Check

​Complete Examples

​Example 1: PostgreSQL with Optimal Settings

​Example 2: Redis with Auth

​Example 3: Keycloak with Slow Startup

​Example 4: Multi-Service with Dependencies

​References

​Summary: Quick Reference

Docker Compose Health Check Patterns - Portable Best Practices

Overview

Table of Contents

Why Portable Health Checks Matter

The Problem

The Solution

Benefits

Common Pitfalls

❌ Anti-Pattern 1: Assuming `curl` Exists

❌ Anti-Pattern 2: Using `wget` Without Verification

❌ Anti-Pattern 3: Hardcoded Shell Paths

❌ Anti-Pattern 4: External Dependencies

Portable Patterns by Service Type

PostgreSQL

Redis

Keycloak

MongoDB

OpenFGA (gRPC Services)

Qdrant (Vector Database)

Elasticsearch

RabbitMQ

Health Check Configuration Guidelines

Timing Parameters Best Practices

Exit Codes

Shell vs. Exec Form

Testing Health Checks

Verify Health Check Works

Test in Different Image Variants

Validate Timing Parameters

Troubleshooting

Health Check Never Becomes Healthy

Health Check Command Not Found

Health Check Times Out

Permission Denied Errors

Decision Tree: Choosing the Right Health Check

Complete Examples

Example 1: PostgreSQL with Optimal Settings

Example 2: Redis with Auth

Example 3: Keycloak with Slow Startup

Example 4: Multi-Service with Dependencies

References

Summary: Quick Reference