Skip to main content

​
Keycloak readOnlyRootFilesystem Implementation

​
Overview

This document describes the implementation of readOnlyRootFilesystem: true for Keycloak pods using a pre-optimized GHCR image. This security hardening prevents filesystem modifications at runtime, reducing attack surface.

​
Current Status

Date: 2025-12-08 Status: ✅ IMPLEMENTED - readOnlyRootFilesystem: true enabled Image: ghcr.io/vishnu2kmohan/keycloak-optimized:26.4.2 File: deployments/base/keycloak-deployment.yaml

​
Solution Summary

​
Approach: Pre-Built Optimized Keycloak Image

The solution uses a custom Keycloak image built via GitHub Actions that pre-compiles Quarkus at build time, eliminating the need for JIT compilation at runtime. Key Benefits:
  • ✅ readOnlyRootFilesystem: true - Full security hardening
  • ✅ Faster container startup (no runtime augmentation)
  • ✅ Matches docker-compose.test.yml for dev/prod parity
  • ✅ Reduced attack surface

​
Architecture

┌─────────────────────────────────────────────────────────────┐
│  GitHub Actions (.github/workflows/build-keycloak-image.yaml)│
├─────────────────────────────────────────────────────────────┤
│  1. Build Stage: kc.sh build --db=postgres                  │
│  2. Push to GHCR: ghcr.io/vishnu2kmohan/keycloak-optimized │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│  Kubernetes Deployment                                       │
├─────────────────────────────────────────────────────────────┤
│  image: ghcr.io/vishnu2kmohan/keycloak-optimized:26.4.2    │
│  args: ["start", "--optimized"]                             │
│  readOnlyRootFilesystem: true                               │
└─────────────────────────────────────────────────────────────┘

​
Implementation Details

​
Dockerfile (docker/Dockerfile.keycloak)

# Stage 1: Build optimized Keycloak
FROM quay.io/keycloak/keycloak:26.4.2 as builder

WORKDIR /opt/keycloak

# Build optimized configuration
RUN /opt/keycloak/bin/kc.sh build \
    --db=postgres \
    --http-enabled=true \
    --health-enabled=true \
    --metrics-enabled=true \
    --http-relative-path=/authn

# Stage 2: Runtime image
FROM quay.io/keycloak/keycloak:26.4.2

# Copy pre-built artifacts
COPY --from=builder /opt/keycloak/lib/quarkus/ /opt/keycloak/lib/quarkus/
COPY --from=builder /opt/keycloak/lib/lib/ /opt/keycloak/lib/lib/

# Set non-root user
USER 10000

ENTRYPOINT ["/opt/keycloak/bin/kc.sh"]

​
Kubernetes Deployment (deployments/base/keycloak-deployment.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: keycloak
spec:
  template:
    spec:
      containers:
      - name: keycloak
        image: ghcr.io/vishnu2kmohan/keycloak-optimized:26.4.2
        args:
        - start
        - --optimized
        - --http-enabled=true
        - --http-port=8080
        - --hostname-strict=false
        - --health-enabled=true
        - --metrics-enabled=true

        env:
        - name: KC_HTTP_RELATIVE_PATH
          value: /authn  # For Traefik gateway routing

        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true  # ✅ Enabled!
          runAsNonRoot: true
          runAsUser: 10000
          runAsGroup: 10000
          capabilities:
            drop:
            - ALL

        volumeMounts:
        - name: tmp
          mountPath: /tmp
        - name: var-tmp
          mountPath: /var/tmp
        - name: cache
          mountPath: /opt/keycloak/data/tmp
        - name: work-dir
          mountPath: /opt/keycloak/data
        - name: providers
          mountPath: /opt/keycloak/providers
        - name: themes
          mountPath: /opt/keycloak/themes

      volumes:
      - name: tmp
        emptyDir: {}
      - name: var-tmp
        emptyDir: {}
      - name: cache
        emptyDir: {}
      - name: work-dir
        emptyDir: {}
      - name: providers
        emptyDir: {}
      - name: themes
        emptyDir: {}

​
Health Check Paths

With KC_HTTP_RELATIVE_PATH=/authn, health checks use:
ProbePathPurpose
startupProbe/authn/health/startedContainer initialization
livenessProbe/authn/health/liveProcess alive check
readinessProbe/authn/health/readyReady to serve traffic

​
Security Controls

​
Pod Security Context

securityContext:
  runAsNonRoot: true
  runAsUser: 10000
  runAsGroup: 10000
  fsGroup: 1000
  seccompProfile:
    type: RuntimeDefault

​
Container Security Context

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 10000
  runAsGroup: 10000
  capabilities:
    drop:
    - ALL

​
EmptyDir Volume Isolation

Writable paths are isolated to ephemeral emptyDir volumes:
  • /tmp - Temporary files
  • /var/tmp - Additional temporary storage
  • /opt/keycloak/data/tmp - Keycloak cache directory
  • /opt/keycloak/data - Keycloak work directory
  • /opt/keycloak/providers - Custom provider JARs
  • /opt/keycloak/themes - Custom themes
IMPORTANT: Do NOT mount emptyDir at /opt/keycloak/lib - this overwrites Quarkus runtime JARs and causes ClassNotFoundException.

​
GitHub Actions Workflow

Image is built automatically via .github/workflows/build-keycloak-image.yaml: 
name: Build Keycloak Optimized Image

on:
  push:
    paths:
      - 'docker/Dockerfile.keycloak'
    branches:
      - main
  workflow_dispatch:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Login to GHCR
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          file: docker/Dockerfile.keycloak
          push: true
          tags: |
            ghcr.io/${{ github.repository_owner }}/keycloak-optimized:26.4.2
            ghcr.io/${{ github.repository_owner }}/keycloak-optimized:latest

​
Verification

​
Test Read-Only Filesystem

# Should fail - filesystem is read-only
kubectl exec -it keycloak-xxx -c keycloak -- touch /test
# Expected: touch: cannot touch '/test': Read-only file system

# Should succeed - emptyDir is writable
kubectl exec -it keycloak-xxx -c keycloak -- touch /tmp/test
# Expected: Success

​
Test Health Endpoints

kubectl exec -it keycloak-xxx -c keycloak -- curl http://localhost:8080/authn/health/ready
# Expected: {"status": "UP", "checks": [...]}

​
Test Admin Console

Access via Traefik gateway: 
https://<ingress-host>/authn/admin/master/console/

​
Troubleshooting

​
Issue: Container CrashLoopBackOff

Symptoms: Pod fails to start with Quarkus errors Cause: Using stock Keycloak image without --optimized flag Solution: Ensure using GHCR optimized image: 
image: ghcr.io/vishnu2kmohan/keycloak-optimized:26.4.2
args: ["start", "--optimized"]

​
Issue: ClassNotFoundException

Symptoms: Java class loading errors at startup Cause: emptyDir mounted at /opt/keycloak/lib overwrites JAR files Solution: Remove /opt/keycloak/lib volume mount. Only mount:
  • /opt/keycloak/data
  • /opt/keycloak/providers
  • /opt/keycloak/themes

​
Issue: Health Check Failures

Symptoms: Pods never become ready Cause: Wrong health check paths (missing /authn prefix) Solution: Update probe paths: 
readinessProbe:
  httpGet:
    path: /authn/health/ready  # Include KC_HTTP_RELATIVE_PATH
    port: http

​
References

  • docker/Dockerfile.keycloak - Optimized image build
  • .github/workflows/build-keycloak-image.yaml - CI/CD pipeline
  • deployments/base/keycloak-deployment.yaml - Base deployment
  • deployments/base/.trivyignore - Security scan suppressions
  • docker-compose.test.yml - Test environment parity
Last Updated: 2025-12-08 Status: ✅ Implemented Owner: DevOps/Security Team Priority: Complete (Security Hardening)