> ## Documentation Index
> Fetch the complete documentation index at: https://mcp-server-langgraph.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# v2.5.0 - Integration Testing

> Structured JSON logging, multi-platform log aggregation, containerized tests, and Infisical installation

<Note type="success">
  **Status**: Released (2025-10-15)
  **Breaking Changes**: None (fully backward compatible)
</Note>

### Overview

Version 2.5.0 delivers **enterprise-grade observability** and **developer experience** improvements with three major enhancements:

1. 📊 **Structured JSON Logging** with OpenTelemetry trace injection
2. ☁️ **Multi-Platform Log Aggregation** (AWS, GCP, Azure, Elasticsearch, Datadog, Splunk)
3. 🐳 **Containerized Integration Testing** with zero-config Docker setup
4. 🔧 **Multi-Track Infisical Installation** for flexible deployment

### What's New

#### 📊 Structured JSON Logging & Multi-Platform Log Aggregation

**Production-grade structured logging with OpenTelemetry trace injection** - automatically correlate logs with distributed traces across 6 cloud platforms.

<AccordionGroup>
  <Accordion title="Features" icon="check">
    * ✅ **Structured JSON Logs** - Machine-readable format with trace context
    * ✅ **Automatic Trace Injection** - trace\_id and span\_id in every log
    * ✅ **6 Platform Integrations** - AWS, GCP, Azure, Elasticsearch, Datadog, Splunk
    * ✅ **Zero Breaking Changes** - Backward compatible (can still use text format)
    * ✅ **Production Ready** - gzip compression, filtering, resource limits
    * ✅ **Easy Platform Switching** - One command to change log destinations
  </Accordion>

  <Accordion title="New Files Created (15)" icon="file">
    **Core Logging**:

    1. **src/mcp\_server\_langgraph/observability/json\_logger.py** (210 lines)
       * CustomJSONFormatter with OTEL trace injection
       * ISO 8601 timestamps, exception stack traces
       * Configurable indentation (compact/pretty-print)

    **OTLP Collector Configurations (7)**:
    2\. **monitoring/otel-collector/aws-cloudwatch.yaml** (168 lines) - CloudWatch + X-Ray
    3\. **monitoring/otel-collector/gcp-cloud-logging.yaml** (152 lines) - Cloud Logging + Trace
    4\. **monitoring/otel-collector/azure-monitor.yaml** (138 lines) - Application Insights
    5\. **monitoring/otel-collector/elasticsearch.yaml** (192 lines) - ELK Stack with ECS
    6\. **monitoring/otel-collector/datadog.yaml** (157 lines) - Unified APM
    7\. **monitoring/otel-collector/splunk.yaml** (179 lines) - Enterprise/Cloud dual mode
    8\. **monitoring/otel-collector/otel-collector.yaml** (enhanced) - Base config with logs pipeline

    **Deployment Automation**:
    9\. **scripts/switch-log-exporter.sh** (105 lines) - Platform switcher CLI
    10\. **deployments/kubernetes/base/otel-collector-deployment.yaml** (305 lines)

    * ConfigMap, Service, Deployment, ServiceAccount, PDB, HPA

    11. **deployments/kubernetes/overlays/aws/kustomization.yaml** - AWS EKS overlay
    12. **deployments/kubernetes/overlays/aws/otel-collector-config.yaml** - AWS config

    **Configuration**:
    13\. **.env.example** (updated +75 lines) - All platform credentials
    14\. **docker/docker-compose.yml** (enhanced) - Multi-platform OTLP support
    15\. **src/mcp\_server\_langgraph/core/config.py** - LOG\_FORMAT, LOG\_JSON\_INDENT settings
  </Accordion>

  <Accordion title="Usage Examples" icon="terminal">
    **Basic Usage** (defaults to JSON):

    ```python theme={null}
    from mcp_server_langgraph.observability.telemetry import logger

    # Logs are automatically JSON formatted with trace context
    logger.info("User logged in", extra={"user_id": "alice", "ip": "192.168.1.100"})
    ```

    **Output**:

    ```json theme={null}
    {
      "timestamp": "2025-10-15T14:23:45.123Z",
      "level": "INFO",
      "service": "mcp-server-langgraph",
      "trace_id": "0af7651916cd43dd8448eb211c80319c",
      "span_id": "b7ad6b7169203331",
      "message": "User logged in",
      "user_id": "alice",
      "ip_address": "192.168.1.100"
    }
    ```

    **Switch to Datadog**:

    ```bash theme={null}
    ./scripts/switch-log-exporter.sh datadog
    echo "DATADOG_API_KEY=your-key" >> .env
    echo "DATADOG_SITE=datadoghq.com" >> .env
    docker compose restart otel-collector
    ```

    **Opt-out of JSON** (use text format):

    ```bash theme={null}
    export LOG_FORMAT=text
    ```
  </Accordion>

  <Accordion title="Platform Support Matrix" icon="cloud">
    | Platform          | Logs                 | Metrics          | Traces        | Auth              |
    | ----------------- | -------------------- | ---------------- | ------------- | ----------------- |
    | **AWS**           | CloudWatch Logs      | EMF Metrics      | X-Ray         | IAM Role          |
    | **GCP**           | Cloud Logging        | Cloud Monitoring | Cloud Trace   | Workload Identity |
    | **Azure**         | Application Insights | App Insights     | App Insights  | Connection String |
    | **Elasticsearch** | Daily indices        | Prometheus       | Daily indices | BasicAuth/API Key |
    | **Datadog**       | Log Management       | Infrastructure   | APM           | API Key           |
    | **Splunk**        | HEC                  | HEC/SignalFx     | HEC/SAPM      | HEC Token         |
  </Accordion>

  <Accordion title="Kubernetes Deployment" icon="dharmachakra">
    **Production-ready OTLP collector** with autoscaling (2-10 replicas):

    ```bash theme={null}
    # Deploy to AWS EKS
    kubectl apply -k deployments/kubernetes/overlays/aws

    # Verify
    kubectl get pods -n mcp-server-langgraph -l app=otel-collector
    ```

    **Features**:

    * HorizontalPodAutoscaler (CPU 70%, Memory 80%)
    * PodDisruptionBudget (minAvailable=1)
    * Resource limits: 1 CPU, 512Mi memory
    * Health checks on :13133
    * Prometheus scraping annotations
  </Accordion>
</AccordionGroup>

***

#### 🐳 Containerized Integration Test Environment

**One command runs all integration tests** - zero manual setup required.

<AccordionGroup>
  <Accordion title="Features" icon="check">
    * ✅ **Zero manual setup** - One command does everything
    * ✅ **100% reliable** - Tests always pass in CI/CD (no more `continue-on-error: true`)
    * ✅ **Complete isolation** - No conflicts with local development
    * ✅ **Fast cleanup** - All data deleted automatically
    * ✅ **Reproducible** - Same environment everywhere (local, CI, team)
  </Accordion>

  <Accordion title="New Files Created (8)" icon="file">
    1. **docker/docker-compose.test.yml** - Test services configuration
       * PostgreSQL (in-memory via tmpfs)
       * OpenFGA (memory datastore)
       * Redis (no persistence)
       * Test runner container

    2. **docker/Dockerfile.test** - Optimized test runner image
       * Python 3.12-slim
       * Test dependencies only
       * Pre-configured environment

    3. **scripts/test-integration.sh** (200+ lines)
       * Orchestration script with options
       * Colored output, progress tracking
       * Automatic error handling

    4. **scripts/wait-for-services.sh** - Service health checker

    5. **tests/utils/docker.py** (250+ lines)
       * Docker test utilities
       * Service wait functions
       * TestEnvironment context manager

    6. **tests/utils/**init**.py** - Test utilities package

    7. **docs/development/integration-testing.md** (400+ lines)
       * Complete integration testing guide
       * Architecture overview
       * Debugging guide

    8. **tests/conftest.py** (MODIFIED)
       * Added real service fixtures
       * Auto-skip if not in Docker
  </Accordion>

  <Accordion title="Usage" icon="terminal">
    ```bash theme={null}
    # Run all integration tests (simplest way)
    make test-integration

    # That's it! Docker handles everything:
    # - Starts PostgreSQL, OpenFGA, Redis
    # - Waits for services to be healthy
    # - Runs pytest -m integration
    # - Cleans up containers
    ```

    **Advanced usage**:

    ```bash theme={null}
    # Rebuild and test
    make test-integration-build

    # Keep containers for debugging
    make test-integration-debug

    # Start services only (no tests)
    make test-integration-services

    # Clean up all test containers
    make test-integration-cleanup
    ```
  </Accordion>

  <Accordion title="Performance" icon="gauge">
    | Phase          | First Run  | Cached    |
    | -------------- | ---------- | --------- |
    | Build image    | \~60s      | \~5s      |
    | Start services | \~10s      | \~10s     |
    | Run tests      | \~30s      | \~30s     |
    | Cleanup        | \~2s       | \~2s      |
    | **Total**      | **\~100s** | **\~50s** |
  </Accordion>
</AccordionGroup>

**Migration from v2.4.0**:

* ✅ **Zero breaking changes** - Old `make test-integration-local` still works
* ✅ **Backward compatible** - All existing tests work unchanged
* ✅ **CI/CD update** - Just remove `continue-on-error: true` from workflows

***

#### 🔧 Infisical Docker-Based Build Solution

**Multi-track dependency strategy** - Choose what works best for your environment.

<AccordionGroup>
  <Accordion title="Problem Solved" icon="lightbulb">
    **Before**:

    * Required Rust toolchain for local development
    * Platform compatibility issues (especially ARM64)
    * Slow builds (\~5 minutes)
    * Failed on some systems

    **After**:

    * 5 installation options (pick what works)
    * No Rust requirement for most users
    * 10x faster builds with caching
    * Graceful fallback to environment variables
  </Accordion>

  <Accordion title="Installation Options" icon="list">
    1. **Docker Build** (Recommended)
       ```bash theme={null}
       docker compose up -d
       ```
       * Zero configuration
       * Automatically includes Infisical

    2. **uv extras** (One-line install)
       ```bash theme={null}
       uv sync --extra secrets
       ```
       * Easiest for local development

    3. **Pre-built wheels** (Fast, no Rust)
       ```bash theme={null}
       uv pip install infisical-python==2.3.5
       ```
       * For CI/CD pipelines

    4. **Build from source** (Latest version)
       ```bash theme={null}
       curl -sSf https://sh.rustup.rs | sh
       uv sync --extra secrets
       ```
       * For advanced users

    5. **Skip Infisical** (Use environment variables)
       ```bash theme={null}
       export JWT_SECRET_KEY=your-secret
       ```
       * Simplest option, no installation
  </Accordion>

  <Accordion title="New Files Created (5)" icon="file">
    1. **requirements-infisical.txt** - Separate dependency file
       * Comprehensive installation documentation
       * 5 installation options documented

    2. **docker/Dockerfile.infisical-builder** - Wheel builder image
       * Multi-stage build for creating pre-compiled wheels
       * Supports Python 3.10, 3.11, 3.12

    3. **scripts/build-infisical-wheels.sh** (270+ lines)
       * Automated wheel building
       * BuildKit support with caching

    4. **docs/deployment/infisical-installation.md** (500+ lines)
       * Comprehensive installation guide
       * Troubleshooting section
       * Performance comparisons

    5. **tests/test\_infisical\_optional.py** (20+ tests)
       * Tests graceful degradation
       * Environment variable fallback
  </Accordion>

  <Accordion title="Build Performance" icon="rocket">
    **With BuildKit caching**:

    * First build: \~5 minutes (unchanged)
    * Cached rebuild: \~30 seconds (10x improvement)
    * Wheel reuse: \~5 seconds (100x improvement)

    **docker/Dockerfile enhancements**:

    ```dockerfile theme={null}
    # syntax=docker/dockerfile:1.4

    # Cache mounts for apt packages
    RUN --mount=type=cache,target=/var/cache/apt \
        apt-get update && apt-get install -y build-essential

    # Cache mounts for uv
    COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
    RUN --mount=type=cache,target=/root/.cache/uv \
        uv sync --extra secrets
    ```
  </Accordion>
</AccordionGroup>

**Migration from v2.4.0**:

* ✅ **Backward compatible** - Docker users unaffected
* ✅ **Optional** - Infisical moved to `uv sync --extra secrets`
* ✅ **Graceful fallback** - App uses environment variables if Infisical unavailable

***

### Files Modified

#### Configuration Files

1. **pyproject.toml**
   * Moved `infisical-python` from core to optional dependencies
   * Added `[secrets]` and `[all]` extras

2. **requirements-pinned.txt**
   * Enhanced with 48-line installation guide

#### Docker Files

3. **docker/Dockerfile**
   * Added BuildKit syntax directive
   * Implemented cache mounts (5-10x faster)

4. **docker-compose.yml**
   * Fixed volume mounts for package structure

#### CI/CD

5. **.github/workflows/ci.yaml**
   * Added Docker Buildx setup
   * Integration tests now run in containers
   * **Removed** `continue-on-error: true`

#### Build System

6. **Makefile**
   * Added 6 new integration test targets

***

### Benefits

#### For Developers

* ✅ **One command** - `make test-integration` does everything
* ✅ **Zero setup** - No manual service configuration
* ✅ **Perfect isolation** - Tests don't affect local dev
* ✅ **Fast cleanup** - `docker compose down -v` removes everything
* ✅ **Debugging** - `--keep` flag preserves containers

#### For CI/CD

* ✅ **100% reliable** - No more allowed failures
* ✅ **Reproducible** - Same environment locally and in CI
* ✅ **Fast** - \~50s with caching
* ✅ **No secrets** - Everything runs in containers
* ✅ **Parallel** - Can run multiple jobs simultaneously

#### For Operations

* ✅ **Reduced build times** - 5-10x faster with cache
* ✅ **Smaller cache footprint** - Shared mounts
* ✅ **Reusable wheel artifacts** - For faster deployments
* ✅ **No breaking changes** - To deployments

***

### Upgrade Guide

#### From v2.4.0

**Step 1: Pull latest changes**

```bash theme={null}
git pull origin main
```

**Step 2: (Optional) Choose Infisical installation method**

```bash theme={null}
## Option 1: Docker (automatic)
docker compose up -d

## Option 2: uv extras
uv sync --extra secrets

## Option 3: Skip Infisical (use .env)
## No action needed - just use environment variables
```

**Step 3: Run integration tests**

```bash theme={null}
make test-integration
```

**Step 4: Update CI/CD**

```bash theme={null}
## Remove this line from .github/workflows/ci.yaml
## continue-on-error: true  # ← DELETE THIS
```

#### Testing the Upgrade

```bash theme={null}
## 1. Run all tests
make test

## 2. Run integration tests in Docker
make test-integration

## 3. Verify Infisical (if using)
python -c "import infisical_python; print('✓ Infisical installed')"

## 4. Check observability
make health-check
```

***

### What's Next?

#### Planned for v2.6.0

* Performance benchmarking suite
* Load testing with Locust
* API rate limiting
* Request throttling

#### Future Enhancements

* Chaos engineering tests
* Multi-region failover testing
* Automated performance regression detection
* Integration test parallelization

***

### Related Documentation

<CardGroup cols={2}>
  <Card title="Integration Testing Guide" icon="flask" href="/advanced/testing">
    Complete testing strategy
  </Card>

  <Card title="Infisical Installation" icon="key" href="/guides/infisical-setup">
    Secrets management setup
  </Card>

  <Card title="Docker Deployment" icon="docker" href="/deployment/docker">
    Docker best practices
  </Card>

  <Card title="CI/CD Pipeline" icon="gears" href="/deployment/platform/ci-cd">
    GitHub Actions workflows
  </Card>
</CardGroup>

***

<Note type="info">
  **Questions or feedback?**
  Join the discussion on [GitHub Discussions](https://github.com/vishnu2kmohan/mcp-server-langgraph/discussions)
</Note>