Architecture

System architecture and component overview

Overview

MCP Server with LangGraph is a production-ready, multi-layered architecture designed for enterprise AI applications. It combines stateful AI agents, enterprise authentication, fine-grained authorization, and comprehensive observability.

High-Level Architecture

Core Components

MCP Server

Model Context Protocol server - Exposes AI agents as standard tools.

stdio Transport

Standard input/output for CLI integration

• Direct agent interaction
• Shell scripting support
• Development and testing

StreamableHTTP

HTTP API with streaming support

• REST endpoints
• Server-Sent Events (SSE)
• Production deployments

Key Features:

• Protocol-compliant MCP implementation
• Tool registration and discovery
• Streaming responses
• Error handling and retries

Code Location: src/mcp_server_langgraph/server.py

---

LangGraph Agent

Stateful AI agent with functional API and conditional routing.

## Agent graph structure
START → authenticate → authorize → execute_tool → END
                ↓           ↓            ↓
          unauthorized  forbidden   tool_error
                ↓           ↓            ↓
               END         END       retry/END

Components:

State Management

AgentState - Pydantic model for type-safe state

class AgentState(BaseModel):
    messages: List[Message]
    user_id: str
    authorized: bool
    context: Dict[str, Any]
    tool_results: List[Any]

Features:

• Immutable state transitions
• Type validation
• Checkpointing support

Tool Execution

Dynamic tool routing based on LLM decisions

tools = {
    "chat": chat_tool,
    "search": search_tool,
    # Add custom tools
}

Authorization:

• Per-tool permission checks
• OpenFGA relationship validation
• Fallback admin access

Checkpointing

State persistence for conversation history

• In-memory checkpoints (development)
• Redis checkpoints (production)
• Resume interrupted conversations
• Audit trail

Code Location: src/mcp_server_langgraph/agent.py

---

Authentication Layer

Pluggable authentication with multiple provider support.

Providers:

InMemoryUserProvider

Development and testing

• Pre-defined users (alice, bob, admin)
• No external dependencies
• Fast iteration
• Zero configuration

users = {
    "alice": User(username="alice", roles=["admin"]),
    "bob": User(username="bob", roles=["user"])
}

KeycloakUserProvider

Production SSO

• OpenID Connect / OAuth2
• JWKS token verification
• Refresh token rotation
• Role/group synchronization

provider = KeycloakUserProvider(
    server_url="https://sso.yourdomain.com",
    realm="mcp-server-langgraph",
    client_id="langgraph-client"
)

Custom Providers

Extensible architecture

Implement UserProvider interface:

class CustomProvider(UserProvider):
    async def authenticate(self, username, password):
        # Custom auth logic
        return AuthResponse(...)

    async def verify_token(self, token):
        # Custom verification
        return TokenVerification(...)

Code Location: src/mcp_server_langgraph/auth/

---

Authorization Layer

Fine-grained, relationship-based access control with OpenFGA.

Authorization Model:

type user

type organization
  relations
    define admin: [user]
    define member: [user] or admin

type tool
  relations
    define owner: [user]
    define executor: [user, organization#member]
    define viewer: [user, organization#member]

Features:

• Relationship-based permissions
• Hierarchical roles (admin → member → viewer)
• Multi-tenancy support
• Audit logging
• Keycloak role synchronization

Code Location: src/mcp_server_langgraph/auth/openfga.py

---

Session Management

Flexible session storage with Redis or in-memory backends.

Architecture

Features

Session Lifecycle:

• Creation with metadata
• Retrieval and validation
• Refresh with sliding window
• Revocation (single or bulk)

Advanced:

• Concurrent session limits
• IP/User agent tracking
• TTL management
• Automatic cleanup

Configuration

# Redis backend
SESSION_BACKEND=redis
REDIS_URL=redis://redis:6379/0
SESSION_TTL_SECONDS=86400
SESSION_SLIDING_WINDOW=true
SESSION_MAX_CONCURRENT=5

# In-memory backend
SESSION_BACKEND=memory

Code Location: src/mcp_server_langgraph/auth/session.py

---

LLM Integration

Multi-LLM routing via LiteLLM with automatic fallback.

Supported Providers (100+):

• Cloud: Anthropic, OpenAI, Google, Azure, AWS Bedrock
• Open Source: Ollama (Llama, Mistral, Qwen, DeepSeek)
• Custom: Bring your own endpoints

Features:

• Automatic fallback on errors
• Load balancing
• Rate limiting
• Cost tracking
• Response caching

Configuration:

LLM_PROVIDER=anthropic
MODEL_NAME=claude-sonnet-4-5-20250929
ENABLE_FALLBACK=true
FALLBACK_MODELS=["gemini-2.5-flash-002", "gpt-5.1"]

Code Location: src/mcp_server_langgraph/llm_factory.py

---

Observability

Dual observability stack - OpenTelemetry + LangSmith.

OpenTelemetry

Distributed tracing and metrics

Traces:

• End-to-end request flow
• LLM call timing
• Authorization decisions
• Tool executions

Metrics (30+):

• Request rate, latency, errors
• Authentication success/failure
• Authorization decisions
• LLM token usage
• Session lifecycle

LangSmith

LLM-specific observability

• Prompt tracking
• LLM response analysis
• Chain visualization
• Cost attribution
• A/B testing
• Evaluations

LANGSMITH_TRACING=true
LANGSMITH_API_KEY=your-key
LANGSMITH_PROJECT=mcp-server-langgraph

Structured Logging

JSON logging with context

{
  "timestamp": "2025-10-12T10:30:00Z",
  "level": "INFO",
  "service": "mcp-server-langgraph",
  "trace_id": "abc123...",
  "span_id": "def456...",
  "user_id": "alice",
  "event": "authorization_check",
  "resource": "tool:chat",
  "authorized": true,
  "duration_ms": 15
}

Code Location: src/mcp_server_langgraph/observability/

---

Secrets Management

Secure secret storage with Infisical or cloud-native solutions.

Infisical

Centralized secret management

• End-to-end encryption
• Secret versioning
• Access controls
• Audit logging
• Secret rotation

INFISICAL_CLIENT_ID=your-client-id
INFISICAL_CLIENT_SECRET=your-secret
INFISICAL_PROJECT_ID=project-id

Cloud Providers

Native secret managers

• GCP: Secret Manager
• AWS: Secrets Manager
• Azure: Key Vault
• Kubernetes: Secrets + RBAC

Automatic injection via workload identity.

Local Development

.env files with validation

# .env
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=AIza...
JWT_SECRET_KEY=$(openssl rand -base64 32)

Never commit .env files to Git!

Code Location: src/mcp_server_langgraph/config.py

---

Data Flow

Request Flow (Authenticated Request)

Authentication Flow (Keycloak SSO)

---

Deployment Architectures

Development (Docker Compose)

Command: docker compose up

---

Production (Kubernetes)

Deployment: See Kubernetes Guide or Helm Guide

---

Design Principles

Modularity

• Pluggable authentication providers
• Swappable session stores
• Custom authorization models
• Extensible tool framework

Type Safety

• Pydantic models for all data
• Type hints throughout
• Validation at boundaries
• Property-based testing

Observability First

• Trace every request
• Structured logging
• Comprehensive metrics
• Correlation IDs

Security by Default

• Authentication required
• Authorization on all tools
• Secrets in secure stores
• Encrypted communications

Production Ready

• Health checks
• Graceful shutdown
• Error recovery
• Rate limiting
• Circuit breakers

---

Technology Stack

Core

• Python 3.10+
• LangGraph: Stateful agent framework
• LiteLLM: Multi-LLM router
• Pydantic: Data validation
• FastAPI: HTTP server (StreamableHTTP)

Authentication

• Keycloak: SSO and OIDC
• PyJWT: Token validation
• Redis: Session storage

Authorization

• OpenFGA: Relationship-based authz
• PostgreSQL: Authorization data

Observability

• OpenTelemetry: Tracing & metrics
• Jaeger: Trace visualization
• Prometheus: Metrics storage
• Grafana: Dashboards
• LangSmith: LLM observability

Infrastructure

• Docker: Containerization
• Kubernetes: Orchestration
• Helm: Package management
• Infisical: Secrets management

---

Next Steps

Quick Start

Get started in 5 minutes

Authentication

Configure authentication

Authorization

Setup fine-grained permissions

Observability

Enable tracing and metrics

---

Enterprise-Grade Architecture: Production-ready components designed for scale and reliability!