Architecture

System architecture and component overview

Overview

MCP Server with LangGraph is a production-ready, multi-layered architecture designed for enterprise AI applications. It combines stateful AI agents, enterprise authentication, fine-grained authorization, and comprehensive observability.

High-Level Architecture

Core Components

MCP Server

Model Context Protocol server - Exposes AI agents as standard tools.

stdio Transport

Standard input/output for CLI integration

• Direct agent interaction
• Shell scripting support
• Development and testing

StreamableHTTP

HTTP API with streaming support

• REST endpoints
• Server-Sent Events (SSE)
• Production deployments

Key Features:

• Protocol-compliant MCP implementation
• Tool registration and discovery
• Streaming responses
• Error handling and retries

Code Location: src/mcp_server_langgraph/server.py

---

LangGraph Agent

Stateful AI agent with functional API and conditional routing.

## Agent graph structure
START → authenticate → authorize → execute_tool → END
                ↓           ↓            ↓
          unauthorized  forbidden   tool_error
                ↓           ↓            ↓
               END         END       retry/END

Components:

State Management

AgentState - Pydantic model for type-safe state

class AgentState(BaseModel):
    messages: List[Message]
    user_id: str
    authorized: bool
    context: Dict[str, Any]
    tool_results: List[Any]

Features:

• Immutable state transitions
• Type validation
• Checkpointing support

Code Location: src/mcp_server_langgraph/agent.py

---

Authentication Layer

Pluggable authentication with multiple provider support.

Providers:

InMemoryUserProvider

Development and testing

• Pre-defined users (alice, bob, admin)
• No external dependencies
• Fast iteration
• Zero configuration

users = {
    "alice": User(username="alice", roles=["admin"]),
    "bob": User(username="bob", roles=["user"])
}

KeycloakUserProvider

Production SSO

• OpenID Connect / OAuth2
• JWKS token verification
• Refresh token rotation
• Role/group synchronization

provider = KeycloakUserProvider(
    server_url="https://sso.yourdomain.com",
    realm="mcp-server-langgraph",
    client_id="langgraph-client"
)

Custom Providers

Extensible architecture

Implement UserProvider interface:

class CustomProvider(UserProvider):
    async def authenticate(self, username, password):
        # Custom auth logic
        return AuthResponse(...)

    async def verify_token(self, token):
        # Custom verification
        return TokenVerification(...)

Code Location: src/mcp_server_langgraph/auth/

---

Authorization Layer

Fine-grained, relationship-based access control with OpenFGA.

Authorization Model:

type user

type organization
  relations
    define admin: [user]
    define member: [user] or admin

type tool
  relations
    define owner: [user]
    define executor: [user, organization#member]
    define viewer: [user, organization#member]

Features:

• Relationship-based permissions
• Hierarchical roles (admin → member → viewer)
• Multi-tenancy support
• Audit logging
• Keycloak role synchronization

Code Location: src/mcp_server_langgraph/auth/openfga.py

---

Session Management

Flexible session storage with Redis or in-memory backends.

Architecture

Code Location: src/mcp_server_langgraph/auth/session.py

---

LLM Integration

Multi-LLM routing via LiteLLM with automatic fallback.

Supported Providers (100+):

• Cloud: Anthropic, OpenAI, Google, Azure, AWS Bedrock
• Open Source: Ollama (Llama, Mistral, Qwen, DeepSeek)
• Custom: Bring your own endpoints

Features:

• Automatic fallback on errors
• Load balancing
• Rate limiting
• Cost tracking
• Response caching

Configuration:

LLM_PROVIDER=anthropic
MODEL_NAME=claude-sonnet-4-5-20250929
ENABLE_FALLBACK=true
FALLBACK_MODELS=["gemini-2.5-flash-002", "gpt-5.1"]

Code Location: src/mcp_server_langgraph/llm_factory.py

---

Observability

Dual observability stack - OpenTelemetry + LangSmith.

OpenTelemetry

Distributed tracing and metrics

Traces:

• End-to-end request flow
• LLM call timing
• Authorization decisions
• Tool executions

Metrics (30+):

• Request rate, latency, errors
• Authentication success/failure
• Authorization decisions
• LLM token usage
• Session lifecycle

Code Location: src/mcp_server_langgraph/observability/

---

Secrets Management

Secure secret storage with Infisical or cloud-native solutions.

Infisical

Centralized secret management

• End-to-end encryption
• Secret versioning
• Access controls
• Audit logging
• Secret rotation

INFISICAL_CLIENT_ID=your-client-id
INFISICAL_CLIENT_SECRET=your-secret
INFISICAL_PROJECT_ID=project-id

Code Location: src/mcp_server_langgraph/config.py

---

Data Flow

Request Flow (Authenticated Request)

Authentication Flow (Keycloak SSO)

---

Deployment Architectures

Development (Docker Compose)

Command: docker compose up

---

Production (Kubernetes)

Deployment: See Kubernetes Guide or Helm Guide

---

Design Principles

Modularity

• Pluggable authentication providers
• Swappable session stores
• Custom authorization models
• Extensible tool framework

Type Safety

• Pydantic models for all data
• Type hints throughout
• Validation at boundaries
• Property-based testing

Observability First

• Trace every request
• Structured logging
• Comprehensive metrics
• Correlation IDs

Security by Default

• Authentication required
• Authorization on all tools
• Secrets in secure stores
• Encrypted communications

Production Ready

• Health checks
• Graceful shutdown
• Error recovery
• Rate limiting
• Circuit breakers

---

Technology Stack

Core

• Python 3.10+
• LangGraph: Stateful agent framework
• LiteLLM: Multi-LLM router
• Pydantic: Data validation
• FastAPI: HTTP server (StreamableHTTP)

---

Next Steps

Quick Start

Get started in 5 minutes

Authentication

Configure authentication

Authorization

Setup fine-grained permissions

Observability

Enable tracing and metrics

---

Enterprise-Grade Architecture: Production-ready components designed for scale and reliability!