Skip to main content

​
Testing Strategy

​
Test Pyramid

​
Test Execution

Local Testing: 
## Unit tests only
make test-unit

## All automated tests
make test

## With coverage
make test-coverage

## Property-based tests
make test-property

## Benchmark tests
pytest -m benchmark -v --benchmark-only

## Mutation tests (slow!)
make test-mutation
CI Testing: 
## Exact CI command
ENABLE_TRACING=false \
ENABLE_METRICS=false \
ENABLE_CONSOLE_EXPORT=false \
pytest -m unit --tb=line -q

​
Test Markers

  • unit: Fast, isolated unit tests
  • integration: Tests requiring external services
  • property: Property-based tests with Hypothesis
  • contract: MCP protocol compliance tests
  • regression: Performance regression tests
  • benchmark: Performance benchmarks
  • mutation: Mutation testing (weekly schedule)

​
Mutation Testing

Purpose: Measure test effectiveness Schedule: Weekly (too slow for every PR) Command: 
## Local execution
make test-mutation

## Or directly
mutmut run --paths-to-mutate=src/mcp_server_langgraph/
mutmut results
mutmut html  # Open html/index.html
Target Score: 80%+ mutation kills

​
Benchmark Testing

Purpose: Track performance regressions Execution: Every PR + weekly Thresholds:
  • Agent response: p95 < 5s
  • LLM call: p95 < 10s
  • Authorization: p95 < 50ms
Alert: Automatic PR comment if 20%+ slower Baseline Tracking: 
## Run benchmarks
pytest -m benchmark -v --benchmark-only --benchmark-autosave

## Compare against baseline
pytest -m benchmark --benchmark-compare

​
Next Steps

Workflows

Configure test workflows

Deployment

Deploy tested code

Back to Overview

Return to CI/CD overview