Testing Strategy

Comprehensive testing strategy including unit, integration, property-based, and mutation testing

Testing Strategy

Test Pyramid

Test Execution

Local Testing:

## Unit tests only
make test-unit

## All automated tests
make test

## With coverage
make test-coverage

## Property-based tests
make test-property

## Benchmark tests
pytest -m benchmark -v --benchmark-only

## Mutation tests (slow!)
make test-mutation

CI Testing:

## Exact CI command
ENABLE_TRACING=false \
ENABLE_METRICS=false \
ENABLE_CONSOLE_EXPORT=false \
pytest -m unit --tb=line -q

Test Markers

• unit: Fast, isolated unit tests
• integration: Tests requiring external services
• property: Property-based tests with Hypothesis
• contract: MCP protocol compliance tests
• regression: Performance regression tests
• benchmark: Performance benchmarks
• mutation: Mutation testing (weekly schedule)

Mutation Testing

Purpose: Measure test effectiveness

Schedule: Weekly (too slow for every PR)

Command:

## Local execution
make test-mutation

## Or directly
mutmut run --paths-to-mutate=src/mcp_server_langgraph/
mutmut results
mutmut html  # Open html/index.html

Target Score: 80%+ mutation kills

Benchmark Testing

Purpose: Track performance regressions

Execution: Every PR + weekly

Thresholds:

• Agent response: p95 < 5s
• LLM call: p95 < 10s
• Authorization: p95 < 50ms

Alert: Automatic PR comment if 20%+ slower

Baseline Tracking:

## Run benchmarks
pytest -m benchmark -v --benchmark-only --benchmark-autosave

## Compare against baseline
pytest -m benchmark --benchmark-compare

Next Steps

Workflows

Configure test workflows

Deployment

Deploy tested code

Back to Overview

Return to CI/CD overview