ADR-0049: Pytest Fixture Consolidation and Organization

Architecture Decision Record: ADR-0049: Pytest Fixture Consolidation and Organization

ADR-0049: Pytest Fixture Consolidation and Organization

Date: 2025-11-07

Status

Accepted

Category

Testing & Quality

Decision Makers

Test Infrastructure Team

Related ADRs

ADR-0042 (Dependency Injection)

Context

During a comprehensive test infrastructure review prompted by OpenAI Codex findings, we discovered 25 duplicate init_test_observability autouse fixtures scattered across test modules. This created several issues:

1. Performance Overhead: Observability initialization ran 25 times (once per module) instead of once per session
2. Maintenance Burden: Changes to observability initialization required updating 25+ files
3. Potential Conflicts: Multiple initializations could lead to race conditions or state pollution
4. Confusion: Unclear which fixture would run first or take precedence
5. Anti-Pattern: Violates DRY (Don’t Repeat Yourself) principle

Investigation Findings

The investigation revealed:

• 25 duplicate fixtures with identical implementations across:

  • tests/test_auth.py:22
  • tests/test_health_check.py:12
  • tests/property/test_auth_properties.py:57
  • …and 22 more files

• Performance Impact: Each fixture initialized logging, tracing, and metrics systems

• Test Execution: Using pytest --collect-only showed 3115 tests total

• Fixture Scope: All duplicates were scope="module" with autouse=True

TDD Validation Approach

Following Test-Driven Development principles:

1. RED Phase: Created tests/test_fixture_organization.py to detect duplicates - test FAILED
2. GREEN Phase: Consolidated fixtures into tests/conftest.py - test PASSED
3. REFACTOR Phase: Added automation and preventive measures

Decision

We will consolidate all session/module-scoped autouse fixtures into tests/conftest.py and enforce this pattern through automated tooling.

Fixture Organization Rules

1. Session-scoped autouse fixtures MUST be in tests/conftest.py

   • Example: observability initialization, global test configuration
   • Rationale: Single initialization point, clear lifecycle

2. Module-scoped autouse fixtures SHOULD be in tests/conftest.py

   • Exception: Test-specific configuration that doesn’t apply globally
   • Must be documented if placed elsewhere

3. Function-scoped fixtures CAN be in individual test files

   • These are test-specific and don’t create overhead
   • Should still avoid duplication

4. Test-specific configuration fixtures should NOT be autouse

   • Use pytest.mark.usefixtures() or direct fixture usage
   • Example: test_circuit_breaker_config in conftest.py

Implementation

Consolidated Fixture (tests/conftest.py:70-102):

@pytest.fixture(scope="session", autouse=True)
def init_test_observability():
    """
    Initialize observability system for all tests (session-scoped).

    Configuration:
    - Text logging (no JSON in tests)
    - No file logging (console only)
    - LangSmith tracing disabled
    - OpenTelemetry backend for tracing

    Consolidates 25+ duplicate module-scoped fixtures.
    """
    from mcp_server_langgraph.core.config import Settings
    from mcp_server_langgraph.observability.telemetry import init_observability, is_initialized

    if not is_initialized():
        test_settings = Settings(
            log_format="text",
            enable_file_logging=False,
            langsmith_tracing=False,
            observability_backend="opentelemetry",
        )
        init_observability(settings=test_settings, enable_file_logging=False)

    yield

Removed: 25 duplicate fixtures from individual test files (-476 lines)

Consequences

Positive

1. Performance Improvement

   • Observability initialized once per session instead of 25 times
   • Estimated 96% reduction in initialization overhead
   • Faster test suite execution

2. Maintainability

   • Single source of truth in tests/conftest.py
   • Changes only need to be made in one place
   • Clear fixture lifecycle and dependencies

3. Code Quality

   • Removed 476 lines of duplicate code
   • Improved code organization
   • Better adherence to DRY principle

4. Reliability

   • Eliminated potential race conditions
   • Consistent test environment across all modules
   • Predictable fixture execution order

5. Developer Experience

   • Clearer test structure
   • Easier to understand fixture behavior
   • Better documentation

Negative

1. Initial Learning Curve

   • Developers need to know where fixtures are defined
   • Requires understanding of pytest fixture scopes

2. Less Flexibility

   • Cannot easily override observability settings per module
   • Module-specific configuration requires different patterns

Mitigation Strategies

1. Documentation: Updated ~/.claude/CLAUDE.md with fixture best practices
2. Enforcement: Created pre-commit hook to prevent duplicate fixtures
3. Testing: Added tests/test_fixture_organization.py to detect violations
4. Plugin: Created tests/conftest_fixtures_plugin.py for runtime enforcement
5. Automation: Created scripts to detect and remove duplicates

Preventive Measures

1. Test-Based Prevention

# tests/test_fixture_organization.py
def test_no_duplicate_autouse_fixtures():
    """Validates no duplicate autouse fixtures exist."""
    # Scans all test files
    # Fails if duplicates found
    # Provides clear error messages with file locations

2. Pre-Commit Hook

# .pre-commit-config.yaml
- id: validate-fixture-organization
  name: Validate Pytest Fixture Organization
  entry: python -m pytest tests/test_fixture_organization.py -v
  files: ^tests/.*\.py$

3. Runtime Plugin

# tests/conftest_fixtures_plugin.py
class FixtureOrganizationPlugin:
    """Enforces fixture organization at pytest collection time."""
    # Scans fixtures during test collection
    # Fails immediately if violations found
    # Prevents tests from running with bad fixtures

4. Automation Scripts

• scripts/remove_duplicate_fixtures.py - AST-based fixture removal
• scripts/fix_decorator_leftover.py - Cleanup orphaned decorators

Alternatives Considered

Option 1: Keep Duplicates, Document Pattern

• Rejected: Doesn’t solve performance or maintenance issues
• Maintains status quo with known problems

Option 2: Use pytest-fixtures Plugin

• Rejected: Additional dependency, doesn’t solve our specific issue
• Would still need custom validation logic

Option 3: Module-Level Configuration Files

• Rejected: More complex, harder to maintain
• Doesn’t align with pytest best practices

Option 4: Fixture Factories

• Considered: Could work but adds complexity
• Session-scoped autouse fixture is simpler and more explicit

References

• Evidence Report: docs-internal/CODEX_FINDINGS_VALIDATION_REPORT.md
• Test Implementation: tests/test_fixture_organization.py
• Plugin Implementation: tests/conftest_fixtures_plugin.py
• Pre-commit Hook: .pre-commit-config.yaml:226-243
• Global TDD Guidelines: ~/.claude/CLAUDE.md (Pytest Fixture Best Practices)
• Pytest Documentation: https://docs.pytest.org/en/stable/how-to/fixtures.html#scope-sharing-fixtures-across-classes-modules-packages-or-session

Implementation Status

• ✅ Complete: Fixture consolidation
• ✅ Complete: Test-based validation
• ✅ Complete: Pre-commit hook
• ✅ Complete: Runtime plugin
• ✅ Complete: Documentation updates
• ✅ Complete: Automation scripts

Metrics

• Files Modified: 35 files
• Lines Removed: -476 lines (duplicate code)
• Lines Added: +969 lines (tests, docs, automation)
• Net Change: +482 lines
• Test Coverage: 100% of fixture organization rules
• Validation: 3115+ tests passing with consolidated fixtures

Review and Update

This ADR should be reviewed when:

• Pytest major version updates
• Significant changes to test infrastructure
• New fixture patterns emerge
• Performance issues with current approach

Last Reviewed: 2025-11-07 Next Review: 2026-01-01 (or when pytest 9.0 released)