mcp-compliance-router

An MCP server that screens text for PHI/PII, classifies it, redacts it, and routes it to an approved model tier under a declarative policy, with an evaluation harness that measures exactly how well the screening works.

This is a reference implementation of the enterprise AI gateway pattern (screen, classify, redact, route, audit) at the smallest size that still demonstrates the whole loop. It is not a certified compliance product; see Design notes and limitations.

Why this exists

Enterprises in regulated industries cannot send raw text to whichever model is best for the task. Something must sit in the path that answers three questions before any call is made: what is in this text, where is it allowed to go, and what record proves the decision. In large healthcare environments that job belongs to an AI gateway. This repo is that pattern, small enough to read in one sitting and instrumented so its accuracy is measured, not assumed.

Architecture

flowchart LR
    A[Incoming text] --> B[Screen: regex entity detectors]
    B --> C{Classify}
    C -- restricted --> D[Redact] --> E[Restricted tier: private models]
    C -- sensitive --> F[Redact] --> G[Standard tier]
    C -- clean --> G
    D --> H[(Audit log: entity types + SHA-256, never raw text)]
    F --> H

Three modules, one policy file:

• core.py detects entities (SSN, MRN, NAME, DOB, ADDRESS, EMAIL, PHONE), classifies the document (clean, sensitive, restricted), redacts spans, and writes containment-friendly audit records that store entity types and a SHA-256 of the text, never the text itself.
• policy.py loads policy/routing_policy.yaml and turns a classification into a routing decision: target tier, allowed models, action, audit flag. Unknown classifications fail closed to the restricted rule.
• server.py exposes the loop as three MCP tools: screen_text, route_request, and get_audit_log.

Quickstart

pip install -e ".[mcp,dev]"
pytest                      # unit tests
python eval/run_eval.py     # regenerate eval/report.md
python -m compliance_router.server   # run the MCP server on stdio

Use it from Claude

Claude Code:

claude mcp add compliance-router -- python -m compliance_router.server

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "compliance-router": {
      "command": "python",
      "args": ["-m", "compliance_router.server"]
    }
  }
}

Then ask, for example: "Screen this support ticket with compliance-router and tell me which tier it routes to and why."

Evaluation

Screening quality is measured, not asserted. eval/dataset.jsonl holds 36 labeled synthetic examples (every identifier is fabricated), and eval/run_eval.py produces eval/report.md with per-entity precision, recall, and F1, document classification accuracy, routing tier accuracy, and a failure analysis listing each miss and false alarm. The current report is committed so the limits of the regex heuristics are visible in the repo.

Design notes and limitations

• Detection is regex-based on purpose: it keeps the reference readable and makes the evaluation honest. It is not production-grade PHI detection. Bare person names without an honorific or patient-context cue are missed by design of the current NAME heuristic, and the report shows it.
• The detector sits behind a small interface, so swapping in a stronger backend (Presidio, a clinical NER model, or an LLM-based screen with conformal abstention) changes one module and rerunning the harness shows exactly what the upgrade bought.
• The audit log is in-memory for the reference; production would ship the same records to a log platform. The records are designed to be safe to ship: entity types and hashes, never content.
• Fail-closed policy: any classification the policy does not recognize is blocked and audited.

Roadmap

• Pluggable detector backends (Presidio, clinical NER) behind the same interface, compared by the same harness
• Conformal abstention thresholds on the classifier, so uncertain documents escalate instead of guessing
• Policy lint: validate the YAML against tiers at startup
• Per-tenant policies keyed by MCP client identity

License

MIT. All example data is synthetic. Developed with AI-assisted workflows (Claude Code and similar agents) with human architecture and review.