⚔️🛡️🎯 Gauntlet

Two-role adversarial MCP server that infers software correctness by observing how code behaves under sustained, targeted attack. Quality control for dark-factory environments where code is written by bots and verified by attack.

Run your service through the gauntlet. Point a host Claude Code agent at a running service, hand it the trial set, and the gauntlet is what the service survives. The host plays Attacker and Inspector; Gauntlet provides the deterministic tools (config loading, plan execution, risk-report assembly).

AI-written code can look correct while hiding behavioral failures. Traditional tests miss this because the same agent wrote code and tests. Gauntlet's Attacker context assumes the code is broken, and each Trial's blockers never load into that context, preserving a train/test split.

An Attacker uses a Trial aimed at a Target to generate Plans. Gauntlet's Drone executes those Plans as a User. An Inspector watches and surfaces Findings. Hidden Vitals are checked independently to produce a Clearance.

See docs/architecture.md for the model, docs/usage.md for the runbook, docs/development.md for dev setup.

Install

Gauntlet ships as a Claude Code plugin bundling the MCP server and the host skill:

claude plugin marketplace add coilysiren/gauntlet
claude plugin install gauntlet@coilysiren-gauntlet

Restart Claude Code so the skill, MCP server, and subagents register. Confirm with /mcp and "run gauntlet". No Anthropic creds needed; the host has auth.

Local dev: git clone ... && claude --plugin-dir path/to/gauntlet. Updates: enable auto-update under /plugin > Marketplaces, or /plugin marketplace update coilysiren-gauntlet + /reload-plugins.

The plugin delivers the MCP server, the gauntlet skill (orchestrator loop), gauntlet-author skill (spec to trial YAMLs), and gauntlet-attacker / -inspector / -holdout-evaluator subagents whose MCP allowlists enforce the train/test split.

MCP tools

• list_trials - attacker-safe views, no blockers.
• get_trial - full trial including blockers (orchestrator + holdout only).
• execute_plan - run a Plan against the SUT.
• start_run - init the per-run buffer, returns run_id.
• record_iteration / read_iteration_records - per-trial iteration buffer (rejects blocker text).
• record_holdout_result / read_holdout_results - holdout buffer.
• assemble_run_report - per-trial RiskReport + Clearance; persists confirmed failures.
• assemble_final_clearance - aggregate to one FinalClearance for the run.
• replay_finding - re-execute a stored finding's ReplayBundle.
• mutate_plans - deterministic plan variants (drop field, rotate users, etc.).
• recurring_failures - findings showing up in 2+ of the last N runs.

The train/test split is enforced via per-role MCP allowlists (see agents/). The Attacker subagent literally cannot call get_trial.

Project config

your-project/
├── .gauntlet/
│   └── trials/
│       ├── task_ownership.yaml
│       └── ...

Trials define reusable attack strategies. blockers are externally observable truths about expected behavior, never loaded into the Attacker context:

title: Users cannot modify each other's tasks
description: >
  The task API must enforce resource ownership.
blockers:
  - A PATCH request by a non-owner is rejected with 403
  - The task body is unchanged after an unauthorized PATCH attempt
  - A GET by the owner after an unauthorized PATCH returns the original data

If the SUT requires auth, the orchestrator passes user_headers to execute_plan (a dict[str, dict[str, str]] mapping user names to headers). Users without an entry fall back to X-User: <name>.

See also

• AGENTS.md, docs/FEATURES.md, docs/architecture.md, .coily/coily.yaml.
• Prior art comparison (RESTler, Schemathesis, ToolFuzz) lives in docs/architecture.md.

Cross-reference convention from coilysiren/agentic-os#59.