Copilot Guardian

The CI Healer That Knows When AI Is Wrong

When GitHub Actions fails, Guardian analyzes logs with multi-hypothesis reasoning, proposes risk-stratified patches, and blocks unsafe AI output before it touches your code.

Production-ready | 90-second setup | Full audit trail

Challenge Submission | Demo | Why This Matters | Real Evidence | 90s Test | Quick Start | Docs

npx copilot-guardian@latest run --repo YOUR/REPO --last-failed --fast

Guardian demonstrates five advanced Copilot usage patterns under real CI failures:

1. Multi-hypothesis reasoning: generate three competing theories with explicit confidence and evidence.
2. Risk-stratified generation: synthesize Conservative/Balanced/Aggressive patches at different risk levels.
3. Deterministic fail-closed guardrails: override AI decisions when bypass patterns or malformed outputs are detected.
4. MCP-enriched context: use Model Context Protocol for deeper repository context.
5. Complete artifact trail: export analysis.json, raw Copilot responses, and patch index for auditability.

Runtime clarification:

• Guardian is a terminal CLI tool built with @github/copilot-sdk as the primary runtime.
• gh copilot CLI is available as an optional fallback for local reproducibility.
• Copilot interactions are logged to .raw.txt files for traceability.

Real proof:

• Conservative patch was auto-blocked because Copilot returned malformed JSON.
• See Real Output Showcase.

Runtime: 3m43s | Profile: --fast --max-log-chars 20000

What happens in this run:

1. Analyze a real CI failure with multi-hypothesis reasoning.
2. Generate three strategies: Conservative, Balanced, Aggressive.
3. Run independent quality review for each strategy.
4. Block malformed AI output with fail-closed guardrails.
5. Export raw artifacts for full auditability.

Browse the exact files from this demo:

• examples/real-output/standard/

The problem:

• AI-assisted CI fixes can silently introduce security bypasses or malformed outputs.
• A green-looking suggestion is not proof of safe behavior.
• Most tools generate one patch and hope it works.

Guardian's approach:

• Generate multiple hypotheses and patch strategies instead of one guess.
• Validate every candidate with deterministic controls.
• Fail closed (NO_GO) when output is malformed or risky.

Real case from the demo above:

1. Copilot returned malformed quality JSON for Conservative review.
2. Deterministic guard detected parse failure.
3. Guardian blocked with NO_GO verdict.
4. Broken output was never auto-applied.

This is fail-closed engineering: trust AI, but verify with hard rules.

This is not a mock. Files below are unmodified outputs from actual Guardian runs.

Conservative strategy was automatically rejected due to malformed Copilot JSON:

{
  "verdict": "NO_GO",
  "risk_level": "high",
  "slop_score": 1.0,
  "reasons": [
    "Parse error: Unbalanced JSON object in Copilot response - missing closing brace"
  ],
  "suggested_adjustments": [
    "Re-run quality review with stricter JSON-only response",
    "Inspect copilot.quality.*.raw.txt for malformed output"
  ]
}

What happened:

1. AI generated a patch for src/engine/github.ts.
2. Quality review response was incomplete JSON.
3. Deterministic guard caught the parse error.
4. System blocked the patch (fail-closed enforcement).
5. User was protected from applying broken code.

Source files:

• quality_review.conservative.json
• copilot.quality.conservative.raw.txt

Source files:

• patch_options.json
• fix.balanced.patch
• fix.aggressive.patch

examples/real-output/standard/
  patch_options.json
  fix.conservative.patch
  fix.balanced.patch
  fix.aggressive.patch
  quality_review.*.json
  copilot.*.raw.txt

Browse all files:

• examples/real-output/standard/

Additional evidence:

• examples/real-output/abstain/guardian.report.json

Key insight: Guardian does not trust AI blindly; deterministic checks can override model output.

Prerequisites:

• gh auth status succeeds.
• GitHub Copilot is enabled for your account/session.

npx copilot-guardian@latest run \
  --repo flamehaven01/copilot-guardian \
  --last-failed \
  --show-options \
  --fast \
  --max-log-chars 20000

Expected output:

1. Structured diagnosis in analysis.json.
2. Strategy index in patch_options.json.
3. Safety verdicts in quality_review.*.json.

For extended trace mode (slower), add --show-reasoning.

• Node.js 18+
• GitHub CLI (gh) authenticated
• GitHub Copilot subscription (SDK access)

# Global install
npm i -g copilot-guardian@latest

# Or run without install
npx copilot-guardian@latest --help

Package:

• https://www.npmjs.com/package/copilot-guardian

# Stable demo profile
copilot-guardian run \
  --repo owner/repo \
  --last-failed \
  --show-options \
  --fast \
  --max-log-chars 20000

# Analysis only
copilot-guardian analyze \
  --repo owner/repo \
  --run-id <run_id> \
  --fast

# Evaluate multiple failed runs
copilot-guardian eval \
  --repo owner/repo \
  --failed-limit 5 \
  --fast

# Interactive follow-up
copilot-guardian debug \
  --repo owner/repo \
  --last-failed

Full architecture:

• docs/ARCHITECTURE.md

graph TB
    A[GitHub Actions Failure] --> B[Guardian CLI]
    B --> C[Context Fetch]
    C --> D[Multi-Hypothesis Analysis]
    D --> E[Copilot SDK]
    E --> F[Patch Strategies]
    F --> G[Deterministic Quality Guard]
    G --> H{GO?}
    H -->|NO_GO| I[Reject and Re-diagnose]
    H -->|GO| J[Patch Candidate]
    J --> K[Safe Branch PR or Auto-Heal]

Guardian intentionally abstains for non-patchable failure classes:

• 401/403 authentication failures
• token permission errors
• API rate-limit or infrastructure unavailability

When abstaining:

• abstain.report.json is emitted with classification.
• Patch generation is skipped.
• User receives actionable recommendations.

Example:

• examples/real-output/abstain/guardian.report.json

Artifacts are generated under .copilot-guardian/:

Tip:

• Check examples/real-output/ for evidence from standard and abstain paths.

• Real output evidence: examples/real-output/README.md
• Demo walkthrough: examples/demo-failure/README.md
• Architecture: docs/ARCHITECTURE.md
• Changelog: CHANGELOG.md
• Security: SECURITY.md
• Contributing: CONTRIBUTING.md

# Test on your repo (90 seconds)
npx copilot-guardian@latest run \
  --repo YOUR/REPO \
  --last-failed \
  --fast

# Or reproduce this exact demo
npx copilot-guardian@latest run \
  --repo flamehaven01/copilot-guardian \
  --last-failed \
  --fast

Questions?

• Open an issue
• Read architecture docs
• Browse real output examples

MIT License. See LICENSE.