[repository-quality] Repository Quality Improvement Report - Error Message UX & Actionability

[github-actions[bot]]

🎯 Repository Quality Improvement Report - Error Message UX & Actionability

Analysis Date: 2026-05-19
Focus Area: Error Message User Experience & Actionability
Strategy Type: Custom
Custom Area: Yes — Error message quality is critical for developer tools like gh-aw but not covered by standard categories

Executive Summary

gh-aw processes 4,817 error sites across the codebase, with 560+ fmt.Errorf calls in the workflow package alone. While the project has a strong foundation with structured error types (WorkflowValidationError, OperationError, ConfigurationError) that include suggestions fields, only 44 workflow package errors (~7.9%) use the structured NewValidationError pattern. The remaining 695+ errors use generic fmt.Errorf without actionable guidance.

Key Findings:

• ✅ Excellent structured error design with Field/Value/Reason/Suggestion pattern
• ✅ 37 CLI files use console formatting for consistent terminal output
• ✅ 267 errors use %w wrapping for error chain preservation
• ⚠️ 518 workflow errors lack suggestions — users get "failed to X" without next steps
• ⚠️ 2,884 generic error messages use negative language ("invalid", "cannot", "must", "failed") without constructive alternatives
• ⚠️ Mixed error patterns — some validation uses NewValidationError (with suggestions), most uses plain fmt.Errorf

Impact: When compilation or validation fails, users must interpret cryptic error messages and guess solutions. This increases support burden, slows development velocity, and creates friction in the developer experience.

Full Analysis Report

Focus Area: Error Message User Experience & Actionability

Current State Assessment

Error Infrastructure:

• WorkflowValidationError: Structured errors with field/value/reason/suggestion (44 uses)
• OperationError: Operation failures with entity context and suggestions
• ConfigurationError: Config errors with actionable suggestions
• ErrorCollector: Aggregates multiple errors with fail-fast or collect-all modes

Metrics Collected:

Metric	Value	Status
Total error sites (pkg/)	4,817	i️ Large codebase
Workflow package fmt.Errorf calls	560	⚠️ Mostly unstructured
Workflow validation files	148	✅ Good coverage
Structured validation errors	44	❌ Only 7.9%
Errors with %w wrapping	267	✅ Good practice
Errors without suggestions	518+	❌ Missing guidance
Files using console formatting	37	✅ Consistent style
Generic negative error patterns	2,884	⚠️ Needs improvement

Findings

Strengths

1. Excellent error type design — WorkflowValidationError includes structured Field/Value/Reason/Suggestion fields with timestamps
2. Actionable examples exist — pkg/stringutil/pat_validation.go provides exemplary error messages:

   errors.New("classic personal access tokens (ghp_...) are not supported for Copilot. 
              Please create a fine-grained PAT at https://github.com/settings/personal-access-tokens/new")

3. "Did you mean?" fuzzy matching — pkg/cli/run_workflow_validation.go suggests similar input names when validation fails
4. Error wrapping — 267 uses of %w preserve error chains for debugging
5. Console formatting — 37 CLI files use console.FormatErrorMessage() for consistent terminal output
6. Cache validation example — pkg/workflow/cache_validation.go shows best practice:

   NewValidationError(
       "tools.cache-memory.key",
       key,
       "cache key must not reference github.run_id — every run would write to a unique cache slot",
       "Remove github.run_id from the key. The compiler appends it automatically...\n\nExample:\n\ntools:\n  cache-memory:\n    key: my-data-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}\n    # ✓ compiler adds run_id automatically"
   )

Areas for Improvement

1. Low adoption of structured errors (High severity)

   • Only 44 of 560+ workflow errors use NewValidationError
   • 92% of errors use generic fmt.Errorf("failed to X: %w", err) without suggestions
   • Inconsistent: some validators use structured errors, most don't

2. Missing actionable suggestions (High severity)

   • 518+ workflow package errors have no suggestion field
   • Example of poor UX: fmt.Errorf("failed to build activation job: %w", err) — what should the user do?
   • Compare to good UX: cache validation provides example YAML showing correct configuration

3. Generic negative language (Medium severity)

   • 2,884 error messages use "invalid", "cannot", "must", "failed" without explaining why or offering alternatives
   • Example: "invalid repo format: %s" vs better: "invalid repo format '%s' — expected 'owner/repo' format (e.g., 'github/gh-aw')"

4. Inconsistent error context (Medium severity)

   • Some errors show expected vs actual: NewValidationError(field, value, reason, suggestion)
   • Most errors lack this context: fmt.Errorf("failed to parse YAML: %w", err)
   • Users can't tell what value caused the error without reading source code

5. Missing file/line context (Low severity)

   • Validation errors don't include workflow file path or line number
   • Users must search entire workflow to find the problematic field
   • Compare to compiler errors that show file:line:column

Detailed Analysis

Error Pattern Distribution:

1. Validation errors (148 files) — Should all use NewValidationError, but most use fmt.Errorf
2. Compilation errors (pkg/workflow/compiler*.go) — Mix of structured and unstructured
3. CLI errors (pkg/cli/) — Better use of suggestions via "Did you mean?" fuzzy matching
4. Network/API errors — Generic wrapping without recovery suggestions

Examples of Good Error UX:

// ✅ EXCELLENT — Explains problem, provides exact solution with link
return errors.New(
    "classic personal access tokens (ghp_...) are not supported for Copilot. " +
    "Please create a fine-grained PAT at https://github.com/settings/personal-access-tokens/new"
)

// ✅ GOOD — Shows expected format with example
suggestions = append(suggestions, fmt.Sprintf(
    "'%s' -> did you mean '%s'?", 
    providedName, 
    strings.Join(matches, "', '")
))

// ✅ GOOD — Provides example YAML configuration
NewValidationError(
    field, value, reason,
    "Example:\n\ntools:\n  cache-memory:\n    key: my-data\n    # ✓ correct"
)

Examples of Poor Error UX:

// ❌ POOR — No suggestion, user doesn't know how to fix
return fmt.Errorf("failed to build activation job: %w", err)

// ❌ POOR — Doesn't show actual value or expected format
return fmt.Errorf("failed to parse lock file YAML: %w", err)

// ❌ POOR — Negative language without constructive alternative
return errors.New("not in a git repository")
// Better: "not in a git repository — run 'git init' or 'cd' to a git repository"

// ❌ POOR — Missing context about which job/step failed
return fmt.Errorf("failed to process steps for job '%s': %w", jobName, err)
// Better: Use NewValidationError with field="jobs.%s.steps", show which step index

Migration Path:

1. Phase 1 (High priority): Migrate all validation errors in pkg/workflow/*validation*.go (148 files) to use NewValidationError
2. Phase 2 (Medium priority): Migrate compilation errors in pkg/workflow/compiler*.go to structured errors
3. Phase 3 (Low priority): Add file/line context to validation errors for better debugging

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: Split the following tasks into individual work items.

Improvement Tasks

Task 1: Migrate High-Impact Validation Errors to Structured Format

Priority: High
Estimated Effort: Medium
Focus Area: Error Message UX & Actionability

Description: Migrate the 10 most frequently hit validation error paths to use NewValidationError with actionable suggestions. Focus on errors that users encounter during initial workflow setup and compilation.

Target files:

• pkg/workflow/compiler_filters_validation.go — Event filter validation (branches/paths conflicts)
• pkg/workflow/strict_mode_network_validation.go — Network permission errors
• pkg/workflow/mcp_property_validation.go — MCP server configuration errors
• pkg/workflow/pip_validation.go — Python dependency validation
• pkg/cli/run_workflow_validation.go — Workflow input validation

Acceptance Criteria:

• All validation errors in target files use NewValidationError(field, value, reason, suggestion)
• Each suggestion includes concrete example YAML showing correct configuration
• Error messages use constructive language ("expected X" not "invalid X")
• Test coverage includes checking suggestion field content
• Run make agent-finish successfully before committing

Code Region: pkg/workflow/*validation*.go, pkg/cli/run_workflow_validation.go

Migrate validation errors in these files to use the structured `NewValidationError` pattern:

1. `pkg/workflow/compiler_filters_validation.go`
2. `pkg/workflow/strict_mode_network_validation.go`
3. `pkg/workflow/mcp_property_validation.go`
4. `pkg/workflow/pip_validation.go`
5. `pkg/cli/run_workflow_validation.go`

For each error site:

**Current pattern:**
```go
return fmt.Errorf("invalid X: %s", value)

Target pattern:

return NewValidationError(
    "field.path",              // Field that failed validation
    value,                     // Actual value provided
    "clear explanation of why this failed",  // Reason
    "Actionable suggestion with example:\n\nExample YAML:\n  field:\n    setting: correct_value",  // Suggestion
)

Requirements:

• Use NewValidationError from pkg/workflow/workflow_errors.go
• Field names should match frontmatter YAML path (e.g., "on.pull_request.branches")
• Reason should explain the constraint in plain language
• Suggestion must include concrete example YAML showing correct usage
• Update corresponding tests to verify suggestion field is present
• Follow examples from pkg/workflow/cache_validation.go (lines 38-47)

Example transformation:

// BEFORE
return fmt.Errorf("%s event cannot specify both 'branches' and 'branches-ignore'", eventName)

// AFTER
return NewValidationError(
    fmt.Sprintf("on.%s.branches", eventName),
    "both branches and branches-ignore specified",
    "branches and branches-ignore are mutually exclusive per GitHub Actions requirements",
    "Use either 'branches' to include specific branches, or 'branches-ignore' to exclude them, but not both.\n\nExample (include):\n\non:\n  pull_request:\n    branches: [main, develop]\n\nExample (exclude):\n\non:\n  pull_request:\n    branches-ignore: [experimental]",
)

Run make agent-finish before creating the PR to ensure all tests pass.

---

#### Task 2: Add Fuzzy Matching "Did You Mean?" Suggestions to Common Configuration Errors

**Priority**: High
**Estimated Effort**: Small

**Focus Area**: Error Message UX & Actionability

**Description:** Implement fuzzy matching for common user errors in workflow configuration fields. When a user provides an invalid value, suggest the closest valid alternative (like `pkg/cli/run_workflow_validation.go` does for input names).

**Target areas:**
- Engine names ("copilot", "claude", "codex", "custom") — suggest on typos like "copiliot" → "copilot"
- MCP server names from registry — suggest when user misspells server name
- GitHub event names ("issues", "pull_request", etc.) — suggest on typos
- Permissions scopes ("contents", "issues", "pull_requests") — suggest corrections

**Acceptance Criteria:**
- [ ] Fuzzy matching function added to `pkg/stringutil` (Levenshtein distance or similar)
- [ ] Engine name validation suggests closest match when invalid engine provided
- [ ] MCP server validation suggests similar registered server names
- [ ] GitHub event name validation suggests closest event type
- [ ] Test coverage includes fuzzy match suggestions in error messages
- [ ] Run `make agent-finish` successfully before committing

**Code Region:** `pkg/stringutil/*`, `pkg/workflow/compiler*.go`

```markdown
Add "Did you mean?" fuzzy matching to common configuration errors.

**Step 1:** Create fuzzy matching utility in `pkg/stringutil/fuzzy.go`:

```go
package stringutil

import "strings"

// FuzzyMatch returns the closest match from candidates to the input string.
// Returns empty string if no good match found (distance > threshold).
// Uses Levenshtein distance or similar simple algorithm.
func FuzzyMatch(input string, candidates []string, maxDistance int) string {
    // Implementation: calculate edit distance to each candidate
    // Return closest match if distance <= maxDistance, otherwise ""
}

// SuggestAlternatives returns up to N closest matches with distances.
func SuggestAlternatives(input string, candidates []string, maxSuggestions int) []string {
    // Return top N suggestions sorted by distance
}

Step 2: Apply fuzzy matching to engine validation (example location: pkg/workflow/compiler.go or similar):

validEngines := []string{"copilot", "claude", "codex", "custom"}
if !slices.Contains(validEngines, engineName) {
    suggestion := stringutil.FuzzyMatch(engineName, validEngines, 2)
    if suggestion != "" {
        return NewValidationError(
            "engine",
            engineName,
            fmt.Sprintf("unknown engine '%s'", engineName),
            fmt.Sprintf("Did you mean '%s'?\n\nValid engines: %s",
                suggestion, strings.Join(validEngines, ", ")),
        )
    }
    // Fall back to generic error if no close match
}

Step 3: Add fuzzy matching to:

• MCP server name validation (check against registry)
• GitHub event name validation (on.X where X is invalid)
• Permission scope validation

Step 4: Write tests in pkg/stringutil/fuzzy_test.go:

• Test edit distance calculations
• Test suggestion generation
• Test integration with validation errors

Reference implementation: pkg/cli/run_workflow_validation.go lines 55-62 (input name suggestions).

Run make agent-finish before creating the PR.

---

#### Task 3: Enhance Compiler Errors with File and Line Context

**Priority**: Medium
**Estimated Effort**: Medium
**Focus Area**: Error Message UX & Actionability

**Description:** Add file path and line number context to validation errors so users can quickly locate the problematic configuration. Currently, errors like "invalid value for field X" don't tell users where in the workflow file the error occurred.

**Acceptance Criteria:**
- [ ] `WorkflowValidationError` includes optional `FilePath` and `Line` fields
- [ ] Parser extracts line numbers during frontmatter parsing
- [ ] Error messages format as `workflow.md:15: invalid value for 'engine'` (similar to compiler errors)
- [ ] Backward compatible — errors without line context still work
- [ ] Test coverage includes line number extraction and formatting
- [ ] Run `make agent-finish` successfully before committing

**Code Region:** `pkg/workflow/workflow_errors.go`, `pkg/parser/*`

```markdown
Add file and line number context to validation errors for better debugging UX.

**Step 1:** Extend `WorkflowValidationError` in `pkg/workflow/workflow_errors.go`:

```go
type WorkflowValidationError struct {
    Field      string
    Value      string
    Reason     string
    Suggestion string
    FilePath   string  // NEW: workflow file path
    Line       int     // NEW: line number in file (0 if unknown)
    Timestamp  time.Time
}

func (e *WorkflowValidationError) Error() string {
    var b strings.Builder
    
    // NEW: Show file:line prefix if available
    if e.FilePath != "" {
        if e.Line > 0 {
            fmt.Fprintf(&b, "%s:%d: ", filepath.Base(e.FilePath), e.Line)
        } else {
            fmt.Fprintf(&b, "%s: ", filepath.Base(e.FilePath))
        }
    }
    
    fmt.Fprintf(&b, "Validation failed for field '%s'", e.Field)
    // ... rest of error formatting
}

Step 2: Update NewValidationError signature (backward compatible):

// NewValidationErrorWithContext creates validation error with file/line context
func NewValidationErrorWithContext(
    field, value, reason, suggestion, filePath string, 
    line int,
) *WorkflowValidationError {
    return &WorkflowValidationError{
        Field:      field,
        Value:      value,
        Reason:     reason,
        Suggestion: suggestion,
        FilePath:   filePath,
        Line:       line,
        Timestamp:  time.Now(),
    }
}

// Keep existing NewValidationError for backward compatibility
func NewValidationError(field, value, reason, suggestion string) *WorkflowValidationError {
    return NewValidationErrorWithContext(field, value, reason, suggestion, "", 0)
}

Step 3: Modify parser to track line numbers during frontmatter extraction (pkg/parser/frontmatter_extractor.go or similar):

type FrontmatterWithContext struct {
    Data      map[string]any
    FieldLines map[string]int  // Maps field path to line number
    FilePath   string
}

Step 4: Update validators to use context when available:

// In validation functions, accept file context
func validateEngine(engine string, ctx *FrontmatterContext) error {
    if engine == "" {
        return NewValidationErrorWithContext(
            "engine",
            engine,
            "engine is required",
            "Add 'engine: copilot' to your workflow frontmatter",
            ctx.FilePath,
            ctx.FieldLines["engine"],
        )
    }
    return nil
}

Step 5: Update tests to verify line numbers appear in error messages.

Note: Line number tracking adds complexity to the parser. Start with high-impact validators (engine, on, tools) and expand coverage incrementally.

Run make agent-finish before creating the PR.

---

#### Task 4: Create Error Message Style Guide and Linter

**Priority**: Medium
**Estimated Effort**: Large
**Focus Area**: Error Message UX & Actionability

**Description:** Document error message best practices and create a custom linter to enforce them. This prevents regression and ensures new errors follow established patterns.

**Acceptance Criteria:**
- [ ] Create `docs/error-message-style-guide.md` with examples and anti-patterns
- [ ] Add custom linter in `pkg/linters/error_messages.go` using Go analysis API
- [ ] Linter detects:
  - [ ] `fmt.Errorf` in validation files without using `NewValidationError`
  - [ ] Error messages starting with "invalid"/"cannot"/"failed" without suggestions
  - [ ] Errors lacking example YAML in suggestions
- [ ] Integrate linter into `make lint`
- [ ] Document linter in `CONTRIBUTING.md`
- [ ] Run `make agent-finish` successfully before committing

**Code Region:** `docs/`, `pkg/linters/`, `Makefile`

```markdown
Create error message style guide and automated linter to enforce best practices.

**Step 1:** Create `docs/error-message-style-guide.md`:

```markdown
# Error Message Style Guide

## Principles

1. **Be Constructive** — Focus on what to do, not what's wrong
2. **Show Examples** — Include YAML snippets showing correct usage
3. **Use Structured Errors** — Prefer `NewValidationError` over `fmt.Errorf`
4. **Preserve Context** — Use `%w` to wrap underlying errors
5. **Guide to Resolution** — Every error should suggest next steps

## Validation Errors

✅ **DO:**
```go
return NewValidationError(
    "on.pull_request.branches",
    value,
    "branches and branches-ignore are mutually exclusive",
    "Use 'branches' OR 'branches-ignore', not both.\n\nExample:\n\non:\n  pull_request:\n    branches: [main]",
)

❌ DON'T:

return fmt.Errorf("invalid branches configuration")
return errors.New("branches and branches-ignore cannot both be specified")

Operation Errors

✅ DO:

return NewOperationError(
    "fetch",
    "workflow",
    workflowID,
    err,
    "Check that the workflow exists and you have read permissions",
)

❌ DON'T:

return fmt.Errorf("failed to fetch workflow %s: %w", workflowID, err)

Language Patterns

❌ Avoid	✅ Prefer
"invalid X"	"expected format: X"
"cannot do X"	"X requires Y — add Y to your config"
"must not do X"	"X is not allowed because Y — use Z instead"
"failed to X"	"could not X: [reason] — try [solution]"

Suggestions

Every suggestion should:

1. Explain the constraint in plain language
2. Provide concrete example YAML
3. Link to docs for complex topics

✅ GOOD:

Suggestion: Remove github.run_id from the cache key. The compiler appends it automatically.

Example:

tools:
  cache-memory:
    key: my-data-${{ env.GH_AW_WORKFLOW_ID_SANITIZED }}

❌ BAD:

Suggestion: Fix the cache key

**Step 2:** Create custom linter `pkg/linters/error_messages/error_messages.go`:

```go
package error_messages

import (
    "go/ast"
    "golang.org/x/tools/go/analysis"
)

var Analyzer = &analysis.Analyzer{
    Name: "error_messages",
    Doc:  "checks error messages follow style guide",
    Run:  run,
}

func run(pass *analysis.Pass) (any, error) {
    // Inspect all files in *validation*.go
    for _, file := range pass.Files {
        filename := pass.Fset.Position(file.Pos()).Filename
        if !strings.Contains(filename, "validation") {
            continue
        }
        
        ast.Inspect(file, func(n ast.Node) bool {
            call, ok := n.(*ast.CallExpr)
            if !ok {
                return true
            }
            
            // Check for fmt.Errorf in validation files
            if isCallTo(call, "fmt", "Errorf") {
                pass.Reportf(call.Pos(), 
                    "use NewValidationError instead of fmt.Errorf in validation files")
            }
            
            // Check for negative language without suggestions
            if hasNegativeLanguage(call) && !hasStructuredError(call) {
                pass.Reportf(call.Pos(),
                    "error message uses negative language — consider using NewValidationError with actionable suggestion")
            }
            
            return true
        })
    }
    return nil, nil
}

// Helper functions: isCallTo, hasNegativeLanguage, hasStructuredError

Step 3: Integrate into Makefile:

lint:
	`@echo` "Running custom linters..."
	go run ./pkg/linters/error_messages ./pkg/workflow
	golangci-lint run

Step 4: Add tests in pkg/linters/error_messages/error_messages_test.go:

func TestErrorMessageLinter(t *testing.T) {
    testdata := analysistest.TestData()
    analysistest.Run(t, testdata, Analyzer, "testdata/validation_errors")
}

Step 5: Document in CONTRIBUTING.md:

## Error Message Guidelines

See [Error Message Style Guide](docs/error-message-style-guide.md) for detailed guidance.

The `error_messages` linter enforces these guidelines automatically.

Reference: Existing linter pattern in pkg/linters/ (if any exist, or reference Go analysis examples).

Run make agent-finish before creating the PR.

---

#### Task 5: Add Error Recovery Suggestions to Compiler Flow

**Priority**: Low
**Estimated Effort**: Small
**Focus Area**: Error Message UX & Actionability

**Description:** When compilation fails with multiple errors, provide a summary with prioritized recovery suggestions. Users should see "Fix these 3 errors first" rather than a wall of 20 error messages.

**Acceptance Criteria:**
- [ ] `ErrorCollector` groups errors by category (validation, compilation, runtime)
- [ ] Summary shows error count per category with priority
- [ ] Top 3 most impactful errors highlighted with "Fix first" label
- [ ] Remaining errors collapsed under "View all errors" section
- [ ] Test coverage includes multi-error scenarios with prioritization
- [ ] Run `make agent-finish` successfully before committing

**Code Region:** `pkg/workflow/workflow_errors.go`, `pkg/cli/compile*.go`

```markdown
Enhance error reporting to provide prioritized recovery guidance when compilation fails.

**Step 1:** Extend `ErrorCollector` in `pkg/workflow/workflow_errors.go`:

```go
type ErrorCategory string

const (
    CategoryValidation  ErrorCategory = "validation"
    CategoryCompilation ErrorCategory = "compilation"
    CategoryRuntime     ErrorCategory = "runtime"
)

type CategorizedError struct {
    Error    error
    Category ErrorCategory
    Priority int  // 1 = critical, 2 = high, 3 = medium, 4 = low
}

type ErrorCollector struct {
    errors   []CategorizedError  // Changed from []error
    failFast bool
}

func (c *ErrorCollector) AddWithPriority(
    err error, 
    category ErrorCategory, 
    priority int,
) error {
    if err == nil {
        return nil
    }
    
    if c.failFast {
        return err
    }
    
    c.errors = append(c.errors, CategorizedError{
        Error:    err,
        Category: category,
        Priority: priority,
    })
    return nil
}

func (c *ErrorCollector) SummarizeWithPriority() string {
    if len(c.errors) == 0 {
        return ""
    }
    
    // Group by category
    byCategory := make(map[ErrorCategory][]CategorizedError)
    for _, ce := range c.errors {
        byCategory[ce.Category] = append(byCategory[ce.Category], ce)
    }
    
    var summary strings.Builder
    fmt.Fprintf(&summary, "Found %d error(s):\n\n", len(c.errors))
    
    // Show category counts
    for cat, errs := range byCategory {
        fmt.Fprintf(&summary, "  %s: %d\n", cat, len(errs))
    }
    
    // Sort all errors by priority
    sort.Slice(c.errors, func(i, j int) bool {
        return c.errors[i].Priority < c.errors[j].Priority
    })
    
    // Show top 3 critical errors
    fmt.Fprintf(&summary, "\n🔴 Fix these first:\n\n")
    for i := 0; i < min(3, len(c.errors)); i++ {
        fmt.Fprintf(&summary, "%d. %s\n", i+1, c.errors[i].Error)
    }
    
    // Collapse remaining errors
    if len(c.errors) > 3 {
        fmt.Fprintf(&summary, "\n<details>\n<summary>View %d more error(s)</summary>\n\n", len(c.errors)-3)
        for i := 3; i < len(c.errors); i++ {
            fmt.Fprintf(&summary, "%d. %s\n", i+1, c.errors[i].Error)
        }
        fmt.Fprintf(&summary, "\n</details>\n")
    }
    
    return summary.String()
}

Step 2: Update validation call sites to use priority:

collector := NewErrorCollector(false)

// Critical: missing required fields
collector.AddWithPriority(
    validateEngine(config.Engine),
    CategoryValidation,
    1,  // Priority 1 = critical
)

// High: configuration errors
collector.AddWithPriority(
    validateTools(config.Tools),
    CategoryValidation,
    2,  // Priority 2 = high
)

// Medium: optional optimizations
collector.AddWithPriority(
    validateCacheConfig(config.Cache),
    CategoryValidation,
    3,  // Priority 3 = medium
)

Step 3: Update CLI output to use summary:

if collector.Count() > 0 {
    summary := collector.SummarizeWithPriority()
    fmt.Fprintln(os.Stderr, console.FormatErrorMessage(summary))
    return errors.New("compilation failed")
}

Step 4: Add tests for prioritization and summarization.

Example output:

Found 12 error(s):

  validation: 8
  compilation: 3
  runtime: 1

🔴 Fix these first:

1. [validation] engine is required — add 'engine: copilot' to frontmatter
2. [validation] on.pull_request.branches and branches-ignore are mutually exclusive
3. [compilation] failed to resolve import 'shared/common.md' — file not found

<details>
<summary>View 9 more error(s)</summary>

4. [validation] cache key must not reference github.run_id
...
</details>

Run make agent-finish before creating the PR.

---

## 📊 Historical Context

<details>
<summary>Previous Focus Areas</summary>

| Date | Focus Area | Type | Custom | Key Outcomes |
|------|------------|------|--------|------------|
| 2026-05-18 | Mega-Function Decomposition in pkg/workflow | Custom | Y | Identified 15 functions >300 lines; generated 5 decomposition tasks; 3 high-priority items |

</details>

---

## 🎯 Recommendations

### Immediate Actions (This Week)

1. **Migrate high-impact validators to structured errors** (Task 1) — Priority: High
   - Focus on `compiler_filters_validation.go`, `strict_mode_network_validation.go`, `mcp_property_validation.go`
   - These validators are hit frequently during initial workflow setup
   - Low effort, high impact on user experience

2. **Add fuzzy matching for engine names and MCP servers** (Task 2) — Priority: High
   - Quick win that significantly improves error messages
   - Reuse existing pattern from `run_workflow_validation.go`
   - Users will see "Did you mean 'copilot'?" instead of "invalid engine: copiliot"

### Short-term Actions (This Month)

1. **Enhance compiler errors with file/line context** (Task 3) — Priority: Medium
   - Requires parser changes but greatly improves debugging UX
   - Users can jump directly to problematic lines
   - Similar to IDE compiler error navigation

2. **Create error message style guide** (Task 4) — Priority: Medium
   - Document best practices to prevent regression
   - Onboard new contributors with clear guidelines
   - Establish foundation for automated linting

### Long-term Actions (This Quarter)

1. **Implement custom linter for error messages** (Task 4 part 2) — Priority: Medium
   - Enforce style guide automatically in CI
   - Catch errors without suggestions during code review
   - Maintain consistency as codebase grows

2. **Add error recovery suggestions to compilation flow** (Task 5) — Priority: Low
   - Prioritize errors so users fix critical issues first
   - Reduce overwhelm from long error lists
   - Improve success rate on first compilation attempt

---

## 📈 Success Metrics

- **Structured error adoption**: 7.9% (44/560) → **Target: 80%** (450/560) — Track NewValidationError usage in workflow package
- **Errors with suggestions**: ~20% → **Target: 90%** — Track suggestion field presence in validation errors
- **Fuzzy match suggestions**: 0 errors → **Target: 5+ error types** — Engine, MCP server, event names, permissions
- **User-reported "unclear error" issues**: Establish baseline from GitHub issues → **Target: -50%** reduction over 3 months
- **Average time to resolve compilation errors**: Measure through user studies → **Target: -30%** reduction

---

## Next Steps

1. Review and prioritise the tasks above
2. Assign Task 1 and Task 2 to Copilot coding agent via planner agent (high priority, quick wins)
3. Schedule Task 3 for next sprint (medium effort, requires parser changes)
4. Create Task 4 style guide first (documentation), then linter (automation)
5. Track progress on migration from `fmt.Errorf` to `NewValidationError` in weekly reports
6. Re-evaluate this focus area in 30 days — measure structured error adoption rate

---

*Generated by Repository Quality Improvement Agent*
*Next analysis: 2026-05-20 — Focus area selected by diversity algorithm*




> Generated by [⚡ Repository Quality Improvement Agent](https://github.com/github/gh-aw/actions/runs/26101312306) · ● 12.4M · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Frepository-quality-improver%22&type=discussions)
> - [x] expires <!-- gh-aw-expires: 2026-05-20T13:58:51.787Z --> on May 20, 2026, 1:58 PM UTC

<!-- gh-aw-agentic-workflow: Repository Quality Improvement Agent, engine: copilot, version: 1.0.48, model: claude-sonnet-4.5, id: 26101312306, workflow_id: repository-quality-improver, run: https://github.com/github/gh-aw/actions/runs/26101312306 -->

<!-- gh-aw-workflow-id: repository-quality-improver -->
<!-- gh-aw-workflow-call-id: github/gh-aw/repository-quality-improver -->

[pelikhan]

/plan

[github-actions[bot]]

🚀 Plan Command has started processing this discussion comment

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-05-20T13:58:51.787Z.

Closed by Workflow