mirror of
https://github.com/glittercowboy/get-shit-done
synced 2026-04-25 17:25:23 +02:00
feat: harden Nyquist defaults, add retroactive validation, compress prompts (#855)
* fix: change nyquist_validation default to true and harden absent-key skip conditions new-project.md never wrote the key, so agents reading config directly treated absent as falsy. Changed all agent skip conditions from "is false" to "explicitly set to false; absent = enabled". Default changed from false to true in core.cjs, config.cjs, and templates/config.json. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: enforce VALIDATION.md creation with verification gate and Check 8e Step 5.5 was narrative markdown that Claude skipped under context pressure. Now MANDATORY with Write tool requirement and file-existence verification. Step 7.5 gates planner spawn on VALIDATION.md presence. Check 8e blocks Dimension 8 if file missing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add W008/W009 health checks and addNyquistKey repair for Nyquist drift detection W008 warns when workflow.nyquist_validation key is absent from config.json (agents may skip validation). W009 warns when RESEARCH.md has Validation Architecture section but no VALIDATION.md file exists. addNyquistKey repair adds the missing key with default true value. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add /gsd:validate-phase command and gsd-nyquist-auditor agent Retroactively applies Nyquist validation to already-executed phases. Works mid-milestone and post-milestone. Detects existing test coverage, maps gaps to phase requirements, writes missing tests, debugs failing ones, and produces {phase}-VALIDATION.md from existing artifacts. Handles three states: VALIDATION.md exists (audit + update), no VALIDATION.md (reconstruct from PLAN.md + SUMMARY.md), phase not yet executed (exit cleanly with guidance). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: audit-milestone reports Nyquist compliance gaps across phases Adds Nyquist coverage table to audit-milestone output when workflow.nyquist_validation is true. Identifies phases missing VALIDATION.md or with nyquist_compliant: false/partial. Routes to /gsd:validate-phase for resolution. Updates USER-GUIDE with retroactive validation documentation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: compress Nyquist prompts to match GSD meta-prompt density conventions Auditor agent: deleted philosophy section (35 lines), compressed execution flow 60%, removed redundant constraints. Workflow: cut purpose bloat, collapsed state narrative, compressed auditor spawn template. Command: removed redundant process section. Plan-phase Steps 5.5/7.5: replaced hedging language with directives. Audit-milestone Step 5.5: collapsed sub-steps into inline instructions. Net: -376 lines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Fana
176
agents/gsd-nyquist-auditor.md
Normal file
176
agents/gsd-nyquist-auditor.md
Normal file
@@ -0,0 +1,176 @@
|
||||
---
|
||||
name: gsd-nyquist-auditor
|
||||
description: Fills Nyquist validation gaps by generating tests and verifying coverage for phase requirements
|
||||
tools:
|
||||
- Read
|
||||
- Write
|
||||
- Edit
|
||||
- Bash
|
||||
- Glob
|
||||
- Grep
|
||||
color: "#8B5CF6"
|
||||
---
|
||||
|
||||
<role>
|
||||
GSD Nyquist auditor. Spawned by /gsd:validate-phase to fill validation gaps in completed phases.
|
||||
|
||||
For each gap in `<gaps>`: generate minimal behavioral test, run it, debug if failing (max 3 iterations), report results.
|
||||
|
||||
**Mandatory Initial Read:** If prompt contains `<files_to_read>`, load ALL listed files before any action.
|
||||
|
||||
**Implementation files are READ-ONLY.** Only create/modify: test files, fixtures, VALIDATION.md. Implementation bugs → ESCALATE. Never fix implementation.
|
||||
</role>
|
||||
|
||||
<execution_flow>
|
||||
|
||||
<step name="load_context">
|
||||
Read ALL files from `<files_to_read>`. Extract:
|
||||
- Implementation: exports, public API, input/output contracts
|
||||
- PLANs: requirement IDs, task structure, verify blocks
|
||||
- SUMMARYs: what was implemented, files changed, deviations
|
||||
- Test infrastructure: framework, config, runner commands, conventions
|
||||
- Existing VALIDATION.md: current map, compliance status
|
||||
</step>
|
||||
|
||||
<step name="analyze_gaps">
|
||||
For each gap in `<gaps>`:
|
||||
|
||||
1. Read related implementation files
|
||||
2. Identify observable behavior the requirement demands
|
||||
3. Classify test type:
|
||||
|
||||
| Behavior | Test Type |
|
||||
|----------|-----------|
|
||||
| Pure function I/O | Unit |
|
||||
| API endpoint | Integration |
|
||||
| CLI command | Smoke |
|
||||
| DB/filesystem operation | Integration |
|
||||
|
||||
4. Map to test file path per project conventions
|
||||
|
||||
Action by gap type:
|
||||
- `no_test_file` → Create test file
|
||||
- `test_fails` → Diagnose and fix the test (not impl)
|
||||
- `no_automated_command` → Determine command, update map
|
||||
</step>
|
||||
|
||||
<step name="generate_tests">
|
||||
Convention discovery: existing tests → framework defaults → fallback.
|
||||
|
||||
| Framework | File Pattern | Runner | Assert Style |
|
||||
|-----------|-------------|--------|--------------|
|
||||
| pytest | `test_{name}.py` | `pytest {file} -v` | `assert result == expected` |
|
||||
| jest | `{name}.test.ts` | `npx jest {file}` | `expect(result).toBe(expected)` |
|
||||
| vitest | `{name}.test.ts` | `npx vitest run {file}` | `expect(result).toBe(expected)` |
|
||||
| go test | `{name}_test.go` | `go test -v -run {Name}` | `if got != want { t.Errorf(...) }` |
|
||||
|
||||
Per gap: Write test file. One focused test per requirement behavior. Arrange/Act/Assert. Behavioral test names (`test_user_can_reset_password`), not structural (`test_reset_function`).
|
||||
</step>
|
||||
|
||||
<step name="run_and_verify">
|
||||
Execute each test. If passes: record success, next gap. If fails: enter debug loop.
|
||||
|
||||
Run every test. Never mark untested tests as passing.
|
||||
</step>
|
||||
|
||||
<step name="debug_loop">
|
||||
Max 3 iterations per failing test.
|
||||
|
||||
| Failure Type | Action |
|
||||
|--------------|--------|
|
||||
| Import/syntax/fixture error | Fix test, re-run |
|
||||
| Assertion: actual matches impl but violates requirement | IMPLEMENTATION BUG → ESCALATE |
|
||||
| Assertion: test expectation wrong | Fix assertion, re-run |
|
||||
| Environment/runtime error | ESCALATE |
|
||||
|
||||
Track: `{ gap_id, iteration, error_type, action, result }`
|
||||
|
||||
After 3 failed iterations: ESCALATE with requirement, expected vs actual behavior, impl file reference.
|
||||
</step>
|
||||
|
||||
<step name="report">
|
||||
Resolved gaps: `{ task_id, requirement, test_type, automated_command, file_path, status: "green" }`
|
||||
Escalated gaps: `{ task_id, requirement, reason, debug_iterations, last_error }`
|
||||
|
||||
Return one of three formats below.
|
||||
</step>
|
||||
|
||||
</execution_flow>
|
||||
|
||||
<structured_returns>
|
||||
|
||||
## GAPS FILLED
|
||||
|
||||
```markdown
|
||||
## GAPS FILLED
|
||||
|
||||
**Phase:** {N} — {name}
|
||||
**Resolved:** {count}/{count}
|
||||
|
||||
### Tests Created
|
||||
| # | File | Type | Command |
|
||||
|---|------|------|---------|
|
||||
| 1 | {path} | {unit/integration/smoke} | `{cmd}` |
|
||||
|
||||
### Verification Map Updates
|
||||
| Task ID | Requirement | Command | Status |
|
||||
|---------|-------------|---------|--------|
|
||||
| {id} | {req} | `{cmd}` | green |
|
||||
|
||||
### Files for Commit
|
||||
{test file paths}
|
||||
```
|
||||
|
||||
## PARTIAL
|
||||
|
||||
```markdown
|
||||
## PARTIAL
|
||||
|
||||
**Phase:** {N} — {name}
|
||||
**Resolved:** {M}/{total} | **Escalated:** {K}/{total}
|
||||
|
||||
### Resolved
|
||||
| Task ID | Requirement | File | Command | Status |
|
||||
|---------|-------------|------|---------|--------|
|
||||
| {id} | {req} | {file} | `{cmd}` | green |
|
||||
|
||||
### Escalated
|
||||
| Task ID | Requirement | Reason | Iterations |
|
||||
|---------|-------------|--------|------------|
|
||||
| {id} | {req} | {reason} | {N}/3 |
|
||||
|
||||
### Files for Commit
|
||||
{test file paths for resolved gaps}
|
||||
```
|
||||
|
||||
## ESCALATE
|
||||
|
||||
```markdown
|
||||
## ESCALATE
|
||||
|
||||
**Phase:** {N} — {name}
|
||||
**Resolved:** 0/{total}
|
||||
|
||||
### Details
|
||||
| Task ID | Requirement | Reason | Iterations |
|
||||
|---------|-------------|--------|------------|
|
||||
| {id} | {req} | {reason} | {N}/3 |
|
||||
|
||||
### Recommendations
|
||||
- **{req}:** {manual test instructions or implementation fix needed}
|
||||
```
|
||||
|
||||
</structured_returns>
|
||||
|
||||
<success_criteria>
|
||||
- [ ] All `<files_to_read>` loaded before any action
|
||||
- [ ] Each gap analyzed with correct test type
|
||||
- [ ] Tests follow project conventions
|
||||
- [ ] Tests verify behavior, not structure
|
||||
- [ ] Every test executed — none marked passing without running
|
||||
- [ ] Implementation files never modified
|
||||
- [ ] Max 3 debug iterations per gap
|
||||
- [ ] Implementation bugs escalated, not fixed
|
||||
- [ ] Structured return provided (GAPS FILLED / PARTIAL / ESCALATE)
|
||||
- [ ] Test files listed for commit
|
||||
</success_criteria>
|
||||
@@ -306,7 +306,7 @@ Verified patterns from official sources:
|
||||
|
||||
## Validation Architecture
|
||||
|
||||
> Skip this section entirely if workflow.nyquist_validation is false in .planning/config.json
|
||||
> Skip this section entirely if workflow.nyquist_validation is explicitly set to false in .planning/config.json. If the key is absent, treat as enabled.
|
||||
|
||||
### Test Framework
|
||||
| Property | Value |
|
||||
@@ -372,7 +372,7 @@ INIT=$(node "$HOME/.claude/get-shit-done/bin/gsd-tools.cjs" init phase-op "${PHA
|
||||
|
||||
Extract from init JSON: `phase_dir`, `padded_phase`, `phase_number`, `commit_docs`.
|
||||
|
||||
Also read `.planning/config.json` — if `workflow.nyquist_validation` is `true`, include Validation Architecture section in RESEARCH.md. If `false`, skip it.
|
||||
Also read `.planning/config.json` — include Validation Architecture section in RESEARCH.md unless `workflow.nyquist_validation` is explicitly `false`. If the key is absent or `true`, include the section.
|
||||
|
||||
Then read CONTEXT.md if exists:
|
||||
```bash
|
||||
@@ -408,7 +408,7 @@ For each domain: Context7 first → Official docs → WebSearch → Cross-verify
|
||||
|
||||
## Step 4: Validation Architecture Research (if nyquist_validation enabled)
|
||||
|
||||
**Skip if** workflow.nyquist_validation is false.
|
||||
**Skip if** workflow.nyquist_validation is explicitly set to false. If absent, treat as enabled.
|
||||
|
||||
### Detect Test Infrastructure
|
||||
Scan for: test config files (pytest.ini, jest.config.*, vitest.config.*), test directories (test/, tests/, __tests__/), test files (*.test.*, *.spec.*), package.json test scripts.
|
||||
|
||||
@@ -316,7 +316,20 @@ issue:
|
||||
|
||||
## Dimension 8: Nyquist Compliance
|
||||
|
||||
Skip if: `workflow.nyquist_validation` is false, phase has no RESEARCH.md, or RESEARCH.md has no "Validation Architecture" section. Output: "Dimension 8: SKIPPED (nyquist_validation disabled or not applicable)"
|
||||
Skip if: `workflow.nyquist_validation` is explicitly set to `false` in config.json (absent key = enabled), phase has no RESEARCH.md, or RESEARCH.md has no "Validation Architecture" section. Output: "Dimension 8: SKIPPED (nyquist_validation disabled or not applicable)"
|
||||
|
||||
### Check 8e — VALIDATION.md Existence (Gate)
|
||||
|
||||
Before running checks 8a-8d, verify VALIDATION.md exists:
|
||||
|
||||
```bash
|
||||
ls "${PHASE_DIR}"/*-VALIDATION.md 2>/dev/null
|
||||
```
|
||||
|
||||
**If missing:** **BLOCKING FAIL** — "VALIDATION.md not found for phase {N}. Re-run `/gsd:plan-phase {N} --research` to regenerate."
|
||||
Skip checks 8a-8d entirely. Report Dimension 8 as FAIL with this single issue.
|
||||
|
||||
**If exists:** Proceed to checks 8a-8d.
|
||||
|
||||
### Check 8a — Automated Verify Presence
|
||||
|
||||
|
||||
35
commands/gsd/validate-phase.md
Normal file
35
commands/gsd/validate-phase.md
Normal file
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: gsd:validate-phase
|
||||
description: Retroactively audit and fill Nyquist validation gaps for a completed phase
|
||||
argument-hint: "[phase number]"
|
||||
allowed-tools:
|
||||
- Read
|
||||
- Write
|
||||
- Edit
|
||||
- Bash
|
||||
- Glob
|
||||
- Grep
|
||||
- Task
|
||||
- AskUserQuestion
|
||||
---
|
||||
<objective>
|
||||
Audit Nyquist validation coverage for a completed phase. Three states:
|
||||
- (A) VALIDATION.md exists — audit and fill gaps
|
||||
- (B) No VALIDATION.md, SUMMARY.md exists — reconstruct from artifacts
|
||||
- (C) Phase not executed — exit with guidance
|
||||
|
||||
Output: updated VALIDATION.md + generated test files.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@~/.claude/get-shit-done/workflows/validate-phase.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
Phase: $ARGUMENTS — optional, defaults to last completed phase.
|
||||
</context>
|
||||
|
||||
<process>
|
||||
Execute @~/.claude/get-shit-done/workflows/validate-phase.md.
|
||||
Preserve all workflow gates.
|
||||
</process>
|
||||
@@ -115,6 +115,37 @@ lack automated verify commands will not be approved.
|
||||
**Disable:** Set `workflow.nyquist_validation: false` in `/gsd:settings` for
|
||||
rapid prototyping phases where test infrastructure isn't the focus.
|
||||
|
||||
### Retroactive Validation (`/gsd:validate-phase`)
|
||||
|
||||
For phases executed before Nyquist validation existed, or for existing codebases
|
||||
with only traditional test suites, retroactively audit and fill coverage gaps:
|
||||
|
||||
```
|
||||
/gsd:validate-phase N
|
||||
|
|
||||
+-- Detect state (VALIDATION.md exists? SUMMARY.md exists?)
|
||||
|
|
||||
+-- Discover: scan implementation, map requirements to tests
|
||||
|
|
||||
+-- Analyze gaps: which requirements lack automated verification?
|
||||
|
|
||||
+-- Present gap plan for approval
|
||||
|
|
||||
+-- Spawn auditor: generate tests, run, debug (max 3 attempts)
|
||||
|
|
||||
+-- Update VALIDATION.md
|
||||
|
|
||||
+-- COMPLIANT -> all requirements have automated checks
|
||||
+-- PARTIAL -> some gaps escalated to manual-only
|
||||
```
|
||||
|
||||
The auditor never modifies implementation code — only test files and
|
||||
VALIDATION.md. If a test reveals an implementation bug, it's flagged as an
|
||||
escalation for you to address.
|
||||
|
||||
**When to use:** After executing phases that were planned before Nyquist was
|
||||
enabled, or after `/gsd:audit-milestone` surfaces Nyquist compliance gaps.
|
||||
|
||||
### Execution Wave Coordination
|
||||
|
||||
```
|
||||
|
||||
@@ -61,7 +61,7 @@ function cmdConfigEnsureSection(cwd, raw) {
|
||||
research: true,
|
||||
plan_check: true,
|
||||
verifier: true,
|
||||
nyquist_validation: false,
|
||||
nyquist_validation: true,
|
||||
},
|
||||
parallelization: true,
|
||||
brave_search: hasBraveSearch,
|
||||
|
||||
@@ -27,6 +27,7 @@ const MODEL_PROFILES = {
|
||||
'gsd-verifier': { quality: 'sonnet', balanced: 'sonnet', budget: 'haiku' },
|
||||
'gsd-plan-checker': { quality: 'sonnet', balanced: 'sonnet', budget: 'haiku' },
|
||||
'gsd-integration-checker': { quality: 'sonnet', balanced: 'sonnet', budget: 'haiku' },
|
||||
'gsd-nyquist-auditor': { quality: 'sonnet', balanced: 'sonnet', budget: 'haiku' },
|
||||
};
|
||||
|
||||
// ─── Output helpers ───────────────────────────────────────────────────────────
|
||||
@@ -76,7 +77,7 @@ function loadConfig(cwd) {
|
||||
research: true,
|
||||
plan_checker: true,
|
||||
verifier: true,
|
||||
nyquist_validation: false,
|
||||
nyquist_validation: true,
|
||||
parallelization: true,
|
||||
brave_search: false,
|
||||
};
|
||||
|
||||
@@ -617,6 +617,18 @@ function cmdValidateHealth(cwd, options, raw) {
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Check 5b: Nyquist validation key presence ──────────────────────────
|
||||
if (fs.existsSync(configPath)) {
|
||||
try {
|
||||
const configRaw = fs.readFileSync(configPath, 'utf-8');
|
||||
const configParsed = JSON.parse(configRaw);
|
||||
if (configParsed.workflow && configParsed.workflow.nyquist_validation === undefined) {
|
||||
addIssue('warning', 'W008', 'config.json: workflow.nyquist_validation absent (defaults to enabled but agents may skip)', 'Run /gsd:health --repair to add key', true);
|
||||
if (!repairs.includes('addNyquistKey')) repairs.push('addNyquistKey');
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
|
||||
// ─── Check 6: Phase directory naming (NN-name format) ─────────────────────
|
||||
try {
|
||||
const entries = fs.readdirSync(phasesDir, { withFileTypes: true });
|
||||
@@ -646,6 +658,24 @@ function cmdValidateHealth(cwd, options, raw) {
|
||||
}
|
||||
} catch {}
|
||||
|
||||
// ─── Check 7b: Nyquist VALIDATION.md consistency ────────────────────────
|
||||
try {
|
||||
const phaseEntries = fs.readdirSync(phasesDir, { withFileTypes: true });
|
||||
for (const e of phaseEntries) {
|
||||
if (!e.isDirectory()) continue;
|
||||
const phaseFiles = fs.readdirSync(path.join(phasesDir, e.name));
|
||||
const hasResearch = phaseFiles.some(f => f.endsWith('-RESEARCH.md'));
|
||||
const hasValidation = phaseFiles.some(f => f.endsWith('-VALIDATION.md'));
|
||||
if (hasResearch && !hasValidation) {
|
||||
const researchFile = phaseFiles.find(f => f.endsWith('-RESEARCH.md'));
|
||||
const researchContent = fs.readFileSync(path.join(phasesDir, e.name, researchFile), 'utf-8');
|
||||
if (researchContent.includes('## Validation Architecture')) {
|
||||
addIssue('warning', 'W009', `Phase ${e.name}: has Validation Architecture in RESEARCH.md but no VALIDATION.md`, 'Re-run /gsd:plan-phase with --research to regenerate');
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {}
|
||||
|
||||
// ─── Check 8: Run existing consistency checks ─────────────────────────────
|
||||
// Inline subset of cmdValidateConsistency
|
||||
if (fs.existsSync(roadmapPath)) {
|
||||
@@ -730,6 +760,23 @@ function cmdValidateHealth(cwd, options, raw) {
|
||||
repairActions.push({ action: repair, success: true, path: 'STATE.md' });
|
||||
break;
|
||||
}
|
||||
case 'addNyquistKey': {
|
||||
if (fs.existsSync(configPath)) {
|
||||
try {
|
||||
const configRaw = fs.readFileSync(configPath, 'utf-8');
|
||||
const configParsed = JSON.parse(configRaw);
|
||||
if (!configParsed.workflow) configParsed.workflow = {};
|
||||
if (configParsed.workflow.nyquist_validation === undefined) {
|
||||
configParsed.workflow.nyquist_validation = true;
|
||||
fs.writeFileSync(configPath, JSON.stringify(configParsed, null, 2), 'utf-8');
|
||||
}
|
||||
repairActions.push({ action: repair, success: true, path: 'config.json' });
|
||||
} catch (err) {
|
||||
repairActions.push({ action: repair, success: false, error: err.message });
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
repairActions.push({ action: repair, success: false, error: err.message });
|
||||
|
||||
@@ -17,6 +17,7 @@ Model profiles control which Claude model each GSD agent uses. This allows balan
|
||||
| gsd-verifier | sonnet | sonnet | haiku |
|
||||
| gsd-plan-checker | sonnet | sonnet | haiku |
|
||||
| gsd-integration-checker | sonnet | sonnet | haiku |
|
||||
| gsd-nyquist-auditor | sonnet | sonnet | haiku |
|
||||
|
||||
## Profile Philosophy
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
"plan_check": true,
|
||||
"verifier": true,
|
||||
"auto_advance": false,
|
||||
"nyquist_validation": false
|
||||
"nyquist_validation": true
|
||||
},
|
||||
"planning": {
|
||||
"commit_docs": true,
|
||||
|
||||
@@ -127,6 +127,30 @@ For each REQ-ID, determine status using all three sources:
|
||||
|
||||
**Orphan detection:** Requirements present in REQUIREMENTS.md traceability table but absent from ALL phase VERIFICATION.md files MUST be flagged as orphaned. Orphaned requirements are treated as `unsatisfied` — they were assigned but never verified by any phase.
|
||||
|
||||
## 5.5. Nyquist Compliance Discovery
|
||||
|
||||
Skip if `workflow.nyquist_validation` is explicitly `false` (absent = enabled).
|
||||
|
||||
```bash
|
||||
NYQUIST_CONFIG=$(node "$HOME/.claude/get-shit-done/bin/gsd-tools.cjs" config get workflow.nyquist_validation --raw 2>/dev/null)
|
||||
```
|
||||
|
||||
If `false`: skip entirely.
|
||||
|
||||
For each phase directory, check `*-VALIDATION.md`. If exists, parse frontmatter (`nyquist_compliant`, `wave_0_complete`).
|
||||
|
||||
Classify per phase:
|
||||
|
||||
| Status | Condition |
|
||||
|--------|-----------|
|
||||
| COMPLIANT | `nyquist_compliant: true` and all tasks green |
|
||||
| PARTIAL | VALIDATION.md exists, `nyquist_compliant: false` or red/pending |
|
||||
| MISSING | No VALIDATION.md |
|
||||
|
||||
Add to audit YAML: `nyquist: { compliant_phases, partial_phases, missing_phases, overall }`
|
||||
|
||||
Discovery only — never auto-calls `/gsd:validate-phase`.
|
||||
|
||||
## 6. Aggregate into v{version}-MILESTONE-AUDIT.md
|
||||
|
||||
Create `.planning/v{version}-v{version}-MILESTONE-AUDIT.md` with:
|
||||
@@ -227,6 +251,14 @@ All requirements covered. Cross-phase integration verified. E2E flows complete.
|
||||
{For each flow gap:}
|
||||
- **{flow name}:** breaks at {step}
|
||||
|
||||
### Nyquist Coverage
|
||||
|
||||
| Phase | VALIDATION.md | Compliant | Action |
|
||||
|-------|---------------|-----------|--------|
|
||||
| {phase} | exists/missing | true/false/partial | `/gsd:validate-phase {N}` |
|
||||
|
||||
Phases needing validation: run `/gsd:validate-phase {N}` for each flagged phase.
|
||||
|
||||
───────────────────────────────────────────────────────────────
|
||||
|
||||
## ▶ Next Up
|
||||
@@ -293,5 +325,7 @@ All requirements met. No critical blockers. Accumulated tech debt needs review.
|
||||
- [ ] Integration checker spawned with milestone requirement IDs
|
||||
- [ ] v{version}-MILESTONE-AUDIT.md created with structured requirement gap objects
|
||||
- [ ] FAIL gate enforced — any unsatisfied requirement forces gaps_found status
|
||||
- [ ] Nyquist compliance scanned for all milestone phases (if enabled)
|
||||
- [ ] Missing VALIDATION.md phases flagged with validate-phase suggestion
|
||||
- [ ] Results presented with actionable next steps
|
||||
</success_criteria>
|
||||
|
||||
@@ -136,6 +136,8 @@ Report final status.
|
||||
| W005 | warning | Phase directory naming mismatch | No |
|
||||
| W006 | warning | Phase in ROADMAP but no directory | No |
|
||||
| W007 | warning | Phase on disk but not in ROADMAP | No |
|
||||
| W008 | warning | config.json: workflow.nyquist_validation absent (defaults to enabled but agents may skip) | Yes |
|
||||
| W009 | warning | Phase has Validation Architecture in RESEARCH.md but no VALIDATION.md | No |
|
||||
| I001 | info | Plan without SUMMARY (may be in progress) | No |
|
||||
|
||||
</error_codes>
|
||||
@@ -147,6 +149,7 @@ Report final status.
|
||||
| createConfig | Create config.json with defaults | None |
|
||||
| resetConfig | Delete + recreate config.json | Loses custom settings |
|
||||
| regenerateState | Create STATE.md from ROADMAP structure | Loses session history |
|
||||
| addNyquistKey | Add workflow.nyquist_validation: true to config.json | None — matches existing default |
|
||||
|
||||
**Not repairable (too risky):**
|
||||
- PROJECT.md, ROADMAP.md content
|
||||
|
||||
@@ -177,6 +177,7 @@ Create `.planning/config.json` with mode set to "yolo":
|
||||
"research": true|false,
|
||||
"plan_check": true|false,
|
||||
"verifier": true|false,
|
||||
"nyquist_validation": depth !== "quick",
|
||||
"auto_advance": true
|
||||
}
|
||||
}
|
||||
@@ -475,7 +476,8 @@ Create `.planning/config.json` with all settings:
|
||||
"workflow": {
|
||||
"research": true|false,
|
||||
"plan_check": true|false,
|
||||
"verifier": true|false
|
||||
"verifier": true|false,
|
||||
"nyquist_validation": depth !== "quick"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -219,30 +219,26 @@ Task(
|
||||
- **`## RESEARCH COMPLETE`:** Display confirmation, continue to step 6
|
||||
- **`## RESEARCH BLOCKED`:** Display blocker, offer: 1) Provide context, 2) Skip research, 3) Abort
|
||||
|
||||
## 5.5. Create Validation Strategy (if Nyquist enabled)
|
||||
## 5.5. Create Validation Strategy
|
||||
|
||||
**Skip if:** `nyquist_validation_enabled` is false from INIT JSON.
|
||||
|
||||
After researcher completes, check if RESEARCH.md contains a Validation Architecture section:
|
||||
MANDATORY unless `nyquist_validation_enabled` is false.
|
||||
|
||||
```bash
|
||||
grep -l "## Validation Architecture" "${PHASE_DIR}"/*-RESEARCH.md 2>/dev/null
|
||||
```
|
||||
|
||||
**If found:**
|
||||
1. Read validation template from `~/.claude/get-shit-done/templates/VALIDATION.md`
|
||||
2. Write to `${PHASE_DIR}/${PADDED_PHASE}-VALIDATION.md`
|
||||
3. Fill frontmatter: replace `{N}` with phase number, `{phase-slug}` with phase slug, `{date}` with current date
|
||||
4. If `commit_docs` is true:
|
||||
1. Read template: `~/.claude/get-shit-done/templates/VALIDATION.md`
|
||||
2. Write to `${PHASE_DIR}/${PADDED_PHASE}-VALIDATION.md` (use Write tool)
|
||||
3. Fill frontmatter: `{N}` → phase number, `{phase-slug}` → slug, `{date}` → current date
|
||||
4. Verify:
|
||||
```bash
|
||||
node "$HOME/.claude/get-shit-done/bin/gsd-tools.cjs" commit-docs "docs(phase-${PHASE}): add validation strategy"
|
||||
test -f "${PHASE_DIR}/${PADDED_PHASE}-VALIDATION.md" && echo "VALIDATION_CREATED=true" || echo "VALIDATION_CREATED=false"
|
||||
```
|
||||
5. If `VALIDATION_CREATED=false`: STOP — do not proceed to Step 6
|
||||
6. If `commit_docs`: `commit-docs "docs(phase-${PHASE}): add validation strategy"`
|
||||
|
||||
**If not found (and nyquist enabled):** Display warning:
|
||||
```
|
||||
⚠ Nyquist validation enabled but researcher did not produce a Validation Architecture section.
|
||||
Continuing without validation strategy. Plans may fail Dimension 8 check.
|
||||
```
|
||||
**If not found:** Warn and continue — plans may fail Dimension 8.
|
||||
|
||||
## 6. Check Existing Plans
|
||||
|
||||
@@ -266,6 +262,21 @@ UAT_PATH=$(printf '%s\n' "$INIT" | jq -r '.uat_path // empty')
|
||||
CONTEXT_PATH=$(printf '%s\n' "$INIT" | jq -r '.context_path // empty')
|
||||
```
|
||||
|
||||
## 7.5. Verify Nyquist Artifacts
|
||||
|
||||
Skip if `nyquist_validation_enabled` is false.
|
||||
|
||||
```bash
|
||||
VALIDATION_EXISTS=$(ls "${PHASE_DIR}"/*-VALIDATION.md 2>/dev/null | head -1)
|
||||
```
|
||||
|
||||
If missing and Nyquist enabled — ask user:
|
||||
1. Re-run: `/gsd:plan-phase {PHASE} --research`
|
||||
2. Disable Nyquist in config
|
||||
3. Continue anyway (plans fail Dimension 8)
|
||||
|
||||
Proceed to Step 8 only if user selects 2 or 3.
|
||||
|
||||
## 8. Spawn gsd-planner Agent
|
||||
|
||||
Display banner:
|
||||
|
||||
@@ -28,7 +28,7 @@ Parse current values (default to `true` if not present):
|
||||
- `workflow.research` — spawn researcher during plan-phase
|
||||
- `workflow.plan_check` — spawn plan checker during plan-phase
|
||||
- `workflow.verifier` — spawn verifier during execute-phase
|
||||
- `workflow.nyquist_validation` — validation architecture research during plan-phase
|
||||
- `workflow.nyquist_validation` — validation architecture research during plan-phase (default: true if absent)
|
||||
- `model_profile` — which model each agent uses (default: `balanced`)
|
||||
- `git.branching_strategy` — branching approach (default: `"none"`)
|
||||
</step>
|
||||
|
||||
166
get-shit-done/workflows/validate-phase.md
Normal file
166
get-shit-done/workflows/validate-phase.md
Normal file
@@ -0,0 +1,166 @@
|
||||
<purpose>
|
||||
Audit Nyquist validation gaps for a completed phase. Generate missing tests. Update VALIDATION.md.
|
||||
</purpose>
|
||||
|
||||
<required_reading>
|
||||
@~/.claude/get-shit-done/references/ui-brand.md
|
||||
</required_reading>
|
||||
|
||||
<process>
|
||||
|
||||
## 0. Initialize
|
||||
|
||||
```bash
|
||||
INIT=$(node "$HOME/.claude/get-shit-done/bin/gsd-tools.cjs" init phase-op "${PHASE_ARG}")
|
||||
```
|
||||
|
||||
Parse: `phase_dir`, `phase_number`, `phase_name`, `phase_slug`, `padded_phase`.
|
||||
|
||||
```bash
|
||||
AUDITOR_MODEL=$(node "$HOME/.claude/get-shit-done/bin/gsd-tools.cjs" resolve-model gsd-nyquist-auditor --raw)
|
||||
NYQUIST_CFG=$(node "$HOME/.claude/get-shit-done/bin/gsd-tools.cjs" config get workflow.nyquist_validation --raw)
|
||||
```
|
||||
|
||||
If `NYQUIST_CFG` is `false`: exit with "Nyquist validation is disabled. Enable via /gsd:settings."
|
||||
|
||||
Display banner: `GSD > VALIDATE PHASE {N}: {name}`
|
||||
|
||||
## 1. Detect Input State
|
||||
|
||||
```bash
|
||||
VALIDATION_FILE=$(ls "${PHASE_DIR}"/*-VALIDATION.md 2>/dev/null | head -1)
|
||||
SUMMARY_FILES=$(ls "${PHASE_DIR}"/*-SUMMARY.md 2>/dev/null)
|
||||
```
|
||||
|
||||
- **State A** (`VALIDATION_FILE` non-empty): Audit existing
|
||||
- **State B** (`VALIDATION_FILE` empty, `SUMMARY_FILES` non-empty): Reconstruct from artifacts
|
||||
- **State C** (`SUMMARY_FILES` empty): Exit — "Phase {N} not executed. Run /gsd:execute-phase {N} first."
|
||||
|
||||
## 2. Discovery
|
||||
|
||||
### 2a. Read Phase Artifacts
|
||||
|
||||
Read all PLAN and SUMMARY files. Extract: task lists, requirement IDs, key-files changed, verify blocks.
|
||||
|
||||
### 2b. Build Requirement-to-Task Map
|
||||
|
||||
Per task: `{ task_id, plan_id, wave, requirement_ids, has_automated_command }`
|
||||
|
||||
### 2c. Detect Test Infrastructure
|
||||
|
||||
State A: Parse from existing VALIDATION.md Test Infrastructure table.
|
||||
State B: Filesystem scan:
|
||||
|
||||
```bash
|
||||
find . -name "pytest.ini" -o -name "jest.config.*" -o -name "vitest.config.*" -o -name "pyproject.toml" 2>/dev/null | head -10
|
||||
find . \( -name "*.test.*" -o -name "*.spec.*" -o -name "test_*" \) -not -path "*/node_modules/*" 2>/dev/null | head -40
|
||||
```
|
||||
|
||||
### 2d. Cross-Reference
|
||||
|
||||
Match each requirement to existing tests by filename, imports, test descriptions. Record: requirement → test_file → status.
|
||||
|
||||
## 3. Gap Analysis
|
||||
|
||||
Classify each requirement:
|
||||
|
||||
| Status | Criteria |
|
||||
|--------|----------|
|
||||
| COVERED | Test exists, targets behavior, runs green |
|
||||
| PARTIAL | Test exists, failing or incomplete |
|
||||
| MISSING | No test found |
|
||||
|
||||
Build: `{ task_id, requirement, gap_type, suggested_test_path, suggested_command }`
|
||||
|
||||
No gaps → skip to Step 6, set `nyquist_compliant: true`.
|
||||
|
||||
## 4. Present Gap Plan
|
||||
|
||||
Call AskUserQuestion with gap table and options:
|
||||
1. "Fix all gaps" → Step 5
|
||||
2. "Skip — mark manual-only" → add to Manual-Only, Step 6
|
||||
3. "Cancel" → exit
|
||||
|
||||
## 5. Spawn gsd-nyquist-auditor
|
||||
|
||||
```
|
||||
Task(
|
||||
prompt="Read ~/.claude/agents/gsd-nyquist-auditor.md for instructions.\n\n" +
|
||||
"<files_to_read>{PLAN, SUMMARY, impl files, VALIDATION.md}</files_to_read>" +
|
||||
"<gaps>{gap list}</gaps>" +
|
||||
"<test_infrastructure>{framework, config, commands}</test_infrastructure>" +
|
||||
"<constraints>Never modify impl files. Max 3 debug iterations. Escalate impl bugs.</constraints>",
|
||||
subagent_type="gsd-nyquist-auditor",
|
||||
model="{AUDITOR_MODEL}",
|
||||
description="Fill validation gaps for Phase {N}"
|
||||
)
|
||||
```
|
||||
|
||||
Handle return:
|
||||
- `## GAPS FILLED` → record tests + map updates, Step 6
|
||||
- `## PARTIAL` → record resolved, move escalated to manual-only, Step 6
|
||||
- `## ESCALATE` → move all to manual-only, Step 6
|
||||
|
||||
## 6. Generate/Update VALIDATION.md
|
||||
|
||||
**State B (create):**
|
||||
1. Read template from `~/.claude/get-shit-done/templates/VALIDATION.md`
|
||||
2. Fill: frontmatter, Test Infrastructure, Per-Task Map, Manual-Only, Sign-Off
|
||||
3. Write to `${PHASE_DIR}/${PADDED_PHASE}-VALIDATION.md`
|
||||
|
||||
**State A (update):**
|
||||
1. Update Per-Task Map statuses, add escalated to Manual-Only, update frontmatter
|
||||
2. Append audit trail:
|
||||
|
||||
```markdown
|
||||
## Validation Audit {date}
|
||||
| Metric | Count |
|
||||
|--------|-------|
|
||||
| Gaps found | {N} |
|
||||
| Resolved | {M} |
|
||||
| Escalated | {K} |
|
||||
```
|
||||
|
||||
## 7. Commit
|
||||
|
||||
```bash
|
||||
git add {test_files}
|
||||
git commit -m "test(phase-${PHASE}): add Nyquist validation tests"
|
||||
|
||||
node "$HOME/.claude/get-shit-done/bin/gsd-tools.cjs" commit-docs "docs(phase-${PHASE}): add/update validation strategy"
|
||||
```
|
||||
|
||||
## 8. Results + Routing
|
||||
|
||||
**Compliant:**
|
||||
```
|
||||
GSD > PHASE {N} IS NYQUIST-COMPLIANT
|
||||
All requirements have automated verification.
|
||||
▶ Next: /gsd:audit-milestone
|
||||
```
|
||||
|
||||
**Partial:**
|
||||
```
|
||||
GSD > PHASE {N} VALIDATED (PARTIAL)
|
||||
{M} automated, {K} manual-only.
|
||||
▶ Retry: /gsd:validate-phase {N}
|
||||
```
|
||||
|
||||
Display `/clear` reminder.
|
||||
|
||||
</process>
|
||||
|
||||
<success_criteria>
|
||||
- [ ] Nyquist config checked (exit if disabled)
|
||||
- [ ] Input state detected (A/B/C)
|
||||
- [ ] State C exits cleanly
|
||||
- [ ] PLAN/SUMMARY files read, requirement map built
|
||||
- [ ] Test infrastructure detected
|
||||
- [ ] Gaps classified (COVERED/PARTIAL/MISSING)
|
||||
- [ ] User gate with gap table
|
||||
- [ ] Auditor spawned with complete context
|
||||
- [ ] All three return formats handled
|
||||
- [ ] VALIDATION.md created or updated
|
||||
- [ ] Test files committed separately
|
||||
- [ ] Results with routing presented
|
||||
</success_criteria>
|
||||
@@ -60,6 +60,7 @@ describe('loadConfig', () => {
|
||||
assert.strictEqual(config.plan_checker, true);
|
||||
assert.strictEqual(config.brave_search, false);
|
||||
assert.strictEqual(config.parallelization, true);
|
||||
assert.strictEqual(config.nyquist_validation, true);
|
||||
});
|
||||
|
||||
test('reads model_profile from config.json', () => {
|
||||
|
||||
@@ -347,6 +347,103 @@ describe('validate health command', () => {
|
||||
);
|
||||
});
|
||||
|
||||
// ─── Check 5b: Nyquist validation key presence (W008) ─────────────────────
|
||||
|
||||
test('detects W008 when workflow.nyquist_validation absent from config', () => {
|
||||
writeMinimalProjectMd(tmpDir);
|
||||
writeMinimalRoadmap(tmpDir, ['1']);
|
||||
writeMinimalStateMd(tmpDir, '# Session State\n\nPhase 1 in progress.\n');
|
||||
// Config with workflow section but WITHOUT nyquist_validation key
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, '.planning', 'config.json'),
|
||||
JSON.stringify({ model_profile: 'balanced', workflow: { research: true } }, null, 2)
|
||||
);
|
||||
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
|
||||
|
||||
const result = runGsdTools('validate health', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
assert.ok(
|
||||
output.warnings.some(w => w.code === 'W008'),
|
||||
`Expected W008 in warnings: ${JSON.stringify(output.warnings)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('does not emit W008 when nyquist_validation is explicitly set', () => {
|
||||
writeMinimalProjectMd(tmpDir);
|
||||
writeMinimalRoadmap(tmpDir, ['1']);
|
||||
writeMinimalStateMd(tmpDir, '# Session State\n\nPhase 1 in progress.\n');
|
||||
// Config with workflow.nyquist_validation explicitly set
|
||||
fs.writeFileSync(
|
||||
path.join(tmpDir, '.planning', 'config.json'),
|
||||
JSON.stringify({ model_profile: 'balanced', workflow: { research: true, nyquist_validation: true } }, null, 2)
|
||||
);
|
||||
fs.mkdirSync(path.join(tmpDir, '.planning', 'phases', '01-a'), { recursive: true });
|
||||
|
||||
const result = runGsdTools('validate health', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
assert.ok(
|
||||
!output.warnings.some(w => w.code === 'W008'),
|
||||
`Should not have W008: ${JSON.stringify(output.warnings)}`
|
||||
);
|
||||
});
|
||||
|
||||
// ─── Check 7b: Nyquist VALIDATION.md consistency (W009) ──────────────────
|
||||
|
||||
test('detects W009 when RESEARCH.md has Validation Architecture but no VALIDATION.md', () => {
|
||||
writeMinimalProjectMd(tmpDir);
|
||||
writeMinimalRoadmap(tmpDir, ['1']);
|
||||
writeMinimalStateMd(tmpDir, '# Session State\n\nPhase 1 in progress.\n');
|
||||
writeValidConfigJson(tmpDir);
|
||||
// Create phase dir with RESEARCH.md containing Validation Architecture
|
||||
const phaseDir = path.join(tmpDir, '.planning', 'phases', '01-setup');
|
||||
fs.mkdirSync(phaseDir, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(phaseDir, '01-RESEARCH.md'),
|
||||
'# Research\n\n## Validation Architecture\n\nSome validation content.\n'
|
||||
);
|
||||
// No VALIDATION.md
|
||||
|
||||
const result = runGsdTools('validate health', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
assert.ok(
|
||||
output.warnings.some(w => w.code === 'W009'),
|
||||
`Expected W009 in warnings: ${JSON.stringify(output.warnings)}`
|
||||
);
|
||||
});
|
||||
|
||||
test('does not emit W009 when VALIDATION.md exists alongside RESEARCH.md', () => {
|
||||
writeMinimalProjectMd(tmpDir);
|
||||
writeMinimalRoadmap(tmpDir, ['1']);
|
||||
writeMinimalStateMd(tmpDir, '# Session State\n\nPhase 1 in progress.\n');
|
||||
writeValidConfigJson(tmpDir);
|
||||
// Create phase dir with both RESEARCH.md and VALIDATION.md
|
||||
const phaseDir = path.join(tmpDir, '.planning', 'phases', '01-setup');
|
||||
fs.mkdirSync(phaseDir, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(phaseDir, '01-RESEARCH.md'),
|
||||
'# Research\n\n## Validation Architecture\n\nSome validation content.\n'
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(phaseDir, '01-VALIDATION.md'),
|
||||
'# Validation\n\nValidation content.\n'
|
||||
);
|
||||
|
||||
const result = runGsdTools('validate health', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
assert.ok(
|
||||
!output.warnings.some(w => w.code === 'W009'),
|
||||
`Should not have W009: ${JSON.stringify(output.warnings)}`
|
||||
);
|
||||
});
|
||||
|
||||
// ─── Overall status ────────────────────────────────────────────────────────
|
||||
|
||||
test("returns 'healthy' when all checks pass", () => {
|
||||
@@ -509,6 +606,32 @@ describe('validate health --repair command', () => {
|
||||
assert.ok(backupContent.includes('Phase 99'), 'backup should contain the original STATE.md content');
|
||||
});
|
||||
|
||||
test('adds nyquist_validation key to config.json via addNyquistKey repair', () => {
|
||||
writeMinimalStateMd(tmpDir, '# Session State\n\nPhase 1 in progress.\n');
|
||||
// Config with workflow section but missing nyquist_validation
|
||||
const configPath = path.join(tmpDir, '.planning', 'config.json');
|
||||
fs.writeFileSync(
|
||||
configPath,
|
||||
JSON.stringify({ model_profile: 'balanced', workflow: { research: true } }, null, 2)
|
||||
);
|
||||
|
||||
const result = runGsdTools('validate health --repair', tmpDir);
|
||||
assert.ok(result.success, `Command failed: ${result.error}`);
|
||||
|
||||
const output = JSON.parse(result.output);
|
||||
assert.ok(
|
||||
Array.isArray(output.repairs_performed),
|
||||
`Expected repairs_performed array: ${JSON.stringify(output)}`
|
||||
);
|
||||
const addKeyAction = output.repairs_performed.find(r => r.action === 'addNyquistKey');
|
||||
assert.ok(addKeyAction, `Expected addNyquistKey action: ${JSON.stringify(output.repairs_performed)}`);
|
||||
assert.strictEqual(addKeyAction.success, true, 'addNyquistKey should succeed');
|
||||
|
||||
// Read config.json and verify workflow.nyquist_validation is true
|
||||
const diskConfig = JSON.parse(fs.readFileSync(configPath, 'utf-8'));
|
||||
assert.strictEqual(diskConfig.workflow.nyquist_validation, true, 'nyquist_validation should be true');
|
||||
});
|
||||
|
||||
test('reports repairable_count correctly', () => {
|
||||
// No config.json (W003, repairable=true) and no STATE.md (E004, repairable=true)
|
||||
const configPath = path.join(tmpDir, '.planning', 'config.json');
|
||||
|
||||
Reference in New Issue
Block a user