mirror of
https://github.com/glittercowboy/get-shit-done
synced 2026-04-25 17:25:23 +02:00
* feat: make model profiles runtime-aware for Codex/non-Claude runtimes (closes #2517) Adds an optional top-level `runtime` config key plus a `model_profile_overrides[runtime][tier]` map. When `runtime` is set, profile tiers (opus/sonnet/haiku) resolve to runtime-native model IDs (and reasoning_effort where supported) instead of bare Claude aliases. Codex defaults from the spec: opus -> gpt-5.4 reasoning_effort: xhigh sonnet -> gpt-5.3-codex reasoning_effort: medium haiku -> gpt-5.4-mini reasoning_effort: medium Claude defaults mirror MODEL_ALIAS_MAP. Unknown runtimes fall back to the Claude-alias safe default rather than emit IDs the runtime cannot accept. reasoning_effort is only emitted into Codex install paths; never returned from resolveModelInternal and never written to Claude agent frontmatter. Backwards compatible: any user without `runtime` set sees identical behavior — the new branch is gated on `config.runtime != null`. Precedence (highest to lowest): 1. per-agent model_overrides 2. runtime-aware tier resolution (when `runtime` is set) 3. resolve_model_ids: "omit" 4. Claude-native default 5. inherit (literal passthrough) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(#2517): address adversarial review of #2609 (findings 1-16) Addresses all 16 findings from the adversarial review of PR #2609. Each finding is enumerated below with its resolution. CRITICAL - F1: readGsdRuntimeProfileResolver(targetDir) now probes per-project .planning/config.json AND ~/.gsd/defaults.json with per-project winning, so the PR's headline claim ("set runtime in project config and Codex TOML emit picks it up") actually holds end-to-end. - F2: resolveTierEntry field-merges user overrides with built-in defaults. The CONFIGURATION.md string-shorthand example `{ codex: { opus: "gpt-5-pro" } }` now keeps reasoning_effort from the built-in entry. Partial-object overrides like `{ opus: { reasoning_effort: 'low' } }` keep the built-in model. Both paths regression-tested. MAJOR - F3: resolveReasoningEffortInternal gates strictly on the RUNTIMES_WITH_REASONING_EFFORT allowlist regardless of override presence. Override + unknown-runtime no longer leaks reasoning_effort. - F4: runtime:"claude" is now a no-op for resolution (it is the implicit default). It no longer hijacks resolve_model_ids:"omit". Existing tests for `runtime:"claude"` returning Claude IDs were rewritten to reflect the no-op semantics; new test asserts the omit case returns "". - F5: _readGsdConfigFile in install.js writes a stderr warning on JSON parse failure instead of silently returning null. Read failure and parse failure are warned separately. Library require is hoisted to top of install.js so it is not co-mingled with config-read failure modes. - F6: install.js requires for core.cjs / model-profiles.cjs are hoisted to the top of the file with __dirname-based absolute paths so global npm install works regardless of cwd. Test asserts both lib paths exist relative to install.js __dirname. - F7: docs/CONFIGURATION.md `runtime` row no longer lists `opencode` as a valid runtime — install-path emission for non-Codex runtimes is explicitly out of scope per #2517 / #2612, and the doc now points at #2612 for the follow-on work. resolveModelInternal still accepts any runtime string (back-compat) and falls back safely for unknown values. - F8: Tests now isolate HOME (and GSD_HOME) to a per-test tmpdir so the developer's real ~/.gsd/defaults.json cannot bleed into assertions. Same pattern CodeRabbit caught on PRs #2603 / #2604. - F9: `runtime` and `model_profile_overrides` documented as flat-only in core.cjs comments — not routed through `get()` because they are top-level keys per docs/CONFIGURATION.md and introducing nested resolution for two new keys was not worth the edge-case surface. - F10/F13: loadConfig now invokes _warnUnknownProfileOverrides on the raw parsed config so direct .planning/config.json edits surface unknown runtime values (e.g. typo `runtime: "codx"`) and unknown tier values (e.g. `model_profile_overrides.codex.banana`) at read time. Warnings only — preserves back-compat for runtimes added later. Per-process warning cache prevents log spam across repeated loadConfig calls. MINOR / NIT - F11: Removed dead `tier || 'sonnet'` defensive shortcut. The local is now `const alias = tier;` with a comment explaining why `tier` is guaranteed truthy at that point (every MODEL_PROFILES entry defines `balanced`, the fallback profile). - F12: Extracted resolveTierEntry() in core.cjs as the single source of truth for runtime-aware tier resolution. core.cjs and bin/install.js both consume it — no duplicated lookup logic between the two files. - F14: Added regression tests for findings #1, #2, #3, #4, #6, #10, #13 in tests/issue-2517-runtime-aware-profiles.test.cjs. Each must-fix path has a corresponding test that fails against the pre-fix code and passes against the post-fix code. - F15: docs/CONFIGURATION.md `model_profile` row cross-references #1713 / #1806 next to the `adaptive` enum value. - F16: RUNTIME_PROFILE_MAP remains in core.cjs as the single source of truth; install.js imports it through the exported resolveTierEntry helper rather than carrying its own copy. Doc files (CONFIGURATION.md, USER-GUIDE.md, settings.md) intentionally still embed the IDs as text — code comment in core.cjs flags that those doc files must be updated whenever the constant changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
593 lines
26 KiB
JavaScript
593 lines
26 KiB
JavaScript
/**
|
|
* Issue #2517 — runtime-aware model profile resolution.
|
|
*
|
|
* Today, profile tiers (opus/sonnet/haiku) only resolve to Claude IDs. On Codex /
|
|
* other runtimes, users must use `inherit` or write large `model_overrides` blocks.
|
|
*
|
|
* This adds a `runtime` config key + `model_profile_overrides[runtime][tier]` map.
|
|
* When `runtime` is set to a non-Claude value, profile tiers resolve to runtime-
|
|
* native model IDs.
|
|
*
|
|
* Codex: opus -> gpt-5.4 (xhigh), sonnet -> gpt-5.3-codex (medium), haiku -> gpt-5.4-mini (medium)
|
|
*
|
|
* `runtime: "claude"` is the implicit default and is treated as a no-op for
|
|
* resolution — it does not override `resolve_model_ids: "omit"` or any other
|
|
* Claude-native semantics (review finding #4).
|
|
*
|
|
* `inherit` keeps current behavior. Unknown runtimes fall back safely (do NOT emit
|
|
* provider-specific IDs the runtime can't accept) and trigger a one-shot stderr
|
|
* warning so typos like `runtime: "codx"` surface immediately (review finding #13).
|
|
*
|
|
* HOME isolation: every test sets `process.env.HOME` to a per-suite tmpdir so the
|
|
* developer's real `~/.gsd/defaults.json` cannot bleed into assertions
|
|
* (review finding #8 / pattern from CodeRabbit on PRs #2603, #2604).
|
|
*/
|
|
|
|
'use strict';
|
|
|
|
const { describe, test, beforeEach, afterEach } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const fs = require('fs');
|
|
const os = require('os');
|
|
const path = require('path');
|
|
const { createTempProject, cleanup } = require('./helpers.cjs');
|
|
|
|
const {
|
|
resolveModelInternal,
|
|
resolveReasoningEffortInternal,
|
|
resolveTierEntry,
|
|
RUNTIME_PROFILE_MAP,
|
|
KNOWN_RUNTIMES,
|
|
_resetRuntimeWarningCacheForTests,
|
|
} = require('../get-shit-done/bin/lib/core.cjs');
|
|
const { isValidConfigKey } = require('../get-shit-done/bin/lib/config-schema.cjs');
|
|
|
|
function writeConfig(tmpDir, obj) {
|
|
fs.writeFileSync(
|
|
path.join(tmpDir, '.planning', 'config.json'),
|
|
JSON.stringify(obj, null, 2)
|
|
);
|
|
}
|
|
|
|
// ─── Shared HOME isolation (#2517 review finding #8) ────────────────────────
|
|
// Without this, a developer's real `~/.gsd/defaults.json` (e.g. one with
|
|
// `runtime: codex` set) silently overrides test assertions about back-compat
|
|
// behavior. Capture HOME, point it at an isolated tmpdir for the duration of
|
|
// each test, restore on teardown.
|
|
let _origHome;
|
|
let _origGsdHome;
|
|
let _isolatedHome;
|
|
function isolateHome() {
|
|
_origHome = process.env.HOME;
|
|
_origGsdHome = process.env.GSD_HOME;
|
|
_isolatedHome = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-home-iso-'));
|
|
process.env.HOME = _isolatedHome;
|
|
process.env.GSD_HOME = _isolatedHome;
|
|
}
|
|
function restoreHome() {
|
|
if (_origHome === undefined) delete process.env.HOME; else process.env.HOME = _origHome;
|
|
if (_origGsdHome === undefined) delete process.env.GSD_HOME; else process.env.GSD_HOME = _origGsdHome;
|
|
if (_isolatedHome) fs.rmSync(_isolatedHome, { recursive: true, force: true });
|
|
_isolatedHome = null;
|
|
}
|
|
|
|
// ─── Backwards compatibility — no `runtime` set ─────────────────────────────
|
|
describe('issue #2517: backwards compat — no runtime key set', () => {
|
|
let tmpDir;
|
|
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
|
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
|
|
|
test('balanced profile returns Claude alias when runtime absent', () => {
|
|
writeConfig(tmpDir, { model_profile: 'balanced' });
|
|
// gsd-planner balanced -> opus
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'opus');
|
|
});
|
|
|
|
test('inherit profile still returns "inherit" with no runtime', () => {
|
|
writeConfig(tmpDir, { model_profile: 'inherit' });
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'inherit');
|
|
});
|
|
|
|
test('resolve_model_ids:true still maps alias -> full Claude ID with no runtime', () => {
|
|
writeConfig(tmpDir, { model_profile: 'balanced', resolve_model_ids: true });
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'claude-opus-4-6');
|
|
});
|
|
|
|
test('resolve_model_ids:"omit" still returns "" with no runtime', () => {
|
|
writeConfig(tmpDir, { model_profile: 'balanced', resolve_model_ids: 'omit' });
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), '');
|
|
});
|
|
|
|
test('reasoning_effort returns null when runtime absent', () => {
|
|
writeConfig(tmpDir, { model_profile: 'balanced' });
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
|
});
|
|
|
|
test('adaptive profile still works without runtime (#1713/#1806)', () => {
|
|
writeConfig(tmpDir, { model_profile: 'adaptive' });
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'opus');
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'haiku');
|
|
});
|
|
});
|
|
|
|
// ─── runtime: "claude" — no-op (preserves Claude-native semantics) ──────────
|
|
describe('issue #2517: runtime "claude" is a no-op for resolution (finding #4)', () => {
|
|
let tmpDir;
|
|
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
|
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
|
|
|
test('runtime:"claude" + balanced returns the alias, not the resolved Claude ID', () => {
|
|
// `runtime: "claude"` is the implicit default — it must not silently flip
|
|
// resolve_model_ids on. The alias passes through identically to the unset case.
|
|
writeConfig(tmpDir, { runtime: 'claude', model_profile: 'balanced' });
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'opus');
|
|
});
|
|
|
|
test('runtime:"claude" + resolve_model_ids:"omit" returns "" (finding #4 regression)', () => {
|
|
// The pre-fix bug: runtime:"claude" hijacked the resolution chain and
|
|
// returned `claude-opus-4-6` even when the user explicitly asked for the
|
|
// omit semantics.
|
|
writeConfig(tmpDir, {
|
|
runtime: 'claude',
|
|
model_profile: 'quality',
|
|
resolve_model_ids: 'omit',
|
|
});
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), '');
|
|
});
|
|
|
|
test('runtime:"claude" + resolve_model_ids:true maps alias -> full Claude ID', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'claude',
|
|
model_profile: 'quality',
|
|
resolve_model_ids: true,
|
|
});
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'claude-opus-4-6');
|
|
});
|
|
|
|
test('reasoning_effort is null on Claude (never leaks)', () => {
|
|
writeConfig(tmpDir, { runtime: 'claude', model_profile: 'quality' });
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
|
});
|
|
});
|
|
|
|
// ─── runtime: "codex" — resolves tiers to Codex IDs + reasoning_effort ──────
|
|
describe('issue #2517: runtime "codex" — Codex tier resolution', () => {
|
|
let tmpDir;
|
|
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
|
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
|
|
|
test('opus tier -> gpt-5.4 with reasoning_effort xhigh', () => {
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'quality' });
|
|
// gsd-planner quality -> opus -> gpt-5.4
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4');
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
|
});
|
|
|
|
test('sonnet tier -> gpt-5.3-codex with reasoning_effort medium', () => {
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'balanced' });
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-roadmapper'), 'gpt-5.3-codex');
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-roadmapper'), 'medium');
|
|
});
|
|
|
|
test('haiku tier -> gpt-5.4-mini with reasoning_effort medium', () => {
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'budget' });
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'gpt-5.4-mini');
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-codebase-mapper'), 'medium');
|
|
});
|
|
|
|
test('adaptive profile resolves on Codex (no #1713/#1806 regression)', () => {
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'adaptive' });
|
|
// gsd-planner adaptive -> opus -> gpt-5.4
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4');
|
|
// gsd-codebase-mapper adaptive -> haiku -> gpt-5.4-mini
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'gpt-5.4-mini');
|
|
});
|
|
|
|
test('inherit profile still returns "inherit" on Codex', () => {
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'inherit' });
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'inherit');
|
|
// No reasoning_effort when inherit
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
|
});
|
|
|
|
test('runtime:"codex" beats resolve_model_ids:"omit" (explicit non-Claude opt-in wins)', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile: 'quality',
|
|
resolve_model_ids: 'omit',
|
|
});
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4');
|
|
});
|
|
});
|
|
|
|
// ─── Precedence chain ───────────────────────────────────────────────────────
|
|
describe('issue #2517: precedence chain', () => {
|
|
let tmpDir;
|
|
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
|
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
|
|
|
test('per-agent model_overrides wins over runtime tier resolution', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile: 'quality',
|
|
model_overrides: { 'gsd-planner': 'gpt-5.4-mini' },
|
|
});
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4-mini');
|
|
});
|
|
|
|
test('model_profile_overrides[runtime][tier] beats built-in defaults', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile: 'quality',
|
|
model_profile_overrides: {
|
|
codex: { opus: 'gpt-5-pro' },
|
|
},
|
|
});
|
|
// gsd-planner quality -> opus -> overridden to gpt-5-pro
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5-pro');
|
|
// haiku not overridden — fall back to spec defaults
|
|
// gsd-codebase-mapper quality -> sonnet -> gpt-5.3-codex
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-codebase-mapper'), 'gpt-5.3-codex');
|
|
});
|
|
|
|
test('partial profile_overrides — only opus overridden, sonnet uses default', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile: 'balanced',
|
|
model_profile_overrides: {
|
|
codex: { opus: 'gpt-5-pro' }, // only opus overridden
|
|
},
|
|
});
|
|
// gsd-planner balanced -> opus -> overridden to gpt-5-pro
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5-pro');
|
|
// gsd-roadmapper balanced -> sonnet -> spec default
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-roadmapper'), 'gpt-5.3-codex');
|
|
});
|
|
|
|
test('per-agent override beats profile override beats default', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile: 'quality',
|
|
model_profile_overrides: { codex: { opus: 'gpt-5-pro' } },
|
|
model_overrides: { 'gsd-planner': 'custom-model' },
|
|
});
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'custom-model');
|
|
});
|
|
});
|
|
|
|
// ─── Field-merge semantics — review findings #2 ─────────────────────────────
|
|
describe('issue #2517: field-merge of overrides with built-in defaults (finding #2)', () => {
|
|
let tmpDir;
|
|
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
|
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
|
|
|
test('string-shorthand override keeps reasoning_effort from built-in (CONFIGURATION.md example)', () => {
|
|
// `{ codex: { opus: "gpt-5-pro" } }` is the documented shorthand. Pre-fix,
|
|
// it silently dropped reasoning_effort. Post-fix, the model is overridden
|
|
// and reasoning_effort comes from the built-in entry.
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile: 'quality',
|
|
model_profile_overrides: { codex: { opus: 'gpt-5-pro' } },
|
|
});
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5-pro');
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'xhigh');
|
|
});
|
|
|
|
test('partial-object override (no model) keeps model from built-in', () => {
|
|
// `{ codex: { opus: { reasoning_effort: "low" } } }` previously dropped
|
|
// the model entirely (returned undefined and fell through). Post-fix, the
|
|
// built-in `gpt-5.4` model is preserved and `low` reasoning_effort wins.
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile: 'quality',
|
|
model_profile_overrides: { codex: { opus: { reasoning_effort: 'low' } } },
|
|
});
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4');
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'low');
|
|
});
|
|
|
|
test('full-object override replaces both fields', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile: 'quality',
|
|
model_profile_overrides: {
|
|
codex: { opus: { model: 'custom-model', reasoning_effort: 'minimal' } },
|
|
},
|
|
});
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'custom-model');
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), 'minimal');
|
|
});
|
|
|
|
test('resolveTierEntry helper: shorthand merge', () => {
|
|
// Direct unit-test of the shared helper used by core + install.js.
|
|
const entry = resolveTierEntry({
|
|
runtime: 'codex',
|
|
tier: 'opus',
|
|
overrides: { codex: { opus: 'gpt-5-pro' } },
|
|
});
|
|
assert.deepStrictEqual(entry, { model: 'gpt-5-pro', reasoning_effort: 'xhigh' });
|
|
});
|
|
|
|
test('resolveTierEntry helper: partial-object merge keeps built-in model', () => {
|
|
const entry = resolveTierEntry({
|
|
runtime: 'codex',
|
|
tier: 'opus',
|
|
overrides: { codex: { opus: { reasoning_effort: 'low' } } },
|
|
});
|
|
assert.deepStrictEqual(entry, { model: 'gpt-5.4', reasoning_effort: 'low' });
|
|
});
|
|
|
|
test('resolveTierEntry helper: unknown runtime + no overrides -> null', () => {
|
|
const entry = resolveTierEntry({
|
|
runtime: 'mystery',
|
|
tier: 'opus',
|
|
overrides: null,
|
|
});
|
|
assert.strictEqual(entry, null);
|
|
});
|
|
});
|
|
|
|
// ─── reasoning_effort allowlist (review finding #3) ─────────────────────────
|
|
describe('issue #2517: reasoning_effort allowlist gates regardless of overrides (finding #3)', () => {
|
|
let tmpDir;
|
|
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
|
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
|
|
|
test('unknown runtime with overrides supplying reasoning_effort yields null effort', () => {
|
|
// Pre-fix: `if (!overrides) return null` left a hole — overrides for an
|
|
// unknown runtime made effort propagate, defeating the typo guard.
|
|
writeConfig(tmpDir, {
|
|
runtime: 'mystery',
|
|
model_profile: 'quality',
|
|
model_profile_overrides: {
|
|
mystery: { opus: { model: 'mystery-opus', reasoning_effort: 'xhigh' } },
|
|
},
|
|
});
|
|
// Model still resolves (overrides are honored).
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'mystery-opus');
|
|
// …but reasoning_effort does NOT propagate to a runtime not in the allowlist.
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
|
});
|
|
|
|
test('typo runtime "codx" with overrides yields null effort (no leak into install path)', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codx',
|
|
model_profile: 'quality',
|
|
model_profile_overrides: { codx: { opus: { model: 'gpt-5.4', reasoning_effort: 'xhigh' } } },
|
|
});
|
|
assert.strictEqual(resolveReasoningEffortInternal(tmpDir, 'gsd-planner'), null);
|
|
});
|
|
});
|
|
|
|
// ─── Unknown runtime / unknown tier ─────────────────────────────────────────
|
|
describe('issue #2517: unknown runtime + safe fallback', () => {
|
|
let tmpDir;
|
|
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
|
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
|
|
|
test('unknown runtime falls back to Claude-alias safe default (no Codex IDs leaked)', () => {
|
|
writeConfig(tmpDir, { runtime: 'mystery-runtime', model_profile: 'quality' });
|
|
// Should NOT emit gpt-5.4 — should fall back to Claude alias
|
|
const resolved = resolveModelInternal(tmpDir, 'gsd-planner');
|
|
assert.notStrictEqual(resolved, 'gpt-5.4');
|
|
assert.strictEqual(resolved, 'opus');
|
|
});
|
|
|
|
test('unknown runtime + user-provided overrides for that runtime — uses overrides', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'mystery-runtime',
|
|
model_profile: 'quality',
|
|
model_profile_overrides: {
|
|
'mystery-runtime': { opus: 'mystery-opus' },
|
|
},
|
|
});
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'mystery-opus');
|
|
});
|
|
|
|
test('runtime:"codex" but missing model_profile_overrides[codex] uses spec defaults', () => {
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'quality' });
|
|
// No model_profile_overrides at all — built-in Codex defaults take over
|
|
assert.strictEqual(resolveModelInternal(tmpDir, 'gsd-planner'), 'gpt-5.4');
|
|
});
|
|
});
|
|
|
|
// ─── Schema validation (config-set time + load time) ────────────────────────
|
|
describe('issue #2517: VALID_CONFIG_KEYS schema', () => {
|
|
test('"runtime" is a valid config key', () => {
|
|
assert.strictEqual(isValidConfigKey('runtime'), true);
|
|
});
|
|
|
|
test('model_profile_overrides.codex.opus is valid', () => {
|
|
assert.strictEqual(isValidConfigKey('model_profile_overrides.codex.opus'), true);
|
|
});
|
|
|
|
test('model_profile_overrides.codex.sonnet is valid', () => {
|
|
assert.strictEqual(isValidConfigKey('model_profile_overrides.codex.sonnet'), true);
|
|
});
|
|
|
|
test('model_profile_overrides.codex.haiku is valid', () => {
|
|
assert.strictEqual(isValidConfigKey('model_profile_overrides.codex.haiku'), true);
|
|
});
|
|
|
|
test('model_profile_overrides.claude.opus is valid', () => {
|
|
assert.strictEqual(isValidConfigKey('model_profile_overrides.claude.opus'), true);
|
|
});
|
|
|
|
test('model_profile_overrides with unknown runtime is valid (free-string runtime)', () => {
|
|
assert.strictEqual(isValidConfigKey('model_profile_overrides.acme.opus'), true);
|
|
});
|
|
|
|
test('model_profile_overrides with bogus tier is rejected', () => {
|
|
assert.strictEqual(isValidConfigKey('model_profile_overrides.codex.banana'), false);
|
|
});
|
|
|
|
test('model_profile_overrides without tier is rejected', () => {
|
|
assert.strictEqual(isValidConfigKey('model_profile_overrides.codex'), false);
|
|
});
|
|
|
|
test('model_profile_overrides root key alone is rejected (must include runtime+tier)', () => {
|
|
assert.strictEqual(isValidConfigKey('model_profile_overrides'), false);
|
|
});
|
|
});
|
|
|
|
// ─── loadConfig validation warnings (review findings #10, #13) ──────────────
|
|
describe('issue #2517: loadConfig warns on unknown runtime/tier (findings #10, #13)', () => {
|
|
const { loadConfig } = require('../get-shit-done/bin/lib/core.cjs');
|
|
let tmpDir;
|
|
let origWrite;
|
|
let captured;
|
|
beforeEach(() => {
|
|
isolateHome();
|
|
tmpDir = createTempProject();
|
|
_resetRuntimeWarningCacheForTests();
|
|
captured = [];
|
|
origWrite = process.stderr.write.bind(process.stderr);
|
|
process.stderr.write = (chunk) => { captured.push(String(chunk)); return true; };
|
|
});
|
|
afterEach(() => { process.stderr.write = origWrite; cleanup(tmpDir); restoreHome(); });
|
|
|
|
test('unknown runtime triggers a stderr warning', () => {
|
|
writeConfig(tmpDir, { runtime: 'codx', model_profile: 'quality' });
|
|
loadConfig(tmpDir);
|
|
const joined = captured.join('');
|
|
assert.match(joined, /unknown value "codx"/);
|
|
});
|
|
|
|
test('known runtime does NOT trigger a runtime warning', () => {
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'quality' });
|
|
loadConfig(tmpDir);
|
|
const joined = captured.join('');
|
|
assert.doesNotMatch(joined, /unknown value/);
|
|
});
|
|
|
|
test('unknown tier in overrides triggers a stderr warning', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile_overrides: { codex: { banana: 'whatever' } },
|
|
});
|
|
loadConfig(tmpDir);
|
|
const joined = captured.join('');
|
|
assert.match(joined, /unknown tier "banana"/);
|
|
});
|
|
|
|
test('unknown runtime in overrides triggers a stderr warning', () => {
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile_overrides: { mystery: { opus: 'whatever' } },
|
|
});
|
|
loadConfig(tmpDir);
|
|
const joined = captured.join('');
|
|
assert.match(joined, /model_profile_overrides\.mystery\.\* uses unknown runtime/);
|
|
});
|
|
|
|
test('every name in KNOWN_RUNTIMES survives the warning gate', () => {
|
|
// Smoke check: `KNOWN_RUNTIMES` must list every runtime `bin/install.js`
|
|
// emits for, otherwise legitimate users get spammed at every loadConfig.
|
|
for (const r of KNOWN_RUNTIMES) {
|
|
assert.ok(typeof r === 'string' && r.length > 0);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── End-to-end: per-project config -> Codex TOML emit (finding #1) ─────────
|
|
describe('issue #2517: install end-to-end — per-project config reaches Codex TOML (finding #1)', () => {
|
|
// Load install.js in test-mode so its module exports are populated.
|
|
const prevTestMode = process.env.GSD_TEST_MODE;
|
|
process.env.GSD_TEST_MODE = '1';
|
|
const installMod = require('../bin/install.js');
|
|
if (prevTestMode === undefined) delete process.env.GSD_TEST_MODE;
|
|
else process.env.GSD_TEST_MODE = prevTestMode;
|
|
const { readGsdRuntimeProfileResolver, generateCodexAgentToml } = installMod;
|
|
|
|
let tmpDir;
|
|
beforeEach(() => { isolateHome(); tmpDir = createTempProject(); _resetRuntimeWarningCacheForTests(); });
|
|
afterEach(() => { cleanup(tmpDir); restoreHome(); });
|
|
|
|
test('readGsdRuntimeProfileResolver picks up runtime from .planning/config.json', () => {
|
|
// No ~/.gsd/defaults.json (HOME is isolated tmpdir). Per-project config alone
|
|
// must drive the resolver — pre-fix, it returned null.
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'quality' });
|
|
const resolver = readGsdRuntimeProfileResolver(tmpDir);
|
|
assert.ok(resolver, 'expected a resolver from per-project config');
|
|
assert.strictEqual(resolver.runtime, 'codex');
|
|
const entry = resolver.resolve('gsd-planner');
|
|
assert.deepStrictEqual(entry, { model: 'gpt-5.4', reasoning_effort: 'xhigh' });
|
|
});
|
|
|
|
test('per-project config wins over global ~/.gsd/defaults.json', () => {
|
|
fs.mkdirSync(path.join(_isolatedHome, '.gsd'), { recursive: true });
|
|
fs.writeFileSync(
|
|
path.join(_isolatedHome, '.gsd', 'defaults.json'),
|
|
JSON.stringify({ runtime: 'claude', model_profile: 'budget' })
|
|
);
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'quality' });
|
|
const resolver = readGsdRuntimeProfileResolver(tmpDir);
|
|
assert.strictEqual(resolver.runtime, 'codex');
|
|
const entry = resolver.resolve('gsd-planner');
|
|
assert.strictEqual(entry.model, 'gpt-5.4');
|
|
});
|
|
|
|
test('generated Codex TOML embeds model = and model_reasoning_effort = lines', () => {
|
|
writeConfig(tmpDir, { runtime: 'codex', model_profile: 'quality' });
|
|
const resolver = readGsdRuntimeProfileResolver(tmpDir);
|
|
const toml = generateCodexAgentToml(
|
|
'gsd-planner',
|
|
'---\nname: gsd-planner\ndescription: Planner agent\n---\nBody.\n',
|
|
null,
|
|
resolver
|
|
);
|
|
assert.match(toml, /^model = "gpt-5\.4"$/m);
|
|
assert.match(toml, /^model_reasoning_effort = "xhigh"$/m);
|
|
});
|
|
|
|
test('generated TOML omits reasoning_effort when runtime has none', () => {
|
|
// For a known runtime with model but no reasoning_effort, only model is emitted.
|
|
// Use the user-override path to simulate this with codex (no built-in returns
|
|
// model alone, so fabricate via override of an unknown-runtime entry).
|
|
writeConfig(tmpDir, {
|
|
runtime: 'codex',
|
|
model_profile: 'quality',
|
|
model_profile_overrides: { codex: { opus: { model: 'custom', reasoning_effort: '' } } },
|
|
});
|
|
const resolver = readGsdRuntimeProfileResolver(tmpDir);
|
|
const toml = generateCodexAgentToml(
|
|
'gsd-planner',
|
|
'---\nname: gsd-planner\n---\nBody.\n',
|
|
null,
|
|
resolver
|
|
);
|
|
assert.match(toml, /^model = "custom"$/m);
|
|
assert.doesNotMatch(toml, /model_reasoning_effort/);
|
|
});
|
|
|
|
test('resolver returns null with no global, no per-project config', () => {
|
|
// Sanity: nothing configured -> nothing emitted. Pre-existing back-compat.
|
|
const resolver = readGsdRuntimeProfileResolver(tmpDir);
|
|
assert.strictEqual(resolver, null);
|
|
});
|
|
|
|
test('inline require paths resolve relative to install.js __dirname (finding #6)', () => {
|
|
// Defensive: assert the lib files install.js requires actually exist at
|
|
// resolver-construction time. Catches accidental relative-path drift in CI.
|
|
const installDir = path.dirname(require.resolve('../bin/install.js'));
|
|
const libDir = path.join(installDir, '..', 'get-shit-done', 'bin', 'lib');
|
|
assert.ok(fs.existsSync(path.join(libDir, 'core.cjs')));
|
|
assert.ok(fs.existsSync(path.join(libDir, 'model-profiles.cjs')));
|
|
});
|
|
});
|
|
|
|
// ─── RUNTIME_PROFILE_MAP single source of truth (finding #16) ───────────────
|
|
describe('issue #2517: RUNTIME_PROFILE_MAP single source of truth (finding #16)', () => {
|
|
test('install.js consumes the same map as core.cjs', () => {
|
|
// `bin/install.js` must NOT carry its own duplicate copy of the map.
|
|
// The shared resolver imported in install.js exposes `runtime` and the
|
|
// entries through `resolveTierEntry`, so any future drift between the two
|
|
// files would surface as a test failure here rather than a silent bug.
|
|
const codexOpus = RUNTIME_PROFILE_MAP.codex?.opus;
|
|
assert.deepStrictEqual(codexOpus, { model: 'gpt-5.4', reasoning_effort: 'xhigh' });
|
|
const claudeOpus = RUNTIME_PROFILE_MAP.claude?.opus;
|
|
assert.deepStrictEqual(claudeOpus, { model: 'claude-opus-4-6' });
|
|
});
|
|
});
|