eliott/claude-mem
1
0
Fork 0
mirror of https://github.com/thedotmack/claude-mem synced 2026-04-25 17:15:04 +02:00
Code Issues Packages Projects Releases Wiki Activity

refactor(phase-1): dead-code deletion (-3,543 LoC)

Browse Source 
Deletes files and symbols with zero importers or self-declared deprecation.
No behavior change — structural cleanup only.

Removed:
- hook-response.ts, utils/bun-path.ts (zero importers)
- cli/handlers/user-message.ts (not wired in hooks.json)
- services/Context.ts + context-generator.ts (deprecated stubs)
- sqlite/migrations.ts (645 lines, pre-SDK schema, unused)
- DatabaseManager singleton + getDatabase + initializeDatabase
- 6 sqlite re-export shells (Observations/Sessions/Summaries/Prompts/Timeline/Import)
- worker/search/{strategies,filters}/ dirs (dead via unused SearchOrchestrator)
- SearchOrchestrator, TimelineBuilder, ResultFormatter
- TimelineService.formatTimeline (137 lines, unused)
- ProcessManager.cleanupOrphanedProcesses + createSignalHandler
- Duplicate php: key in smart-file-read/parser.ts

Rewired:
- SearchRoutes dynamic imports (services/Context → services/context)
- CorpusBuilder: SearchOrchestrator → SearchManager.search({format:'json'})
- build-hooks.js entry: context-generator.ts → services/context/index.ts
- scripts/ imports for moved transcript-parser.ts

Moved:
- utils/transcript-parser.ts → scripts/transcript-parser.ts (only callers)

Skipped (plan was wrong):
- consecutiveRestarts: live backoff math at SessionRoutes.ts:348
- AgentFormatter stubs: wired to HeaderRenderer with passing tests

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Alex Newman
2026-04-20 22:55:10 -07:00
parent 49ab404c08
commit 5a1a17c0b2
64 changed files with 1251 additions and 5676 deletions
Download Patch File Download Diff File Expand all files Collapse all files

233
greptile-field-report.md Normal file
View File

@@ -0,0 +1,233 @@
# Greptile in the Wild: A Field Report from claude-mem
**Source:** 8 weeks of production use on the [claude-mem](https://github.com/thedotmack/claude-mem) open-source repository (Jan 26 – Apr 20, 2026), reconstructed from the maintainer's persistent-memory system (which records every tool call, review, and decision across sessions).
**Scope:** 22 Greptile-specific observations / 102,154 tokens of captured context, covering 15+ PRs Greptile formally reviewed, cross-referenced against CodeRabbit (Phase 1 + Pro), Claude bot (`claude-code-review.yml`), and GitHub Copilot on the same repo.
**Intent:** Share novel insights — patterns that wouldn't show up in Greptile's own analytics — with the product team so you can act on them.
---
## TL;DR
1. **The confidence score is the actual product.** A 0/5 on PR #968 caused the author to self-close within 8 hours without any human weighing in. That's the single most decisive automated-review action captured in 8 months of this repo's history.
2. **Greptile's real job description is heavier than "review assistant."** On Feb 24, 2026, it was the *sole reviewer* across 34 open PRs from 13+ contributors with zero human approvals on any of them. Design for "only voice in the room," not "augments human reviewers."
3. **The 2–4 comment depth ceiling is the #1 product miss in this repo's memory.** Specific receipts where it caused real bugs to ship inside.
4. **False positives cluster on `finally`-block cleanup paths.** A control-flow pass for cleanup semantics would eliminate the most memorable Greptile miss in the repo.
5. **"Confidence" genuinely confuses users.** A per-score label (Halt / Rework / Revise / Discuss / Polish / Ship) would resolve it without rebranding.
---
## 1. What Greptile got right
### 1.1 The confidence score is the product, not the comments
The 0/5 – 5/5 per-PR score did more decision-shaping than any individual comment. Concretely:
- **PR #968** (MemU backend swap): Greptile scored **0/5**. The author self-closed **7h 45m later** without a human weighing in. No other tool in the repo — CodeRabbit, Claude bot, Copilot — produced that kind of decisive author-side action.
- **PR #1006** (Windows platform improvements): scored **4/5**. The maintainer merged it but rejected Greptile's specific suggested patches. The score was the merge signal; the patches were advisory.
The score is the merge-risk prior maintainers actually use. If the PM ever considers de-emphasizing it, don't.
### 1.2 Fast CI check UX
The `Greptile Review` status check finishes in **2–3 minutes** (PR #953: 2m33s; PR #894: 1m59s; PR #863: 2m45s). On small PRs it's sometimes the only check that runs. That speed enables it to be in front of maintainers before they lose context on the diff.
### 1.3 "Prompt To Fix With AI" button is a real differentiator
The maintainer specifically noted this feature on PR #917's CORS review. Other tools produce comments; Greptile converts comments into AI-actionable remediation prompts. No other reviewer in this repo has this.
### 1.4 You don't have the minified-bundle problem Copilot has
`plugin/scripts/*.js` in this repo is pre-minified. Copilot flags every single-letter variable as "unused" and produces a **60–65% false-positive rate** on PRs that touch that directory. Greptile handles the same PRs cleanly. This is a direct competitive advantage you can measure.
### 1.5 The PR #917 CORS review is your case-study material
Greptile's highest-quality review in the captured history: **confidence 4/5**, four findings, all valid — IPv6 localhost support, portless localhost check, tests-exercising-duplicated-logic instead of real middleware, and explicit coverage gap. This is the shape of what Greptile does best: medium-complexity security-adjacent diffs. Worth showing to sales.
---
## 2. Where Greptile hit a ceiling (receipts)
### 2.1 The 2–4 comment depth limit is the structural #1 miss
Across every complex PR in the memory, Greptile posted between 2 and 4 inline comments. On simple PRs this is fine. On complex ones it's where the real bugs live that Greptile doesn't catch.
**Canonical case — PR #1176** (ChromaMcpManager migration):
- Greptile posted **3 findings**, **2 of which were false positives** (see §2.2).
- Claude bot, reviewing the same PR, caught:
- **SQL injection** via unvalidated ID interpolation in `ensureBackfilled`
- **`distances` / `metadatas` array desync** in `queryChroma` causing silent data corruption
- Greptile missed both.
This isn't a model-quality problem. It's an architectural budget problem. On a PR diff of that size, the tool hits its comment quota before it hits the real bugs. The fix is either raising the ceiling on complex diffs or exposing the budget as a knob.
### 2.2 False positives cluster on `finally`-block cleanup
Both FPs on PR #1176 were "leak" accusations against cleanup paths that were already correctly guarded by existing `finally` blocks:
1. Connection-lock leak in `ensureConnected()` — already handled in `finally`
2. Timer leak on successful connection — already handled in `finally`
A control-flow pass that marks `finally` reachability for cleanup claims would eliminate the single most memorable Greptile miss in this repo. Predictable, low-cost, high-brand-impact win.
### 2.3 No CLAUDE.md / repo-convention awareness
The Claude bot is unique among the four reviewers in this repo for reading `CLAUDE.md` before reviewing. That's why Claude bot caught repo-specific conventions (e.g., the "fail fast" error-handling discipline) that Greptile didn't.
This is a 1-day integration: look for `CLAUDE.md`, `AGENTS.md`, `.cursorrules`, or a `CONTRIBUTING.md` at repo root, prepend to the review context. It would step-function improve suggestion-correctness on opinionated repos, which are exactly the repos with the best-informed maintainers and the loudest opinions about review tools.
### 2.4 No cross-PR memory or duplicate-PR detection
On **Feb 24, 2026**, two different contributors independently shipped fixes for the same Windows `uvx.cmd` spawn bug:
- **PR #1191** (4 days old, minimal approach)
- **PR #1211** (1 day old, more robust approach)
**Greptile reviewed both without noticing they were duplicates.** The maintainer caught it during triage, not Greptile.
Meanwhile, CodeRabbit Pro's review of PR #1461 in March 2026 explicitly cited PR #1422 by number — cross-PR memory. That capability is table stakes if you want to compete at the top of the market, especially given Greptile's breadth-of-review positioning.
### 2.5 No stale-PR / conflict-awareness nudging
At the Feb 24 snapshot, **20 of 34 open PRs (59%) were in `CONFLICTING` state**. Greptile had reviewed most of them — but nothing in its output flagged that the review was rotting because main had drifted. Automated review does nothing to keep PRs mergeable if the only thing it produces is comments on a stale diff.
Product idea: a "freshness" signal on prior Greptile reviews when the base branch has moved significantly — "this review was against commit abc123; main is now at def456, consider re-running."
---
## 3. The structural pattern you might not see in your analytics
This is the insight that surprised me most from the data.
### 3.1 The Feb 24, 2026 snapshot
Raw numbers from a `gh pr list` audit on that date (captured as observations #55594–#55597):
| Metric | Value |
|---|---|
| Open PRs | **34** |
| Unique contributors | **13+** |
| `CONFLICTING` state | **20 of 34 (59%)** |
| Oldest PR | ~2 months |
| PRs with any human approval | **0** |
| PRs with any human `CHANGES_REQUESTED` | **0** |
| PRs with Greptile review | "most" |
Verbatim from the captured observation: *"No PR has been approved or had changes requested by a human reviewer. The only reviews recorded are from the Greptile AI code review bot. This means triage decisions rest entirely with the maintainer based on PR content rather than peer review signals."*
### 3.2 "Sole informant, not gatekeeper"
I initially wrote "Greptile was gating these PRs." That was wrong. Greptile posts `COMMENTED`-only reviews — never `CHANGES_REQUESTED` — so it never literally blocked anything. The accurate framing is:
**Greptile was the sole outside voice the maintainer consulted when making unilateral merge decisions on 34 PRs from 13+ contributors.**
That's a different job than "augment human reviewers." The product wasn't enhancing peer review — it was **substituting for peer review** for an open-source project in a contributor-heavy phase. It worked well enough that the maintainer let it run for ~24 days before auditing.
### 3.3 Implication: contributors see Greptile as "the reviewer"
For most of those 13+ contributors, **Greptile was the only feedback they ever got** before the PR was merged or closed. That raises the brand stakes considerably:
- A first-time contributor sees a 3/5 score with two suggestions. That's the entire review UX.
- If Greptile is wrong (PR #1176's FPs), there's no human to correct it.
- If Greptile is right (PR #968's 0/5), the contributor self-closes based on a bot's verdict.
You are sometimes the ONLY reviewer. If product decisions were made under that assumption instead of "we augment," the design choices would change.
### 3.4 Trial-expiry-notice-as-first-impression is brand friction
The earliest Greptile message captured on this repo (PR #863, Feb 1, 2026) was **a trial-expiry notice**, not a review. It's the literal first impression the memory system captured of your product.
Worth an in-channel billing/UX audit: trial-expiry notices should not appear as PR review comments. They should appear in the dashboard, as repo-admin DMs, or as a one-time install-time banner. Not as the first Greptile message a maintainer sees.
---
## 4. The "confidence in what?" problem
The maintainer was genuinely confused by what "Confidence 5/5" is making a claim about. The stock answer — "confident in safety and quality" — conflates three distinct dimensions:
1. **Epistemic** — "how sure are we of our own analysis"
2. **Outcome prediction** — "will this merge safely"
3. **Code quality claim** — "is this code good"
The word "confidence" silently slides between all three. When Greptile is wrong, which dimension failed is a matter of interpretation.
### 4.1 Five single-word replacements evaluated
Against six tests (graceful failure on PR #1006, missed-bug failure on PR #1176, disagreement with CodeRabbit on PR #2073, first-time-contributor readability, CI-integration fit, ceiling-honesty):
| Name | Polarity | Summary |
|---|---|---|
| **Judgment** | 5 = good | Highest honesty, survives failure best, culturally resonant. Weak CI fit. |
| **Merge Readiness** | 5 = ship | Best newcomer readability + CI fit. Overclaims when wrong. |
| **Risk** | 5 = dangerous | Inverted polarity is a permanent UX tax. Reject. |
| **Change Quality** | 5 = good | Conflates too many sub-dimensions. Reject. |
| **Review Depth** | 5 = deep | Semantically cleanest but self-falsifies when Greptile misses a bug. Reject unless depth improves. |
### 4.2 The best option: keep "Confidence," add per-score labels
The strongest proposal in our review session was to keep the noun "Confidence" (preserves brand equity, no polarity inversion, no rebrand cost) but attach an **action label to each score**:
| Score | Label | What the reader should do |
|:---:|:---|:---|
| **0/5** | **Halt** | Close or redesign from the top |
| **1/5** | **Rework** | Significant problems; rebuild before re-review |
| **2/5** | **Revise** | Real bugs flagged; address them |
| **3/5** | **Discuss** | Concerns worth talking through before merge |
| **4/5** | **Polish** | Looks good; minor nits only |
| **5/5** | **Ship** | Merge with confidence |
Why this is better than any single-word replacement:
- **Preserves brand.** Keeps "Confidence" on the header.
- **Resolves the ambiguity at the score-label level.** "Confidence 4/5 — Polish" is unambiguous: *confident that this PR is ready to polish and merge*.
- **CI-gateable without losing humility.** Branch protection can require `≥ 4 (Polish)`. Matches CodeRabbit Pro's `CHANGES_REQUESTED` gating without overclaiming on the single-word noun.
- **Survives failure modes.** "Ship + missed bug" is no worse than "5/5 + missed bug" today; the label adds signal without adding liability.
- **Newcomer-parseable.** "Confidence 2/5 — Revise" tells a first-time contributor exactly what to do.
- **Sharpens disagreements.** On PR #2073 (Greptile 5/5 vs CodeRabbit 15 issues), "Ship" vs `CHANGES_REQUESTED` reads as two votes in the same semantic space — which is better than two numbers that look like they're measuring different things.
Low cost, high legibility win. Recommend shipping.
---
## 5. Competitive positioning (from real in-repo evidence)
| Tool | Greptile is… | Receipt |
|---|---|---|
| **CodeRabbit Pro** | complementary, secondary | CodeRabbit wins on depth + cross-PR memory (PR #1461 cited #1422); Greptile wins on speed + triage score. March 23 bake-off (#61319) concluded: "Keep CodeRabbit Pro primary, triage with Greptile." |
| **Claude bot** | cleaner, more focused | Claude bot's `synchronize` trigger causes 5–6 re-runs per PR; Greptile posts once. Claude bot reads CLAUDE.md; Greptile doesn't. |
| **GitHub Copilot** | direct competitor, Greptile wins | Greptile handles minified-bundle PRs cleanly; Copilot's 60–65% FP rate on those PRs is catastrophic. |
**Divergence as a product surface:** The PR #2073 dynamic — Greptile 5/5 safe-to-merge, CodeRabbit 15 issues with `CHANGES_REQUESTED` — is a recurring pattern in April 2026 PRs (also #2078, #2079). When the two tools disagree at magnitude, that divergence is itself a signal. Worth surfacing: *"CodeRabbit disagrees with your Ship verdict on this PR."* Could be a paid-tier feature or a free differentiator.
---
## 6. Ranked product recommendations
1. **Fix the `finally`-block false-positive class.** A control-flow pass for cleanup semantics. Highest ROI, cleanest win. The PR #1176 FPs are the most memorable Greptile miss in this repo — fix the class and the story flips.
2. **Read `CLAUDE.md` / `AGENTS.md` / `CONTRIBUTING.md`.** 1-day integration. Step-function improvement on opinionated repos.
3. **Ship per-score labels** (Halt / Rework / Revise / Discuss / Polish / Ship). Resolves "confidence in what?" without a rebrand.
4. **Raise the 2–4 comment depth ceiling on complex diffs.** Either adaptive (bigger diff → more budget) or a user-facing "depth" knob.
5. **Add `CHANGES_REQUESTED` support.** Close the gating-capability gap with CodeRabbit Pro. Without this, Greptile can never be primary on repos that want automated gates.
6. **Add duplicate-PR / cross-PR memory.** Table stakes at the top of the market. The PR #1191 / #1211 duplicate-miss is the canonical case.
7. **Add stale-PR / conflict-awareness signals.** Note when a prior Greptile review was against an outdated base commit. 59% `CONFLICTING` PR rates are real.
8. **Fix the trial-expiry-as-first-impression UX.** Trial messaging should not appear as a PR review comment.
---
## 7. The single most important sentence
On this repo, at peak load, Greptile was the only outside voice in a maintainer's head during 34 unilateral merge decisions across 13+ contributors. **The product's real job description is heavier than "review assistant."** Design for that reality and the roadmap writes itself.
---
## Appendix: PR-level evidence table
Compressed from the source timeline analysis. Every PR where Greptile left a captured review, chronological:
| PR | Date | Greptile score | Finding summary | Valid? |
|---|---|---|---|---|
| #856 | Feb 1 | (trial) | Race condition in `lastActivityTime` reset | ✅ |
| #863 | Feb 1 | 4/5 | Template literal spacing nit | ✅ (trivial) |
| #879 | Feb 6 | — | ps-output parsing robustness | ✅ minor |
| #882 | Feb 6 | — | Windows notes in wrong doc | ✅ structural |
| #894 | Feb 6 | pass | CI check only (docs-only PR) | — |
| #917 | Feb 5 | 4/5 | IPv6 / portless localhost / test coverage gaps on CORS | ✅ (best Greptile review in repo) |
| #953 | Feb 6 | pass | CI check only (README formatting) | — |
| #968 | Feb 6 | **0/5** | Deleted files still referenced by imports | ✅ — author self-closed in 7h45m |
| #1006 | Feb 7 | 4/5 | PowerShell quoting + stale docstrings | ✅ (already fixed); maintainer rejected patches |
| #1138 | Feb 16 | 2/5 | Missing empty-response guard + cross-session concurrency bug | ✅ |
| #1154 | Feb 18 | — | Orphaned Chroma collection routing bug | ✅ (critical architectural catch) |
| #1176 | Feb 18 | — | 3 findings, 2 FP (`finally`-guarded); missed SQL injection + array desync | ⚠️ most memorable miss |
| #2059 | Apr 18 | — | 6 findings across settings/auth/env | ✅ |
| #2060 | Apr 18 | — | Migration breadth + prepared-statement loss + column-count mismatch | ✅ |
| #2072 | Apr 19 | — | P1: circuit-breaker counter incrementing per observation | ✅ (spawned whole fix session) |
| #2073 | Apr 19 | **5/5** | 3 P2 items, safe-to-merge verdict | ✅ but CodeRabbit blocked with 15 issues |
| #2078 | Apr 19 | — | 4 P1/P2 items | ✅ (CodeRabbit flagged 15 critical/major) |
| #2079 | Apr 19 | — | P2: FTS5 DDL probe runs every query | ✅ (fixed in commit `2472cf36`) |
---
*Compiled from the claude-mem persistent-memory system: 22 Greptile-specific observations, 102,154 tokens of context, 8 weeks of production use on an active open-source repo. Happy to share the raw observation IDs if your team wants to dig into any specific finding.*

221
journey-into-claude-mem-greptile.md Normal file
View File

@@ -0,0 +1,221 @@
# Greptile in claude-mem: A Timeline Analysis
## 1. Executive Summary
Greptile entered the claude-mem project on Feb 1, 2026 at PR #856, became the project's dominant automated reviewer through Feb 18, 2026, and then faded from primary duty once CodeRabbit Pro was re-adopted in March 2026. Across 22 Greptile-specific observations totaling 102,154 discovery tokens, it played two overlapping roles: a CI status check (`Greptile Review`) that gated merges and a formal PR reviewer that emitted confidence-scored (0/5 – 5/5), severity-tagged (P1/P2) inline comments. It filled the 79-day gap between CodeRabbit's Phase 1 exit (PR #67, Nov 7, 2025) and CodeRabbit Pro's arrival in March 2026, and by the March 23, 2026 "bake-off" (#61317) the project concluded: **keep CodeRabbit Pro as primary, triage with Greptile, fix the Claude-bot redundancy problem** (#61319). Greptile remained active into April 2026 (PRs #2059, #2060, #2072, #2073, #2078, #2079) alongside CodeRabbit Pro, with the final captured interaction being a P2 FTS5 fix on PR #2079 on Apr 19, 2026 (#70958).
## 2. Timeline of Greptile Usage
### First appearance — PR #856 (Feb 1, 2026)
The earliest Greptile touchpoint in the timeline is observation **#42883 (2026-02-01 07:55)** — *PR #856 Review Comments Analysis*. PR #856 addressed zombie observer subprocess accumulation by adding a 3-minute idle timeout to `SessionQueueProcessor.waitForMessage()`. Four reviews were recorded: three from `claude` and one from `greptile-apps`. Greptile's review aligned with Claude's: it praised the root-cause approach but flagged a race condition in `lastActivityTime` reset logic and the absence of test coverage. The PR description was later updated (#42920) to formally document "Review Feedback Addressed" citing both `claude` and `greptile-apps`. **#42972 (2026-02-01)** then noted that PR #856 was MERGEABLE but BLOCKED, with only a single Greptile comment from Jan 30, 2026 — indicating Greptile activity actually began approximately Jan 26–30, 2026 (the exact `PR #809` start date is corroborated by the later reconstruction in #61307).
### Trial period — Feb 5–6, 2026
During the PR-Triage-10 sweep on Feb 5–6, 2026, Greptile functioned as the sole CI check on multiple documentation-heavy PRs:
- **#42936 (2026-02-05)** — *PR #863 all CI checks passing with additional Greptile review*. PR #863 (Ragtime email investigation refactor) was "unique among the five in having the Greptile Review check enabled and passing" in 2m45s.
- **#43845 (2026-02-06 07:25)** — *PR #953 Passes CI Checks with Greptile Review Success*. Single automated check, passed in 2m33s on a README formatting PR.
- **#43913 (2026-02-06 07:34)** — *PR #894 Passes Greptile Review CI Check*. Single check, passed in 1m59s on docs URL updates across 29 language files.
- **#43957 (2026-02-06 07:45)** — *PR #882 Greptile Review Feedback Identifies Placement and Documentation Issues*. Greptile flagged that Windows-specific setup notes belonged in `docs/public/development.mdx` rather than the README footer, and that the "instructions above" reference was vague.
### Trial expiry — PR #863 (Feb 1, 2026)
**#61302 (2026-03-23 21:59)** — *Greptile Free Trial Expired Mid-Project at PR #863 — Renewed After Gap* — is the definitive statement on the activation model. The direct GitHub review API revealed Greptile's first review on PR #863 was a trial-expiry notice. The project then renewed access and Greptile posted a substantive second review (the 4/5 confidence Ragtime review referenced in #44166). This is the only explicit evidence in the timeline of a payment/activation event, and it implies the "renewal" was either a trial extension or a paid conversion.
### Peak density — Feb 5–18, 2026
Between Feb 5 and Feb 18, 2026, Greptile reviewed every non-trivial PR that surfaced in the timeline:
- **PR #917** (CORS security fix) — **#61303** *Four Technically Precise Security-Adjacent Comments on CORS Fix* (confidence 4/5)
- **PR #879** (daemon child-process cleanup) — **#43414** identified the approach as sound with minor ps-parsing and registry-verification concerns
- **PR #968** (MemU backend swap) — **#44156** issued 0/5 confidence; the author self-closed the PR (**#44157**) 7h45m after Greptile's review
- **PR #1006** (Windows platform improvements) — **#46231, #46237, #46268, #46607** — Greptile flagged PowerShell quoting and stale docstrings; fixes were already in place when re-checked
- **PR #1138** (four post-merge bug bundle) — **#50224, #50225, #50244** — Greptile confidence 2/5, caught the missing empty-response guard on the summary path and the global `resetStaleProcessingMessages(0)` concurrency bug
- **PR #1154** (Chroma backfill) — **#51128, #51158, #51159, #51160, #51161** — Greptile caught the critical collection routing bug where `backfillAllProjects` wrote to orphaned per-project collections
- **PR #1176** (ChromaMcpManager migration) — **#51619, #51620, #51621, #51633, #51634, #51636** — three findings, two of which turned out to be false positives (already guarded by `finally` blocks)
### Zero-approval environment — Feb 23–24, 2026
**#55594 (2026-02-24 01:46)** — *Open PR Inventory: 10+ PRs Across Windows Fixes, New Providers, and Core Features* — "Only Greptile bot has reviewed most PRs — no human maintainer reviews are present." **#55597 (2026-02-24 01:47)** extended this: across 34 open PRs, "No PR has been approved or had changes requested by a human reviewer. The only reviews recorded are from the Greptile AI code review bot." This is the structural peak of Greptile's influence — it was effectively the sole gatekeeper.
### Comparative analysis — Mar 23, 2026
The March 23, 2026 investigation (**#61295 – #61328**, detailed in §5) produced two reports: `reports/automated-code-review-comparison.md` (**#61316, #61321**) and `reports/journey-into-automated-code-review.md` (**#61326, #61327, #61328**). Both were saved to the repository, and the decision artifact **#61319** *Recommendation: Keep CodeRabbit Pro Primary, Fix Claude Bot Redundancy, Triage with Greptile* codified Greptile's role going forward.
### Continued use — April 2026
Greptile did not disappear after the bake-off. April 2026 observations show it running in parallel with CodeRabbit Pro:
- **#70071, #70074 (2026-04-18)** — PR #2059 got both CodeRabbit and Greptile reviews with 6 findings across 4 files
- **#70075, #70078 (2026-04-18)** — PR #2060: CodeRabbit one nitpick, Greptile four database-migration findings
- **S6935, S6937 (2026-04-19)** — "Fix Greptile P1 circuit breaker bug in ResponseProcessor.ts" session
- **#70220 – #70279 (2026-04-19)** — PR #2073: Greptile posted 3 P2 issues with **5/5 confidence** safe-to-merge verdict; CodeRabbit blocked with CHANGES_REQUESTED. The two tools were in active tension.
- **#70727, #70740 (2026-04-20)** — PR #2078: Greptile flagged 4 P1/P2 items; CodeRabbit flagged 15 critical/major items
- **#70953, #70958 (2026-04-20 05:05)** — PR #2079: reply posted to Greptile confirming the FTS5 availability caching fix (commit `2472cf36`). CodeRabbit had no inline comments on #2079 (**#70991**).
### Final interactions
The most recent Greptile-related observations are **#71305, #71306 (2026-04-20 22:47)** — both tied to this very report's data-gathering pass, not production engineering.
## 3. What Greptile Caught
| PR | Date | Finding | Valid? | Evidence |
|----|------|---------|--------|----------|
| #856 | Feb 1 2026 | `lastActivityTime` race condition reset before processing | Valid | #42883; code fix in commit 5fa218ce (#42920) |
| #863 | Feb 1 2026 | Template literal spacing on ragtime script line 196 (confidence 4/5) | Valid but trivial | #44166 |
| #882 | Feb 6 2026 | Windows setup notes should live in `docs/public/development.mdx`, not README | Valid (structural) | #43957; PR marked closed in triage (#43964) |
| #879 | Feb 6 2026 | ps-output parsing robustness; no registry verification before killing zombies | Minor | #43414 |
| #917 | Feb 5 2026 | Portless localhost CORS check; missing IPv6 support; tests exercise duplicated logic rather than real middleware; test coverage gap (confidence 4/5) | Valid — highest-quality review | #43209, #61303 |
| #968 | Feb 6 2026 | PR deleted `SessionStore.js`, `SessionSearch.js`, `ChromaSync.js` while imports still reference them (confidence 0/5) | Valid — decisive | #44156, #44157 (author self-closed) |
| #1006 | Feb 7 2026 | PowerShell single-quote injection in `spawnDaemon` at `ProcessManager.ts`; stale "No-op on Windows" docstrings in `ChromaSync.ts` lines 530-534, 591-595, 883-887 | Valid, already fixed by time of recheck | #46231, #46237 |
| #1138 | Feb 16 2026 | Missing empty-Gemini-response guard on summary path line 291 (#50224); global `resetStaleProcessingMessages(0)` at `worker-service.ts:615` causing cross-session duplicate processing (#50225) | Valid — led to session-scoped fix (#50246, #50247) | #50224, #50225, #50244 |
| #1154 | Feb 18 2026 | `backfillAllProjects` writes to orphaned per-project Chroma collections (`cm__YC_Stuff`) instead of shared `cm__claude-mem` collection read by SearchManager | Valid — critical architectural bug | #51128, #51158; fix in #51133, #51134 |
| #1176 | Feb 18 2026 | (a) Connection-lock leak in `ensureConnected()`, (b) timer leak on successful connection, (c) race in `reset()` | (a)/(b) **false positives** — already guarded by `finally` blocks (#51636, #61324); (c) valid | #51619, #51620, #51633 |
| #2059 | Apr 18 2026 | 6 findings across settings/auth/env — Bedrock credential check, AUTH_TOKEN leakage, whitespace-only API key bypass, log spam | Valid | #70074 |
| #2060 | Apr 18 2026 | Migration 26 rebuild-logic breadth, 30-min `PendingMessageStore` recreation losing prepared-statement cache, INSERT column-count mismatch when source has extra columns | Valid | #70078 |
| #2072 | Apr 19 2026 | P1: circuit-breaker counter increments on every observation response in `ResponseProcessor.ts` | Valid — drove whole session S6935/S6937 | #70181, #70182, #70183, #70184 |
| #2073 | Apr 19 2026 | (a) Unreachable `agentId`/`agentType` spreads in `summarize.ts`, (b) missing `agent_id` index in migration 010, (c) stale subagent identity in `SDKAgent.ts`/`GeminiAgent.ts`/`OpenRouterAgent.ts` (5/5 safe-to-merge) | Valid — all three actioned | #70221, #70222, #70225 |
| #2078 | Apr 19 2026 | P1: `SyntaxError` in `logger.ts:159-161` noisily triggered on Bash input; P1: `logger.debug` in tight PID loop in `ProcessManager.ts:329-333`; P1: new `error as Error` unsafe casts in `ChromaSync.ts:568`, `GeminiAgent.ts:377`, `OpenRouterAgent.ts:348`; P2: non-Error FTS failure drops details in `runner.ts:426` | Valid | #70727 |
| #2079 | Apr 19 2026 | P2: `isFts5Available()` DDL probe runs on every query | Valid — fixed with cached `_fts5Available` in commit `2472cf36` | #70953, #70956, #70957, #70958 |
**False positives documented:** PR #1176 connection lock leak and timer leak (#51636, #61324). The comparative analysis correction in #61324 revised the narrative: "Greptile had two false positives on PR #1176, while the Claude bot caught the real critical bugs that Greptile missed" (distances/metadatas desync and SQL injection via `ensureBackfilled`). #61319 also noted PR #1006 scored 4/5 (Greptile's highest confidence) "yet the maintainer rejected the suggested code changes" — distinguishing architectural soundness from fix-suggestion correctness.
## 4. Comparison to Other Code Review Tools
| Dimension | CodeRabbit (Phase 1 Free) | CodeRabbit (Phase 2 Pro) | Greptile | Claude bot (`claude-code-review.yml`) | GitHub Copilot |
|-----------|---------------------------|--------------------------|----------|----------------------------------------|----------------|
| Active period | Sep 10 – Nov 7, 2025 (PRs #2 – #67) | Mar 19, 2026 onward | Jan 26 – Feb 18, 2026 (primary); Apr 2026 (secondary) | Dec 2025 – Apr 2026 (continuous) | Dec 15, 2025 onward |
| Cost model | Free tier | Paid (Pro) | Free trial → renewed | Operated via GitHub Actions, consumes Claude quota | Included with GitHub |
| Activation evidence | PR #2 onward (#61309) | #63247 (PR #1592) | PR #863 trial-expiry notice (#61302) | `claude-code-review.yml` workflow | PR #332 (#61315) |
| Review format | Mermaid diagrams + severity-rated inline + "poet summaries" (#61312, #61322) | Same + multi-round convergence + cross-PR memory | Confidence 0/5 – 5/5 per PR + P1/P2 inline (#61301) | Full text review, CLAUDE.md-aware | "Pull request overview" table + inline |
| Severity / confidence system | 🔴 Critical / 🟠 Major / 🟡 Minor (#61308) | Same + CHANGES_REQUESTED status | Confidence 0–5 + P1/P2 priority | Unstructured narrative | COMMENTED state only |
| Blocking behavior | COMMENTED-only (no CHANGES_REQUESTED recorded) | **Uses CHANGES_REQUESTED** (#70739, #70759, #70765) | COMMENTED-only (#61301) | COMMENTED-only | COMMENTED-only (#61315) |
| False positive rate | ~88–100% accuracy (one FP: collection-name mismatch PR #41, #61324) | 32/32 valid, 0% FP (S6304) | Low on source, 2 FPs on PR #1176 (#61324) | ~95–97% accuracy (S6305) | ~8% on source-only; 60–65% on PRs with minified bundles (#61314) |
| Signal-to-noise | High | Very high | High on TypeScript source; shallow on complex PRs | Low — 5–6 full re-evaluations per PR from `synchronize` trigger (#61317, #61319) | Catastrophic on bundle-inclusive PRs (#61314); good otherwise |
| Known wins | NULL column bug, missing FTS table, `which` vs `where` Windows gap, race in `pm2 start` (#61308) | Cross-PR memory citing PR #1422 on #1461; architectural catches on PR #1418 (#61318); 7→6→1→0 convergence on #1455 | CORS IPv6+portless localhost (#917); Chroma collection routing (#1154); FTS5 caching (#2079); PR #968 kill | ChromaSync distances/metadatas desync + SQL injection on PR #1176 (#51636); command injection on PR #1138 (#50244); only tool that reads CLAUDE.md | Atomicity/transaction bug on PR #332 source files (#61313) |
| Known misses | Missed bugs that later emerged (4-month ChromaSync lineage, #61325) | None captured — 100% accuracy on sampled PRs | Missed PR #1176 desync+SQL injection caught by Claude bot (#61324); only 2–4 comments on complex PRs (#61317) | Redundancy buries findings — same SQL snippet appeared verbatim 5 times on PR #522 (#61317) | Flags every single-letter minified variable as "unused" (#61314) |
| Integration | CI via PR comments; live shell script verification (#61311, #61322) | Same + interactive 32s reply on contributor comments (#61318) | `Greptile Review` CI check + formal PR reviewer (#42936); "Prompt To Fix With AI" button (#61303) | GitHub Actions workflow `claude-code-review.yml` triggered on `synchronize` | GitHub native |
### Narrative per tool
**CodeRabbit (Phase 1 Free, Sep–Nov 2025).** The project's original automated reviewer, active from PR #2 (#61309) through PR #67. Produced the richest feature set: full PR walkthroughs, Mermaid sequence diagrams, effort estimation, related-PR detection, bot-skip behavior, and competitor detection (it noticed Copilot on PR #26, #61322). The developer formalized its output by creating `docs/coderabbit-PR-41.md` (#61312) — the first and for a long time only case of review findings being tracked as a standalone document. After PR #67 it vanished for 79 days. No Greptile-PR overlap exists (#61306, #61307): **Greptile and CodeRabbit never reviewed the same PR in Phase 1**.
**CodeRabbit (Phase 2 Pro, Mar 2026+).** Returned via a paid Pro upgrade on the same account (#61312 — the Phase-2 retroactive failure on PR #41 proves account continuity). Demonstrated novel capabilities unavailable in Phase 1: cross-PR memory citing PR #1422 during the PR #1461 review (#61318), 4-round convergence on PR #1455 (7→6→1→0 findings), and 32-second interactive replies to contributor comments. By April 2026 it was the only tool using `CHANGES_REQUESTED` to block merges (#70739, #70759 on PR #2078, #70268 on PR #2073).
**Greptile (Jan–Feb 2026 primary, Apr 2026 secondary).** See §2 and §3. Greptile's distinctive feature is the 0/5 – 5/5 per-PR confidence score, which proved to be a **reliable triage signal for merge decisions but not a guarantee of implementation quality** (#61319). Its "Prompt To Fix With AI" button (#61303) converted findings into AI-actionable remediation prompts. Comment depth was limited to roughly 2–4 items per PR (#61317), which caused it to miss secondary bugs on complex PRs like #1176.
**Claude bot.** Operated continuously via `claude-code-review.yml` in GitHub Actions. **Unique strength: the only reviewer that reads CLAUDE.md for project-specific conventions** (#61317). **Unique flaw: the `synchronize` trigger caused 5–6 full re-evaluations per PR with no awareness of prior reviews.** The same SQL snippet appeared verbatim 5 times on PR #522 (#61317). #44156 (PR #968 integration-failure catch) and #51636 (PR #1176 distances/metadatas + SQL injection catches) are its standout wins.
**GitHub Copilot.** Entered Dec 15, 2025 at PR #332 (#61315). Performs well on source-only diffs (~92% accuracy with real bugs like the missing-transaction atomicity bug on PR #332, #61313) but collapses into 60–65% false-positive rate on PRs containing minified bundles in `plugin/scripts/*.js` because it flags every single-letter minified variable as unused (#61314). A `.copilotignore` or `.gitattributes` linguist-generated marking would fix the problem (#61319). Only recorded on PR #1006 as a co-reviewer with Greptile (#46266, #46267, #46268).
## 5. The March 23, 2026 Bake-off
The bake-off was a 24-hour investigation kicked off by session **S6296** ("CodeRabbit & Greptile Comprehensive PR Review Quality Report"). Five parallel subagents investigated CodeRabbit Phase 1, CodeRabbit Phase 2 (Pro), Greptile, claude[bot], and Copilot — each running direct GitHub API queries across 15+ PRs (#61313). The synthesis produced two deliverables: `reports/automated-code-review-comparison.md` (#61316, #61321) and `reports/journey-into-automated-code-review.md` (#61326, #61327, #61328). The process consumed ~316k tokens and 197 tool calls (#61316).
### What was compared
**#61317 — *Definitive Comparative Analysis of 4 Automated PR Review Tools on claude-mem***. The four tools, their eras, and per-tool subagent findings were cross-indexed against claude-mem's own observation database. Key structural finding: CodeRabbit and Greptile operated in **completely non-overlapping eras** (#61307, #61322). Any quality comparison is between different project maturity levels (pre-v1 vs v9+), not a controlled A/B test.
### Conclusions
- **CodeRabbit Pro** emerged as the clear leader: 100% accuracy, multi-round convergence, cross-PR memory, 32s interactive reply (#61317, #61318).
- **Greptile** excelled at triage: the 0/5 on PR #968 was "the most decisive automated review action in the project's history" — the author self-closed within hours (#61317). But the 2–4 comment depth limit caused secondary-bug misses on complex PRs.
- **Claude bot** had the highest redundancy cost — 5–6 full re-evaluations per PR — but was uniquely aware of CLAUDE.md (#61317).
- **Copilot** needed a `.gitattributes linguist-generated` fix to stop reviewing minified bundles (#61314, #61319).
### Recommendation
**#61319 — *Recommendation: Keep CodeRabbit Pro Primary, Fix Claude Bot Redundancy, Triage with Greptile*.** The five-point optimization strategy:
1. **CodeRabbit Pro as primary reviewer** — highest accuracy and convergence.
2. **Fix the Claude bot `synchronize` trigger** — the single most impactful change; converts 5–6x content amplification into a one-time review, surfacing genuine findings like `continuesExecution` logic inversion that are currently buried.
3. **Triage with Greptile's confidence score** — reliable merge signal, not an implementation-quality oracle.
4. **Apply `.gitattributes linguist-generated` to Copilot** — one-line fix raises signal-to-noise from ~49% to ~90%+.
5. **Caveat around Greptile** — PR #1006 scored 4/5 yet the maintainer rejected the changes; architectural soundness ≠ correct fix suggestions.
### Key corrections surfaced during the bake-off
- **#61320** — gap confirmed: no direct observations exist for CodeRabbit Pro PRs #1418, #1455, #1461 in claude-mem (external capability inferred from GitHub).
- **#61322** — Mermaid diagrams/effort estimation/related-PR detection/poems were all in the **Free tier** (PR #41), not Pro innovations as the original draft implied.
- **#61324** — CodeRabbit Phase 1 had at least **one** false positive (PR #41 collection-name mismatch), revising accuracy to 88–100%. Also: Greptile's two PR #1176 findings were false positives; the Claude bot caught the real critical bugs.
- **#61325** — `src/services/sync/ChromaSync.ts` received confirmed bug catches from **all three** tools across 4 months, making it the single strongest argument for continuous multi-tool review.
## 6. PR-Level Evidence
**PR #856** (Feb 1 2026) — zombie observer cleanup. Four reviews; Greptile aligned with Claude on the race condition and test-coverage gap (#42883). Actioned in commit `5fa218ce` (#42920).
**PR #863** (Feb 1 2026) — Ragtime email investigation. First Greptile review was a trial-expiry notice (#61302); post-renewal Greptile gave 4/5 confidence with a minor template-literal spacing flag on line 196 (#44166). Merged with that caveat accepted (#44170).
**PR #879** (Feb 6 2026) — daemon child-process cleanup. Greptile rated approach sound; flagged ps-output parsing and registry verification (#43414). Test evidence in the PR showed memory drop from 4.3GB → 2.3GB.
**PR #882** (Feb 6 2026) — Windows README patch. Greptile's placement/structure feedback (#43957) caused the maintainer to mark it closed in PR-Triage-10.md (#43964) rather than merge — a direct operational outcome from a purely non-functional Greptile review.
**PR #917** (Feb 5 2026) — CORS security fix. Greptile's highest-quality review (#43209, #61303): four findings including IPv6 support, portless localhost, tests-of-duplicated-logic, and coverage gap. All four categorized as technically precise.
**PR #968** (Feb 6 2026) — MemU backend swap. Greptile 0/5 confidence (#44156); author self-closed 7h45m later (#44157). **The single most decisive Greptile action captured in the timeline.**
**PR #1006** (Feb 7 2026) — Windows platform improvements. Greptile flagged PowerShell quoting (`ProcessManager.spawnDaemon`) and stale docstrings in `ChromaSync.ts` lines 530-534, 591-595, 883-887 (#46231). Recheck (#46237) confirmed both already fixed — paths passed via `$env:_DAEMON_EXEC`/`$env:_DAEMON_SCRIPT` and five "No-op on Windows" docstrings removed. Later (#46268) Copilot reviewed a newer commit `e0391f2` and added 4 comments; both remained COMMENTED, neither approved or blocked. This is the only PR where Copilot and Greptile both formally reviewed.
**PR #1138** (Feb 16 2026) — four post-merge fixes. Greptile confidence 2/5 (#50244). Greptile caught the empty-response guard missing at line 291 and the global `resetStaleProcessingMessages(0)` session-scope bug at `worker-service.ts:615` (#50225). Claude bot escalated by identifying command-injection in `sync-marketplace.cjs:41` via gitignore-pattern shell interpolation. Both reviews converged on merge-blocking verdicts. Fix landed as session-scoped variant (#50246, #50247).
**PR #1154** (Feb 18 2026) — Chroma backfill fix. Greptile identified the **orphaned-collection routing bug** (#51128): `backfillAllProjects` wrote to `cm__YC_Stuff` etc. but SearchManager only queries the shared `cm__claude-mem` collection. Claude bot converged on the same issue (#51158). Fix iteratively landed via `sync.project` mutation pattern (#51133, #51134, #51159, #51160, #51161). Also caught trailing-non-alphanumeric sanitization gap.
**PR #1176** (Feb 18 2026) — ChromaMcpManager migration. Greptile flagged three bugs: `this.connecting` stale rejected promise, 30s timer leak on successful connect, race in `reset()` (#51619, #51620, #51633). **Two of three were false positives** (#51636, #61324) — already correctly handled by existing `finally` blocks. Claude bot, reviewing the same PR, caught **distances/metadatas desync in `queryChroma`** and **SQL injection via unvalidated ID interpolation in `ensureBackfilled`** — the real critical bugs Greptile missed. This PR is the single strongest case against over-trusting Greptile.
**PR #2052 – #2079** (Apr 2026). Captured but not exhaustively analyzed — #69039 notes "Five CodeRabbit Review Comments Identified on PR #2052" without a paired Greptile mention, consistent with CodeRabbit Pro being primary. PR #2072 spawned an explicit Greptile-P1 session (S6935, S6937). PR #2073 produced the sharpest divergence: Greptile 5/5 safe-to-merge vs CodeRabbit CHANGES_REQUESTED with 15 issues (#70220, #70225, #70268). PR #2078 similar: Greptile 4 P1/P2 items vs CodeRabbit 15 critical/major (#70727, #70740). PR #2079: Greptile P2 on FTS5 probe, CodeRabbit empty (#70953, #70991).
## 7. Economic / Operational Lessons
**No human gatekeepers.** #55597 is the starkest statement: 34 open PRs, zero human approvals or change-requests, only Greptile. Automated review became load-bearing for merge decisions, not advisory.
**Fast merges defeat slow reviews.** #61310 on PR #58: CodeRabbit returned 12 valid findings but the PR merged 6 minutes after creation. The issue is "not a quality failure but a timing architecture problem that accuracy improvements cannot solve" (#61328). Relevant for Greptile too — its 2–3 minute CI check means some small PRs merge before review completes.
**Redundancy costs.** The Claude bot's `synchronize`-triggered redundancy (5–6 full re-evaluations per PR, #61317) made genuine findings invisible. The recommendation in #61319 explicitly prioritizes fixing this over any capability addition.
**Trust decay.** #44170 on PR #863 explicitly overrode Claude bot's test-coverage recommendation because the task instruction said "review and merge if ready" not "add tests first." Reviews started being treated as suggestions to weigh, not gates to pass. #61319's caveat on Greptile (4/5 confidence + rejected changes on PR #1006) is the same trust-calibration pattern.
**Continuous coverage pays off.** #61325 — the ChromaSync.ts 4-month bug discovery chain — showed that different tools caught different bug classes at different codebase maturity levels. CodeRabbit caught the Nov 2025 data-consistency bugs. Greptile caught the Feb 2026 architectural routing bug. Claude bot caught the Feb 2026 array-alignment and SQL-injection bugs. None of those could have been caught at a single point in time.
**No decisions to shut Greptile off.** The timeline contains no observation explicitly disabling Greptile. Instead, #61319 subordinates it: "triage with Greptile." April 2026 observations show it still running alongside CodeRabbit Pro.
## 8. Token Economics for Greptile-Related Work
Database queries (Apr 20 2026):
**All Greptile-related observations**: 22 rows, first observation Feb 1 2026 (#42883), last Apr 20 2026 (#71306). Total discovery tokens: **102,154**.
**Tool-by-tool breakdown**:
| Tool | Observations | Total discovery tokens |
|------|-------------:|-----------------------:|
| CodeRabbit | 25 | 178,091 |
| Greptile | 22 | 102,154 |
| Claude (bot/review) | 6 | 17,457 |
CodeRabbit's memory footprint is larger than Greptile's in absolute terms, but per-observation the two tools track closely (CodeRabbit 7,124 t/obs, Greptile 4,643 t/obs). #61297 — *CodeRabbit Has Minimal Direct Memory Footprint vs Greptile's 61 Results* — is contradicted by the final count once the full search was run: CodeRabbit has *more* memory mass (178k vs 102k tokens), but Greptile has more **directly-named PR observations** because it is a named reviewer while CodeRabbit Phase 1's inline comments were not labeled in claude-mem's own observation extraction pipeline.
**Top 10 most expensive review-related observations**:
| ID | Date | Title | Tokens |
|----|------|-------|-------:|
| #61327 | Mar 23 | Narrative Report Ready to Write — Planning Phase Complete | 110,012 |
| #49875 | Feb 16 | PR #1125 Implementation Review — Parallel Fetch and Default Settings Changes | 87,527 |
| #19456 | Dec 3 | Modal Header and Preview Layout Restructure | 71,246 |
| #38027 | Jan 6 | Posted PR review response addressing all four items | 65,578 |
| #38008 | Jan 6 | Implementation plan created for PR #556 final review items | 53,677 |
| #38006 | Jan 6 | Scope PR review fixes to items 1-3, defer race condition discussion | 48,448 |
| #58686 | Mar 13 | PR Wizard Session Reviewing claude-mem Playbook Progress | 44,016 |
| #17156 | Nov 29 | Successfully extracted Opus 4.5 thinking transcript to reviewable file | 35,332 |
| #19408 | Dec 3 | Art Deco Preview Column Redesign with Geometric Patterns | 30,274 |
| #11098 | Nov 18 | DUH Naming Convention Documentation Structure Reviewed | 27,390 |
The single most expensive review-related observation — **#61327 at 110,012 tokens** — is the planning phase for the narrative report, i.e. the meta-investigation of Greptile itself. The most expensive **Greptile-specific** observations are **#61296 (16,941 t)** *Full PR History Dataset Retrieved*, **#61301 (16,316 t)** *Greptile Review Quality Patterns Documented*, and **#61305 (18,447 t)** *Full Automated PR Review Ecosystem Timeline Reconstructed*.
## 9. Verdict
Greptile did exactly what the project needed during the 79-day gap between CodeRabbit's two eras, and it stayed on as a useful triage voice afterward. The confidence score (0/5–5/5) is the single most valuable artifact any of the four tools produces — it converted PR #968 from an open architectural rewrite into a self-closed mistake in under 8 hours (#44156, #44157), and it gave the maintainer a merge-safety prior on every subsequent PR. Its ceiling was depth: at 2–4 comments per PR (#61317) it missed the secondary bugs on complex PRs like #1176 that Claude bot caught, and it posted false positives twice on that same PR because it didn't understand the existing `finally`-block guards (#51636, #61324).
Against CodeRabbit, Greptile is simpler, less capable, and less expensive per PR, but operates the same way: COMMENTED-only, non-blocking, advisory. **CodeRabbit Pro won the comparative analysis (#61317) on accuracy, multi-round convergence, cross-PR memory, and interactive response.** Against Claude bot, Greptile is cleaner (no synchronize-trigger redundancy) but lacks the CLAUDE.md context awareness. Against Copilot, Greptile is cheaper in noise — no minified-bundle false-positive floods.
The timeline shows the conclusion stayed consistent from March 23, 2026 (#61319) through April 20, 2026 (the present moment): **CodeRabbit Pro is primary, Greptile is supplementary.** This was neither a rejection nor an endorsement — it was a correct classification. The claude-mem project needed both: Greptile's fast, confidence-scored merge-safety signal and CodeRabbit's deeper iterative convergence on the findings that actually matter.

38
plugin/scripts/context-generator.cjs
View File

File diff suppressed because one or more lines are too long

67
plugin/scripts/mcp-server.cjs
View File

File diff suppressed because one or more lines are too long

477
plugin/scripts/worker-service.cjs
View File

File diff suppressed because one or more lines are too long

22
plugin/ui/viewer-bundle.js
View File

File diff suppressed because one or more lines are too long

2
scripts/build-hooks.js
View File

@@ -24,7 +24,7 @@ const MCP_SERVER = {
const CONTEXT_GENERATOR = {
name: 'context-generator',
source: 'src/services/context-generator.ts'
source: 'src/services/context/index.ts'
};
/**

2
scripts/debug-transcript-structure.ts
View File

@@ -4,7 +4,7 @@
* Examines the first few entries to understand the conversation flow
*/
import { TranscriptParser } from '../src/utils/transcript-parser.js';
import { TranscriptParser } from './transcript-parser.js';
const transcriptPath = process.argv[2];

2
scripts/dump-transcript-readable.ts
View File

@@ -4,7 +4,7 @@
* Shows exactly what's in the transcript, chronologically
*/
import { TranscriptParser } from '../src/utils/transcript-parser.js';
import { TranscriptParser } from './transcript-parser.js';
import { writeFileSync } from 'fs';
const transcriptPath = process.argv[2];

2
scripts/extract-rich-context-examples.ts
View File

@@ -4,7 +4,7 @@
* Shows what data we have available for memory worker using TranscriptParser API
*/
import { TranscriptParser } from '../src/utils/transcript-parser.js';
import { TranscriptParser } from './transcript-parser.js';
import { writeFileSync } from 'fs';
import type { AssistantTranscriptEntry, UserTranscriptEntry } from '../src/types/transcript.js';

2
scripts/format-transcript-context.ts
View File

@@ -6,7 +6,7 @@
* that could be used for improved observation generation.
*/
import { TranscriptParser } from '../src/utils/transcript-parser.js';
import { TranscriptParser } from './transcript-parser.js';
import { writeFileSync } from 'fs';
import { basename } from 'path';

2
scripts/test-transcript-parser.ts
View File

@@ -6,7 +6,7 @@
* Usage: npx tsx scripts/test-transcript-parser.ts <path-to-transcript.jsonl>
*/
import { TranscriptParser } from '../src/utils/transcript-parser.js';
import { TranscriptParser } from './transcript-parser.js';
import { existsSync } from 'fs';
import { resolve } from 'path';

0
src/utils/transcript-parser.ts → scripts/transcript-parser.ts
View File

2
scripts/transcript-to-markdown.ts
View File

@@ -4,7 +4,7 @@
* Shows ALL available context data from a Claude Code transcript
*/
import { TranscriptParser } from '../src/utils/transcript-parser.js';
import { TranscriptParser } from './transcript-parser.js';
import type { UserTranscriptEntry, AssistantTranscriptEntry, ToolResultContent } from '../types/transcript.js';
import { writeFileSync } from 'fs';
import { basename } from 'path';

4
src/cli/handlers/index.ts
View File

@@ -11,7 +11,6 @@ import { contextHandler } from './context.js';
import { sessionInitHandler } from './session-init.js';
import { observationHandler } from './observation.js';
import { summarizeHandler } from './summarize.js';
import { userMessageHandler } from './user-message.js';
import { fileEditHandler } from './file-edit.js';
import { fileContextHandler } from './file-context.js';
import { sessionCompleteHandler } from './session-complete.js';
@@ -22,7 +21,6 @@ export type EventType =
| 'observation' // PostToolUse - save observation
| 'summarize' // Stop - generate summary (phase 1)
| 'session-complete' // Stop - complete session (phase 2) - fixes #842
| 'user-message' // SessionStart (parallel) - display to user
| 'file-edit' // Cursor afterFileEdit
| 'file-context'; // PreToolUse - inject file observation history
@@ -32,7 +30,6 @@ const handlers: Record<EventType, EventHandler> = {
'observation': observationHandler,
'summarize': summarizeHandler,
'session-complete': sessionCompleteHandler,
'user-message': userMessageHandler,
'file-edit': fileEditHandler,
'file-context': fileContextHandler
};
@@ -65,7 +62,6 @@ export { contextHandler } from './context.js';
export { sessionInitHandler } from './session-init.js';
export { observationHandler } from './observation.js';
export { summarizeHandler } from './summarize.js';
export { userMessageHandler } from './user-message.js';
export { fileEditHandler } from './file-edit.js';
export { fileContextHandler } from './file-context.js';
export { sessionCompleteHandler } from './session-complete.js';

53
src/cli/handlers/user-message.ts
View File

@@ -1,53 +0,0 @@
/**
* User Message Handler - SessionStart (parallel)
*
* Displays context info to user via stderr.
* Uses exit code 0 (SUCCESS) - stderr is not shown to Claude with exit 0.
*/
import { basename } from 'path';
import type { EventHandler, NormalizedHookInput, HookResult } from '../types.js';
import { ensureWorkerRunning, getWorkerPort, workerHttpRequest } from '../../shared/worker-utils.js';
import { HOOK_EXIT_CODES } from '../../shared/hook-constants.js';
async function fetchAndDisplayContext(project: string, colorsParam: string, port: number): Promise<void> {
const response = await workerHttpRequest(
`/api/context/inject?project=${encodeURIComponent(project)}${colorsParam}`
);
if (!response.ok) {
return;
}
const output = await response.text();
process.stderr.write(
"\n\n" + String.fromCodePoint(0x1F4DD) + " Claude-Mem Context Loaded\n\n" +
output +
"\n\n" + String.fromCodePoint(0x1F4A1) + " Wrap any message with <private> ... </private> to prevent storing sensitive information.\n" +
"\n" + String.fromCodePoint(0x1F4AC) + " Community https://discord.gg/J4wttp9vDu" +
`\n` + String.fromCodePoint(0x1F4FA) + ` Watch live in browser http://localhost:${port}/\n`
);
}
export const userMessageHandler: EventHandler = {
async execute(input: NormalizedHookInput): Promise<HookResult> {
// Ensure worker is running
const workerReady = await ensureWorkerRunning();
if (!workerReady) {
// Worker not available — skip user message gracefully
return { exitCode: HOOK_EXIT_CODES.SUCCESS };
}
const port = getWorkerPort();
const project = basename(input.cwd ?? process.cwd());
const colorsParam = input.platform === 'claude-code' ? '&colors=true' : '';
try {
await fetchAndDisplayContext(project, colorsParam, port);
} catch {
// Worker unreachable — skip user message gracefully
}
return { exitCode: HOOK_EXIT_CODES.SUCCESS };
}
};

11
src/hooks/hook-response.ts
View File

@@ -1,11 +0,0 @@
/**
* Standard hook response for all hooks.
* Tells Claude Code to continue processing and suppress the hook's output.
*
* Note: SessionStart uses context-hook.ts which constructs its own response
* with hookSpecificOutput for context injection.
*/
export const STANDARD_HOOK_RESPONSE = JSON.stringify({
continue: true,
suppressOutput: true
});

8
src/services/Context.ts
View File

@@ -1,8 +0,0 @@
/**
* Context - Named re-export facade
*
* Provides a clean import path for context generation functionality.
* Import from './Context.js' or './context/index.js'.
*/
export * from './context/index.js';

19
src/services/context-generator.ts
View File

@@ -1,19 +0,0 @@
/**
* Context Generator - DEPRECATED
*
* This file is maintained for backward compatibility.
* New code should import from './Context.js' or './context/index.js'.
*
* The context generation logic has been restructured into:
* - src/services/context/ContextBuilder.ts - Main orchestrator
* - src/services/context/ContextConfigLoader.ts - Configuration loading
* - src/services/context/TokenCalculator.ts - Token economics
* - src/services/context/ObservationCompiler.ts - Data retrieval
* - src/services/context/formatters/ - Output formatting
* - src/services/context/sections/ - Section rendering
*/
import { logger } from '../utils/logger.js';
// Re-export everything from the new context module
export { generateContext } from './context/index.js';
export type { ContextInput, ContextConfig } from './context/types.js';

185
src/services/infrastructure/ProcessManager.ts
View File

@@ -388,159 +388,6 @@ export function parseElapsedTime(etime: string): number {
return -1;
}
/**
* Enumerate orphaned claude-mem processes matching ORPHAN_PROCESS_PATTERNS.
* Returns PIDs of processes older than ORPHAN_MAX_AGE_MINUTES.
*/
async function enumerateOrphanedProcesses(isWindows: boolean, currentPid: number): Promise<number[]> {
const pidsToKill: number[] = [];
if (isWindows) {
// Windows: Use WQL -Filter for server-side filtering (no $_ pipeline syntax).
// Avoids Git Bash $_ interpretation (#1062) and PowerShell syntax errors (#1024).
const wqlPatternConditions = ORPHAN_PROCESS_PATTERNS
.map(p => `CommandLine LIKE '%${p}%'`)
.join(' OR ');
const cmd = `powershell -NoProfile -NonInteractive -Command "Get-CimInstance Win32_Process -Filter '(${wqlPatternConditions}) AND ProcessId != ${currentPid}' | Select-Object ProcessId, CreationDate | ConvertTo-Json"`;
const { stdout } = await execAsync(cmd, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, windowsHide: true });
if (!stdout.trim() || stdout.trim() === 'null') {
logger.debug('SYSTEM', 'No orphaned claude-mem processes found (Windows)');
return [];
}
const processes = JSON.parse(stdout);
const processList = Array.isArray(processes) ? processes : [processes];
const now = Date.now();
for (const proc of processList) {
const pid = proc.ProcessId;
// SECURITY: Validate PID is positive integer and not current process
if (!Number.isInteger(pid) || pid <= 0 || pid === currentPid) continue;
// Parse Windows WMI date format: /Date(1234567890123)/
const creationMatch = proc.CreationDate?.match(/\/Date\((\d+)\)\//);
if (creationMatch) {
const creationTime = parseInt(creationMatch[1], 10);
const ageMinutes = (now - creationTime) / (1000 * 60);
if (ageMinutes >= ORPHAN_MAX_AGE_MINUTES) {
pidsToKill.push(pid);
logger.debug('SYSTEM', 'Found orphaned process', { pid, ageMinutes: Math.round(ageMinutes) });
}
}
}
} else {
// Unix: Use ps with elapsed time for age-based filtering
const patternRegex = ORPHAN_PROCESS_PATTERNS.join('|');
const { stdout } = await execAsync(
`ps -eo pid,etime,command | grep -E "${patternRegex}" | grep -v grep || true`
);
if (!stdout.trim()) {
logger.debug('SYSTEM', 'No orphaned claude-mem processes found (Unix)');
return [];
}
const lines = stdout.trim().split('\n');
for (const line of lines) {
// Parse: " 1234 01:23:45 /path/to/process"
const match = line.trim().match(/^(\d+)\s+(\S+)\s+(.*)$/);
if (!match) continue;
const pid = parseInt(match[1], 10);
const etime = match[2];
// SECURITY: Validate PID is positive integer and not current process
if (!Number.isInteger(pid) || pid <= 0 || pid === currentPid) continue;
const ageMinutes = parseElapsedTime(etime);
if (ageMinutes >= ORPHAN_MAX_AGE_MINUTES) {
pidsToKill.push(pid);
logger.debug('SYSTEM', 'Found orphaned process', { pid, ageMinutes, command: match[3].substring(0, 80) });
}
}
}
return pidsToKill;
}
/**
* Clean up orphaned claude-mem processes from previous worker sessions
*
* Targets mcp-server.cjs, worker-service.cjs, and chroma-mcp processes
* that survived a previous daemon crash. Only kills processes older than
* ORPHAN_MAX_AGE_MINUTES to avoid killing the current session.
*
* The periodic ProcessRegistry reaper handles in-session orphans;
* this function handles cross-session orphans at startup.
*/
export async function cleanupOrphanedProcesses(): Promise<void> {
const isWindows = process.platform === 'win32';
const currentPid = process.pid;
let pidsToKill: number[];
try {
pidsToKill = await enumerateOrphanedProcesses(isWindows, currentPid);
} catch (error: unknown) {
// Orphan cleanup is non-critical - log and continue
if (error instanceof Error) {
logger.error('SYSTEM', 'Failed to enumerate orphaned processes', {}, error);
} else {
logger.error('SYSTEM', 'Failed to enumerate orphaned processes', {}, new Error(String(error)));
}
return;
}
if (pidsToKill.length === 0) {
return;
}
logger.info('SYSTEM', 'Cleaning up orphaned claude-mem processes', {
platform: isWindows ? 'Windows' : 'Unix',
count: pidsToKill.length,
pids: pidsToKill,
maxAgeMinutes: ORPHAN_MAX_AGE_MINUTES
});
// Kill all found processes
if (isWindows) {
for (const pid of pidsToKill) {
// SECURITY: Double-check PID validation before using in taskkill command
if (!Number.isInteger(pid) || pid <= 0) {
logger.warn('SYSTEM', 'Skipping invalid PID', { pid });
continue;
}
try {
execSync(`taskkill /PID ${pid} /T /F`, { timeout: HOOK_TIMEOUTS.POWERSHELL_COMMAND, stdio: 'ignore', windowsHide: true });
} catch (error: unknown) {
// [ANTI-PATTERN IGNORED]: Cleanup loop - process may have exited, continue to next PID
if (error instanceof Error) {
logger.debug('SYSTEM', 'Failed to kill process, may have already exited', { pid }, error);
} else {
logger.debug('SYSTEM', 'Failed to kill process, may have already exited', { pid }, new Error(String(error)));
}
}
}
} else {
for (const pid of pidsToKill) {
try {
process.kill(pid, 'SIGKILL');
} catch (error: unknown) {
// [ANTI-PATTERN IGNORED]: Cleanup loop - process may have exited, continue to next PID
if (error instanceof Error) {
logger.debug('SYSTEM', 'Process already exited', { pid }, error);
} else {
logger.debug('SYSTEM', 'Process already exited', { pid }, new Error(String(error)));
}
}
}
}
logger.info('SYSTEM', 'Orphaned processes cleaned up', { count: pidsToKill.length });
}
// Patterns that should be killed immediately at startup (no age gate)
// These are child processes that should not outlive their parent worker
const AGGRESSIVE_CLEANUP_PATTERNS = ['worker-service.cjs', 'chroma-mcp'];
@@ -1123,35 +970,3 @@ export function cleanStalePidFile(): ValidateWorkerPidStatus {
return validateWorkerPidFile({ logAlive: false });
}
/**
* Create signal handler factory for graceful shutdown
* Returns a handler function that can be passed to process.on('SIGTERM') etc.
*/
export function createSignalHandler(
shutdownFn: () => Promise<void>,
isShuttingDownRef: { value: boolean }
): (signal: string) => Promise<void> {
return async (signal: string) => {
if (isShuttingDownRef.value) {
logger.warn('SYSTEM', `Received ${signal} but shutdown already in progress`);
return;
}
isShuttingDownRef.value = true;
logger.info('SYSTEM', `Received ${signal}, shutting down...`);
try {
await shutdownFn();
process.exit(0);
} catch (error: unknown) {
// Top-level signal handler - log any shutdown error and exit
if (error instanceof Error) {
logger.error('SYSTEM', 'Error during shutdown', {}, error);
} else {
logger.error('SYSTEM', 'Error during shutdown', {}, new Error(String(error)));
}
// Exit gracefully: Windows Terminal won't keep tab open on exit 0
// Even on shutdown errors, exit cleanly to prevent tab accumulation
process.exit(0);
}
};
}

9
src/services/smart-file-read/parser.ts
View File

@@ -387,15 +387,6 @@ const QUERIES: Record<string, string> = {
(class_declaration name: (type_identifier) @name) @cls
(protocol_declaration name: (type_identifier) @name) @iface
(import_declaration) @imp
`,
php: `
(function_definition name: (name) @name) @func
(class_declaration name: (name) @name) @cls
(interface_declaration name: (name) @name) @iface
(trait_declaration name: (name) @name) @trait_def
(method_declaration name: (name) @name) @method
(namespace_use_declaration) @imp
`,
lua: `

195
src/services/sqlite/Database.ts
View File

@@ -11,14 +11,6 @@ import { MigrationRunner } from './migrations/runner.js';
const SQLITE_MMAP_SIZE_BYTES = 256 * 1024 * 1024; // 256MB
const SQLITE_CACHE_SIZE_PAGES = 10_000;
export interface Migration {
version: number;
up: (db: Database) => void;
down?: (db: Database) => void;
}
let dbInstance: Database | null = null;
/**
* Repair malformed database schema before migrations run.
*
@@ -180,170 +172,6 @@ export class ClaudeMemDatabase {
}
}
/**
* SQLite Database singleton with migration support and optimized settings
* @deprecated Use ClaudeMemDatabase instead for new code
*/
export class DatabaseManager {
private static instance: DatabaseManager;
private db: Database | null = null;
private migrations: Migration[] = [];
static getInstance(): DatabaseManager {
if (!DatabaseManager.instance) {
DatabaseManager.instance = new DatabaseManager();
}
return DatabaseManager.instance;
}
/**
* Register a migration to be run during initialization
*/
registerMigration(migration: Migration): void {
this.migrations.push(migration);
// Keep migrations sorted by version
this.migrations.sort((a, b) => a.version - b.version);
}
/**
* Initialize database connection with optimized settings
*/
async initialize(): Promise<Database> {
if (this.db) {
return this.db;
}
// Ensure the data directory exists
ensureDir(DATA_DIR);
this.db = new Database(DB_PATH, { create: true, readwrite: true });
// Repair any malformed schema before applying settings or running migrations.
// Must happen first — even PRAGMA calls can fail on a corrupted schema.
this.db = repairMalformedSchemaWithReopen(DB_PATH, this.db);
// Apply optimized SQLite settings
this.db.run('PRAGMA journal_mode = WAL');
this.db.run('PRAGMA synchronous = NORMAL');
this.db.run('PRAGMA foreign_keys = ON');
this.db.run('PRAGMA temp_store = memory');
this.db.run(`PRAGMA mmap_size = ${SQLITE_MMAP_SIZE_BYTES}`);
this.db.run(`PRAGMA cache_size = ${SQLITE_CACHE_SIZE_PAGES}`);
// Initialize schema_versions table
this.initializeSchemaVersions();
// Run migrations
await this.runMigrations();
dbInstance = this.db;
return this.db;
}
/**
* Get the current database connection
*/
getConnection(): Database {
if (!this.db) {
throw new Error('Database not initialized. Call initialize() first.');
}
return this.db;
}
/**
* Execute a function within a transaction
*/
withTransaction<T>(fn: (db: Database) => T): T {
const db = this.getConnection();
const transaction = db.transaction(fn);
return transaction(db);
}
/**
* Close the database connection
*/
close(): void {
if (this.db) {
this.db.close();
this.db = null;
dbInstance = null;
}
}
/**
* Initialize the schema_versions table
*/
private initializeSchemaVersions(): void {
if (!this.db) return;
this.db.run(`
CREATE TABLE IF NOT EXISTS schema_versions (
id INTEGER PRIMARY KEY,
version INTEGER UNIQUE NOT NULL,
applied_at TEXT NOT NULL
)
`);
}
/**
* Run all pending migrations
*/
private async runMigrations(): Promise<void> {
if (!this.db) return;
const query = this.db.query('SELECT version FROM schema_versions ORDER BY version');
const appliedVersions = query.all().map((row: any) => row.version);
const maxApplied = appliedVersions.length > 0 ? Math.max(...appliedVersions) : 0;
for (const migration of this.migrations) {
if (migration.version > maxApplied) {
logger.info('DB', `Applying migration ${migration.version}`);
const transaction = this.db.transaction(() => {
migration.up(this.db!);
const insertQuery = this.db!.query('INSERT INTO schema_versions (version, applied_at) VALUES (?, ?)');
insertQuery.run(migration.version, new Date().toISOString());
});
transaction();
logger.info('DB', `Migration ${migration.version} applied successfully`);
}
}
}
/**
* Get current schema version
*/
getCurrentVersion(): number {
if (!this.db) return 0;
const query = this.db.query('SELECT MAX(version) as version FROM schema_versions');
const result = query.get() as { version: number } | undefined;
return result?.version || 0;
}
}
/**
* Get the global database instance (for compatibility)
*/
export function getDatabase(): Database {
if (!dbInstance) {
throw new Error('Database not initialized. Call DatabaseManager.getInstance().initialize() first.');
}
return dbInstance;
}
/**
* Initialize and get database manager
*/
export async function initializeDatabase(): Promise<Database> {
const manager = DatabaseManager.getInstance();
return await manager.initialize();
}
// Re-export bun:sqlite Database type
export { Database };
@@ -351,10 +179,21 @@ export { Database };
export { MigrationRunner } from './migrations/runner.js';
// Re-export all module functions for convenient imports
export * from './Sessions.js';
export * from './Observations.js';
export * from './Summaries.js';
export * from './Prompts.js';
export * from './Timeline.js';
export * from './Import.js';
export * from './sessions/types.js';
export * from './sessions/create.js';
export * from './sessions/get.js';
export * from './observations/types.js';
export * from './observations/store.js';
export * from './observations/get.js';
export * from './observations/recent.js';
export * from './observations/files.js';
export * from './summaries/types.js';
export * from './summaries/store.js';
export * from './summaries/get.js';
export * from './summaries/recent.js';
export * from './prompts/types.js';
export * from './prompts/store.js';
export * from './prompts/get.js';
export * from './timeline/queries.js';
export * from './import/bulk.js';
export * from './transactions.js';

6
src/services/sqlite/Import.ts
View File

@@ -1,6 +0,0 @@
/**
* Import functions for bulk data import with duplicate checking
*/
import { logger } from '../../utils/logger.js';
export * from './import/bulk.js';

11
src/services/sqlite/Observations.ts
View File

@@ -1,11 +0,0 @@
/**
* Observations module - named re-exports
* Provides all observation-related database operations
*/
import { logger } from '../../utils/logger.js';
export * from './observations/types.js';
export * from './observations/store.js';
export * from './observations/get.js';
export * from './observations/recent.js';
export * from './observations/files.js';

11
src/services/sqlite/Prompts.ts
View File

@@ -1,11 +0,0 @@
/**
* User prompts module - named re-exports
*
* Provides all user prompt database operations as standalone functions.
* Each function takes `db: Database` as first parameter.
*/
import { logger } from '../../utils/logger.js';
export * from './prompts/types.js';
export * from './prompts/store.js';
export * from './prompts/get.js';

12
src/services/sqlite/Sessions.ts
View File

@@ -1,12 +0,0 @@
/**
* Sessions module - re-exports all session-related functions
*
* Usage:
* import { createSDKSession, getSessionById } from './Sessions.js';
* const sessionId = createSDKSession(db, contentId, project, prompt);
*/
import { logger } from '../../utils/logger.js';
export * from './sessions/types.js';
export * from './sessions/create.js';
export * from './sessions/get.js';

9
src/services/sqlite/Summaries.ts
View File

@@ -1,9 +0,0 @@
/**
* Summaries module - Named re-exports for summary-related database operations
*/
import { logger } from '../../utils/logger.js';
export * from './summaries/types.js';
export * from './summaries/store.js';
export * from './summaries/get.js';
export * from './summaries/recent.js';

9
src/services/sqlite/Timeline.ts
View File

@@ -1,9 +0,0 @@
/**
* Timeline module re-exports
* Provides time-based context queries for observations, sessions, and prompts
*
* grep-friendly: Timeline, getTimelineAroundTimestamp, getTimelineAroundObservation, getAllProjects
*/
import { logger } from '../../utils/logger.js';
export * from './timeline/queries.js';

29
src/services/sqlite/index.ts
View File

@@ -1,9 +1,6 @@
// Export main components
export {
ClaudeMemDatabase,
DatabaseManager,
getDatabase,
initializeDatabase,
MigrationRunner
} from './Database.js';
@@ -17,16 +14,24 @@ export { SessionSearch } from './SessionSearch.js';
// Export types
export * from './types.js';
// Export migrations
export { migrations } from './migrations.js';
// Export transactions
export { storeObservations, storeObservationsAndMarkComplete } from './transactions.js';
// Re-export all modular functions for convenient access
export * from './Sessions.js';
export * from './Observations.js';
export * from './Summaries.js';
export * from './Prompts.js';
export * from './Timeline.js';
export * from './Import.js';
export * from './sessions/types.js';
export * from './sessions/create.js';
export * from './sessions/get.js';
export * from './observations/types.js';
export * from './observations/store.js';
export * from './observations/get.js';
export * from './observations/recent.js';
export * from './observations/files.js';
export * from './summaries/types.js';
export * from './summaries/store.js';
export * from './summaries/get.js';
export * from './summaries/recent.js';
export * from './prompts/types.js';
export * from './prompts/store.js';
export * from './prompts/get.js';
export * from './timeline/queries.js';
export * from './import/bulk.js';

645
src/services/sqlite/migrations.ts
View File

@@ -1,645 +0,0 @@
import { Database } from 'bun:sqlite';
import { Migration } from './Database.js';
import { logger } from '../../utils/logger.js';
// Re-export MigrationRunner for SessionStore migration extraction
export { MigrationRunner } from './migrations/runner.js';
/**
* Initial schema migration - creates all core tables
*/
export const migration001: Migration = {
version: 1,
up: (db: Database) => {
// Sessions table - core session tracking
db.run(`
CREATE TABLE IF NOT EXISTS sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT UNIQUE NOT NULL,
project TEXT NOT NULL,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
source TEXT NOT NULL DEFAULT 'compress',
archive_path TEXT,
archive_bytes INTEGER,
archive_checksum TEXT,
archived_at TEXT,
metadata_json TEXT
);
CREATE INDEX IF NOT EXISTS idx_sessions_project ON sessions(project);
CREATE INDEX IF NOT EXISTS idx_sessions_created_at ON sessions(created_at_epoch DESC);
CREATE INDEX IF NOT EXISTS idx_sessions_project_created ON sessions(project, created_at_epoch DESC);
`);
// Memories table - compressed memory chunks
db.run(`
CREATE TABLE IF NOT EXISTS memories (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT NOT NULL,
text TEXT NOT NULL,
document_id TEXT UNIQUE,
keywords TEXT,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
project TEXT NOT NULL,
archive_basename TEXT,
origin TEXT NOT NULL DEFAULT 'transcript',
FOREIGN KEY (session_id) REFERENCES sessions(session_id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_memories_session ON memories(session_id);
CREATE INDEX IF NOT EXISTS idx_memories_project ON memories(project);
CREATE INDEX IF NOT EXISTS idx_memories_created_at ON memories(created_at_epoch DESC);
CREATE INDEX IF NOT EXISTS idx_memories_project_created ON memories(project, created_at_epoch DESC);
CREATE INDEX IF NOT EXISTS idx_memories_document_id ON memories(document_id);
CREATE INDEX IF NOT EXISTS idx_memories_origin ON memories(origin);
`);
// Overviews table - session summaries (one per project)
db.run(`
CREATE TABLE IF NOT EXISTS overviews (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT NOT NULL,
content TEXT NOT NULL,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
project TEXT NOT NULL,
origin TEXT NOT NULL DEFAULT 'claude',
FOREIGN KEY (session_id) REFERENCES sessions(session_id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_overviews_session ON overviews(session_id);
CREATE INDEX IF NOT EXISTS idx_overviews_project ON overviews(project);
CREATE INDEX IF NOT EXISTS idx_overviews_created_at ON overviews(created_at_epoch DESC);
CREATE INDEX IF NOT EXISTS idx_overviews_project_created ON overviews(project, created_at_epoch DESC);
CREATE UNIQUE INDEX IF NOT EXISTS idx_overviews_project_latest ON overviews(project, created_at_epoch DESC);
`);
// Diagnostics table - system health and debug info
db.run(`
CREATE TABLE IF NOT EXISTS diagnostics (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT,
message TEXT NOT NULL,
severity TEXT NOT NULL DEFAULT 'info',
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
project TEXT NOT NULL,
origin TEXT NOT NULL DEFAULT 'system',
FOREIGN KEY (session_id) REFERENCES sessions(session_id) ON DELETE SET NULL
);
CREATE INDEX IF NOT EXISTS idx_diagnostics_session ON diagnostics(session_id);
CREATE INDEX IF NOT EXISTS idx_diagnostics_project ON diagnostics(project);
CREATE INDEX IF NOT EXISTS idx_diagnostics_severity ON diagnostics(severity);
CREATE INDEX IF NOT EXISTS idx_diagnostics_created ON diagnostics(created_at_epoch DESC);
`);
// Transcript events table - raw conversation events
db.run(`
CREATE TABLE IF NOT EXISTS transcript_events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
session_id TEXT NOT NULL,
project TEXT,
event_index INTEGER NOT NULL,
event_type TEXT,
raw_json TEXT NOT NULL,
captured_at TEXT NOT NULL,
captured_at_epoch INTEGER NOT NULL,
FOREIGN KEY (session_id) REFERENCES sessions(session_id) ON DELETE CASCADE,
UNIQUE(session_id, event_index)
);
CREATE INDEX IF NOT EXISTS idx_transcript_events_session ON transcript_events(session_id, event_index);
CREATE INDEX IF NOT EXISTS idx_transcript_events_project ON transcript_events(project);
CREATE INDEX IF NOT EXISTS idx_transcript_events_type ON transcript_events(event_type);
CREATE INDEX IF NOT EXISTS idx_transcript_events_captured ON transcript_events(captured_at_epoch DESC);
`);
console.log('✅ Created all database tables successfully');
},
down: (db: Database) => {
db.run(`
DROP TABLE IF EXISTS transcript_events;
DROP TABLE IF EXISTS diagnostics;
DROP TABLE IF EXISTS overviews;
DROP TABLE IF EXISTS memories;
DROP TABLE IF EXISTS sessions;
`);
}
};
/**
* Migration 002 - Add hierarchical memory fields (v2 format)
*/
export const migration002: Migration = {
version: 2,
up: (db: Database) => {
// Add new columns for hierarchical memory structure
db.run(`
ALTER TABLE memories ADD COLUMN title TEXT;
ALTER TABLE memories ADD COLUMN subtitle TEXT;
ALTER TABLE memories ADD COLUMN facts TEXT;
ALTER TABLE memories ADD COLUMN concepts TEXT;
ALTER TABLE memories ADD COLUMN files_touched TEXT;
`);
// Create indexes for the new fields to improve search performance
db.run(`
CREATE INDEX IF NOT EXISTS idx_memories_title ON memories(title);
CREATE INDEX IF NOT EXISTS idx_memories_concepts ON memories(concepts);
`);
console.log('✅ Added hierarchical memory fields to memories table');
},
down: (_db: Database) => {
// Note: SQLite doesn't support DROP COLUMN in all versions
// In production, we'd need to recreate the table without these columns
// For now, we'll just log a warning
console.log('⚠️ Warning: SQLite ALTER TABLE DROP COLUMN not fully supported');
console.log('⚠️ To rollback, manually recreate the memories table');
}
};
/**
* Migration 003 - Add streaming_sessions table for real-time session tracking
*/
export const migration003: Migration = {
version: 3,
up: (db: Database) => {
// Streaming sessions table - tracks active SDK compression sessions
db.run(`
CREATE TABLE IF NOT EXISTS streaming_sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
content_session_id TEXT UNIQUE NOT NULL,
memory_session_id TEXT,
project TEXT NOT NULL,
title TEXT,
subtitle TEXT,
user_prompt TEXT,
started_at TEXT NOT NULL,
started_at_epoch INTEGER NOT NULL,
updated_at TEXT,
updated_at_epoch INTEGER,
completed_at TEXT,
completed_at_epoch INTEGER,
status TEXT NOT NULL DEFAULT 'active'
);
CREATE INDEX IF NOT EXISTS idx_streaming_sessions_claude_id ON streaming_sessions(content_session_id);
CREATE INDEX IF NOT EXISTS idx_streaming_sessions_sdk_id ON streaming_sessions(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_streaming_sessions_project ON streaming_sessions(project);
CREATE INDEX IF NOT EXISTS idx_streaming_sessions_status ON streaming_sessions(status);
CREATE INDEX IF NOT EXISTS idx_streaming_sessions_started ON streaming_sessions(started_at_epoch DESC);
`);
console.log('✅ Created streaming_sessions table for real-time session tracking');
},
down: (db: Database) => {
db.run(`
DROP TABLE IF EXISTS streaming_sessions;
`);
}
};
/**
* Migration 004 - Add SDK agent architecture tables
* Implements the refactor plan for hook-driven memory with SDK agent synthesis
*/
export const migration004: Migration = {
version: 4,
up: (db: Database) => {
// SDK sessions table - tracks SDK streaming sessions
db.run(`
CREATE TABLE IF NOT EXISTS sdk_sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
content_session_id TEXT UNIQUE NOT NULL,
memory_session_id TEXT UNIQUE,
project TEXT NOT NULL,
user_prompt TEXT,
started_at TEXT NOT NULL,
started_at_epoch INTEGER NOT NULL,
completed_at TEXT,
completed_at_epoch INTEGER,
status TEXT CHECK(status IN ('active', 'completed', 'failed')) NOT NULL DEFAULT 'active'
);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_claude_id ON sdk_sessions(content_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_sdk_id ON sdk_sessions(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_project ON sdk_sessions(project);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_status ON sdk_sessions(status);
CREATE INDEX IF NOT EXISTS idx_sdk_sessions_started ON sdk_sessions(started_at_epoch DESC);
`);
// Observation queue table - tracks pending observations for SDK processing
db.run(`
CREATE TABLE IF NOT EXISTS observation_queue (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT NOT NULL,
tool_name TEXT NOT NULL,
tool_input TEXT NOT NULL,
tool_output TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
processed_at_epoch INTEGER,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_observation_queue_sdk_session ON observation_queue(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_observation_queue_processed ON observation_queue(processed_at_epoch);
CREATE INDEX IF NOT EXISTS idx_observation_queue_pending ON observation_queue(memory_session_id, processed_at_epoch);
`);
// Observations table - stores extracted observations (what SDK decides is important)
db.run(`
CREATE TABLE IF NOT EXISTS observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT NOT NULL,
project TEXT NOT NULL,
text TEXT NOT NULL,
type TEXT NOT NULL,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_observations_sdk_session ON observations(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_observations_project ON observations(project);
CREATE INDEX IF NOT EXISTS idx_observations_type ON observations(type);
CREATE INDEX IF NOT EXISTS idx_observations_created ON observations(created_at_epoch DESC);
`);
// Session summaries table - stores structured session summaries
db.run(`
CREATE TABLE IF NOT EXISTS session_summaries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT UNIQUE NOT NULL,
project TEXT NOT NULL,
request TEXT,
investigated TEXT,
learned TEXT,
completed TEXT,
next_steps TEXT,
files_read TEXT,
files_edited TEXT,
notes TEXT,
created_at TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE
);
CREATE INDEX IF NOT EXISTS idx_session_summaries_sdk_session ON session_summaries(memory_session_id);
CREATE INDEX IF NOT EXISTS idx_session_summaries_project ON session_summaries(project);
CREATE INDEX IF NOT EXISTS idx_session_summaries_created ON session_summaries(created_at_epoch DESC);
`);
console.log('✅ Created SDK agent architecture tables');
},
down: (db: Database) => {
db.run(`
DROP TABLE IF EXISTS session_summaries;
DROP TABLE IF EXISTS observations;
DROP TABLE IF EXISTS observation_queue;
DROP TABLE IF EXISTS sdk_sessions;
`);
}
};
/**
* Migration 005 - Remove orphaned tables
* Drops streaming_sessions (superseded by sdk_sessions)
* Drops observation_queue (superseded by Unix socket communication)
*/
export const migration005: Migration = {
version: 5,
up: (db: Database) => {
// Drop streaming_sessions - superseded by sdk_sessions in migration004
// This table was from v2 architecture and is no longer used
db.run(`DROP TABLE IF EXISTS streaming_sessions`);
// Drop observation_queue - superseded by Unix socket communication
// Worker now uses sockets instead of database polling for observations
db.run(`DROP TABLE IF EXISTS observation_queue`);
console.log('✅ Dropped orphaned tables: streaming_sessions, observation_queue');
},
down: (db: Database) => {
// Recreate tables if needed (though they should never be used)
db.run(`
CREATE TABLE IF NOT EXISTS streaming_sessions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
content_session_id TEXT UNIQUE NOT NULL,
memory_session_id TEXT,
project TEXT NOT NULL,
title TEXT,
subtitle TEXT,
user_prompt TEXT,
started_at TEXT NOT NULL,
started_at_epoch INTEGER NOT NULL,
updated_at TEXT,
updated_at_epoch INTEGER,
completed_at TEXT,
completed_at_epoch INTEGER,
status TEXT NOT NULL DEFAULT 'active'
)
`);
db.run(`
CREATE TABLE IF NOT EXISTS observation_queue (
id INTEGER PRIMARY KEY AUTOINCREMENT,
memory_session_id TEXT NOT NULL,
tool_name TEXT NOT NULL,
tool_input TEXT NOT NULL,
tool_output TEXT NOT NULL,
created_at_epoch INTEGER NOT NULL,
processed_at_epoch INTEGER,
FOREIGN KEY(memory_session_id) REFERENCES sdk_sessions(memory_session_id) ON DELETE CASCADE
)
`);
console.log('⚠️ Recreated streaming_sessions and observation_queue (for rollback only)');
}
};
/**
* Migration 006 - Add FTS5 full-text search tables
* Creates virtual tables for fast text search on observations and session_summaries
*/
export const migration006: Migration = {
version: 6,
up: (db: Database) => {
// FTS5 may be unavailable on some platforms (e.g., Bun on Windows #791).
// Probe before creating tables — search falls back to ChromaDB when unavailable.
try {
db.run('CREATE VIRTUAL TABLE _fts5_probe USING fts5(test_column)');
db.run('DROP TABLE _fts5_probe');
} catch (error) {
logger.warn('DB', 'FTS5 not available on this platform — skipping FTS migration (search uses ChromaDB)', {}, error instanceof Error ? error : undefined);
return;
}
// FTS5 virtual table for observations
// Note: This assumes the hierarchical fields (title, subtitle, etc.) already exist
// from the inline migrations in SessionStore constructor
db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS observations_fts USING fts5(
title,
subtitle,
narrative,
text,
facts,
concepts,
content='observations',
content_rowid='id'
);
`);
// Populate FTS table with existing data
db.run(`
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
SELECT id, title, subtitle, narrative, text, facts, concepts
FROM observations;
`);
// Triggers to keep observations_fts in sync
db.run(`
CREATE TRIGGER IF NOT EXISTS observations_ai AFTER INSERT ON observations BEGIN
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
CREATE TRIGGER IF NOT EXISTS observations_ad AFTER DELETE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
END;
CREATE TRIGGER IF NOT EXISTS observations_au AFTER UPDATE ON observations BEGIN
INSERT INTO observations_fts(observations_fts, rowid, title, subtitle, narrative, text, facts, concepts)
VALUES('delete', old.id, old.title, old.subtitle, old.narrative, old.text, old.facts, old.concepts);
INSERT INTO observations_fts(rowid, title, subtitle, narrative, text, facts, concepts)
VALUES (new.id, new.title, new.subtitle, new.narrative, new.text, new.facts, new.concepts);
END;
`);
// FTS5 virtual table for session_summaries
db.run(`
CREATE VIRTUAL TABLE IF NOT EXISTS session_summaries_fts USING fts5(
request,
investigated,
learned,
completed,
next_steps,
notes,
content='session_summaries',
content_rowid='id'
);
`);
// Populate FTS table with existing data
db.run(`
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
SELECT id, request, investigated, learned, completed, next_steps, notes
FROM session_summaries;
`);
// Triggers to keep session_summaries_fts in sync
db.run(`
CREATE TRIGGER IF NOT EXISTS session_summaries_ai AFTER INSERT ON session_summaries BEGIN
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
CREATE TRIGGER IF NOT EXISTS session_summaries_ad AFTER DELETE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
END;
CREATE TRIGGER IF NOT EXISTS session_summaries_au AFTER UPDATE ON session_summaries BEGIN
INSERT INTO session_summaries_fts(session_summaries_fts, rowid, request, investigated, learned, completed, next_steps, notes)
VALUES('delete', old.id, old.request, old.investigated, old.learned, old.completed, old.next_steps, old.notes);
INSERT INTO session_summaries_fts(rowid, request, investigated, learned, completed, next_steps, notes)
VALUES (new.id, new.request, new.investigated, new.learned, new.completed, new.next_steps, new.notes);
END;
`);
console.log('✅ Created FTS5 virtual tables and triggers for full-text search');
},
down: (db: Database) => {
db.run(`
DROP TRIGGER IF EXISTS observations_au;
DROP TRIGGER IF EXISTS observations_ad;
DROP TRIGGER IF EXISTS observations_ai;
DROP TABLE IF EXISTS observations_fts;
DROP TRIGGER IF EXISTS session_summaries_au;
DROP TRIGGER IF EXISTS session_summaries_ad;
DROP TRIGGER IF EXISTS session_summaries_ai;
DROP TABLE IF EXISTS session_summaries_fts;
`);
}
};
/**
* Migration 007 - Add discovery_tokens column for ROI metrics
* Tracks token cost of discovering/creating each observation and summary
*/
export const migration007: Migration = {
version: 7,
up: (db: Database) => {
// Add discovery_tokens to observations table
db.run(`ALTER TABLE observations ADD COLUMN discovery_tokens INTEGER DEFAULT 0`);
// Add discovery_tokens to session_summaries table
db.run(`ALTER TABLE session_summaries ADD COLUMN discovery_tokens INTEGER DEFAULT 0`);
console.log('✅ Added discovery_tokens columns for ROI tracking');
},
down: (db: Database) => {
// Note: SQLite doesn't support DROP COLUMN in all versions
// In production, would need to recreate tables without these columns
console.log('⚠️ Warning: SQLite ALTER TABLE DROP COLUMN not fully supported');
console.log('⚠️ To rollback, manually recreate the observations and session_summaries tables');
}
};
/**
* All migrations in order
*/
/**
* Migration 008: Observation feedback table for tracking observation usage
*
* Tracks how observations are used (semantic injection hits, search access,
* explicit retrieval). Foundation for future Thompson Sampling optimization.
*/
export const migration008: Migration = {
version: 25,
up: (db: Database) => {
db.run(`
CREATE TABLE IF NOT EXISTS observation_feedback (
id INTEGER PRIMARY KEY AUTOINCREMENT,
observation_id INTEGER NOT NULL,
signal_type TEXT NOT NULL,
session_db_id INTEGER,
created_at_epoch INTEGER NOT NULL,
metadata TEXT,
FOREIGN KEY (observation_id) REFERENCES observations(id) ON DELETE CASCADE
)
`);
db.run(`CREATE INDEX IF NOT EXISTS idx_feedback_observation ON observation_feedback(observation_id)`);
db.run(`CREATE INDEX IF NOT EXISTS idx_feedback_signal ON observation_feedback(signal_type)`);
console.log('✅ Created observation_feedback table for usage tracking');
},
down: (db: Database) => {
db.run(`DROP TABLE IF EXISTS observation_feedback`);
}
};
/**
* Migration 009: Add missing columns to observations table
*
* The generated_by_model column tracks which model generated each observation
* (required for model selection optimization via Thompson Sampling).
* The relevance_count column tracks how many times an observation was reused
* (incremented by the feedback recording pipeline).
*
* Both columns may already exist in databases created by the compiled binary
* (v10.6.3) but are missing from the migration source. This migration
* conditionally adds them.
*/
export const migration009: Migration = {
version: 26,
up: (db: Database) => {
const columns = db.prepare('PRAGMA table_info(observations)').all() as any[];
const hasGeneratedByModel = columns.some((c: any) => c.name === 'generated_by_model');
const hasRelevanceCount = columns.some((c: any) => c.name === 'relevance_count');
if (!hasGeneratedByModel) {
db.run('ALTER TABLE observations ADD COLUMN generated_by_model TEXT');
}
if (!hasRelevanceCount) {
db.run('ALTER TABLE observations ADD COLUMN relevance_count INTEGER DEFAULT 0');
}
},
down: (_db: Database) => {
// SQLite does not support DROP COLUMN in older versions; no-op
}
};
/**
* Migration 010: Label observations (and their queue rows) with the subagent identity.
*
* Claude Code hooks that fire inside a subagent carry agent_id and agent_type on the
* stdin payload. These flow hook → worker → pending_messages → SDK storage so that
* observation rows can be attributed to the originating subagent. Main-session rows
* keep NULL for both columns.
*/
export const migration010: Migration = {
version: 27,
up: (db: Database) => {
const added: string[] = [];
const obsColumns = db.prepare('PRAGMA table_info(observations)').all() as Array<{ name: string }>;
const obsHasAgentType = obsColumns.some(c => c.name === 'agent_type');
const obsHasAgentId = obsColumns.some(c => c.name === 'agent_id');
if (!obsHasAgentType) {
db.run('ALTER TABLE observations ADD COLUMN agent_type TEXT');
added.push('observations.agent_type');
}
if (!obsHasAgentId) {
db.run('ALTER TABLE observations ADD COLUMN agent_id TEXT');
added.push('observations.agent_id');
}
db.run('CREATE INDEX IF NOT EXISTS idx_observations_agent_type ON observations(agent_type)');
db.run('CREATE INDEX IF NOT EXISTS idx_observations_agent_id ON observations(agent_id)');
// Also thread the same fields through the pending_messages queue so the label
// survives worker restarts between enqueue and SDK-agent processing.
const pendingColumns = db.prepare('PRAGMA table_info(pending_messages)').all() as Array<{ name: string }>;
if (pendingColumns.length > 0) {
const pendingHasAgentType = pendingColumns.some(c => c.name === 'agent_type');
const pendingHasAgentId = pendingColumns.some(c => c.name === 'agent_id');
if (!pendingHasAgentType) {
db.run('ALTER TABLE pending_messages ADD COLUMN agent_type TEXT');
added.push('pending_messages.agent_type');
}
if (!pendingHasAgentId) {
db.run('ALTER TABLE pending_messages ADD COLUMN agent_id TEXT');
added.push('pending_messages.agent_id');
}
}
logger.debug(
'DB',
added.length > 0
? `[migration010] Added columns: ${added.join(', ')}`
: '[migration010] Subagent identity columns already present; ensured indexes'
);
},
down: (_db: Database) => {
// SQLite DROP COLUMN not fully supported; no-op
}
};
/**
* All migrations in order
*/
export const migrations: Migration[] = [
migration001,
migration002,
migration003,
migration004,
migration005,
migration006,
migration007,
migration008,
migration009,
migration010
];

10
src/services/worker-service.ts
View File

@@ -440,16 +440,10 @@ export class WorkerService {
this.server.registerRoutes(this.searchRoutes);
logger.info('WORKER', 'SearchManager initialized and search routes registered');
// Register corpus routes (knowledge agents) — needs SearchOrchestrator from search module
const { SearchOrchestrator } = await import('./worker/search/SearchOrchestrator.js');
const corpusSearchOrchestrator = new SearchOrchestrator(
this.dbManager.getSessionSearch(),
this.dbManager.getSessionStore(),
this.dbManager.getChromaSync()
);
// Register corpus routes (knowledge agents) — uses SearchManager for observation lookup
const corpusBuilder = new CorpusBuilder(
this.dbManager.getSessionStore(),
corpusSearchOrchestrator,
searchManager,
this.corpusStore
);
const knowledgeAgent = new KnowledgeAgent(this.corpusStore);

21
src/services/worker/SearchManager.ts
View File

@@ -25,32 +25,17 @@ import { getProjectContext } from '../../utils/project-name.js';
import { formatDate, formatTime, formatDateTime, extractFirstFile, groupByDate, estimateTokens } from '../../shared/timeline-formatting.js';
import { ModeManager } from '../domain/ModeManager.js';
import {
SearchOrchestrator,
TimelineBuilder,
SEARCH_CONSTANTS
} from './search/index.js';
import type { TimelineData } from './search/index.js';
import { SEARCH_CONSTANTS } from './search/index.js';
import type { TimelineData } from './TimelineService.js';
export class SearchManager {
private orchestrator: SearchOrchestrator;
private timelineBuilder: TimelineBuilder;
constructor(
private sessionSearch: SessionSearch,
private sessionStore: SessionStore,
private chromaSync: ChromaSync | null,
private formatter: FormattingService,
private timelineService: TimelineService
) {
// Initialize the new modular search infrastructure
this.orchestrator = new SearchOrchestrator(
sessionSearch,
sessionStore,
chromaSync
);
this.timelineBuilder = new TimelineBuilder();
}
) {}
/**
* Query Chroma vector database via ChromaSync

195
src/services/worker/TimelineService.ts
View File

@@ -4,8 +4,6 @@
*/
import type { ObservationSearchResult, SessionSummarySearchResult, UserPromptSearchResult } from '../sqlite/types.js';
import { ModeManager } from '../domain/ModeManager.js';
import { logger } from '../../utils/logger.js';
/**
* Timeline item for unified chronological display
@@ -67,197 +65,4 @@ export class TimelineService {
return items.slice(startIndex, endIndex);
}
/**
* Format timeline items as markdown with grouped days and tables
*/
formatTimeline(
items: TimelineItem[],
anchorId: number | string | null,
query?: string,
depth_before?: number,
depth_after?: number
): string {
if (items.length === 0) {
return query
? `Found observation matching "${query}", but no timeline context available.`
: 'No timeline items found';
}
const lines: string[] = [];
// Header
if (query && anchorId) {
const anchorObs = items.find(item => item.type === 'observation' && (item.data as ObservationSearchResult).id === anchorId);
const anchorTitle = anchorObs ? ((anchorObs.data as ObservationSearchResult).title || 'Untitled') : 'Unknown';
lines.push(`# Timeline for query: "${query}"`);
lines.push(`**Anchor:** Observation #${anchorId} - ${anchorTitle}`);
} else if (anchorId) {
lines.push(`# Timeline around anchor: ${anchorId}`);
} else {
lines.push(`# Timeline`);
}
if (depth_before !== undefined && depth_after !== undefined) {
lines.push(`**Window:** ${depth_before} records before → ${depth_after} records after | **Items:** ${items.length}`);
} else {
lines.push(`**Items:** ${items.length}`);
}
lines.push('');
// Legend
lines.push(`**Legend:** 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | 🧠 decision`);
lines.push('');
// Group by day
const dayMap = new Map<string, TimelineItem[]>();
for (const item of items) {
const day = this.formatDate(item.epoch);
if (!dayMap.has(day)) {
dayMap.set(day, []);
}
dayMap.get(day)!.push(item);
}
// Sort days chronologically
const sortedDays = Array.from(dayMap.entries()).sort((a, b) => {
const aDate = new Date(a[0]).getTime();
const bDate = new Date(b[0]).getTime();
return aDate - bDate;
});
// Render each day
for (const [day, dayItems] of sortedDays) {
lines.push(`### ${day}`);
lines.push('');
let currentFile: string | null = null;
let lastTime = '';
let tableOpen = false;
for (const item of dayItems) {
const isAnchor = (
(typeof anchorId === 'number' && item.type === 'observation' && (item.data as ObservationSearchResult).id === anchorId) ||
(typeof anchorId === 'string' && anchorId.startsWith('S') && item.type === 'session' && `S${(item.data as SessionSummarySearchResult).id}` === anchorId)
);
if (item.type === 'session') {
if (tableOpen) {
lines.push('');
tableOpen = false;
currentFile = null;
lastTime = '';
}
const sess = item.data as SessionSummarySearchResult;
const title = sess.request || 'Session summary';
const marker = isAnchor ? ' ← **ANCHOR**' : '';
lines.push(`**🎯 #S${sess.id}** ${title} (${this.formatDateTime(item.epoch)})${marker}`);
lines.push('');
} else if (item.type === 'prompt') {
if (tableOpen) {
lines.push('');
tableOpen = false;
currentFile = null;
lastTime = '';
}
const prompt = item.data as UserPromptSearchResult;
const truncated = prompt.prompt_text.length > 100 ? prompt.prompt_text.substring(0, 100) + '...' : prompt.prompt_text;
lines.push(`**💬 User Prompt #${prompt.prompt_number}** (${this.formatDateTime(item.epoch)})`);
lines.push(`> ${truncated}`);
lines.push('');
} else if (item.type === 'observation') {
const obs = item.data as ObservationSearchResult;
const file = 'General';
if (file !== currentFile) {
if (tableOpen) {
lines.push('');
}
lines.push(`**${file}**`);
lines.push(`| ID | Time | T | Title | Tokens |`);
lines.push(`|----|------|---|-------|--------|`);
currentFile = file;
tableOpen = true;
lastTime = '';
}
const icon = this.getTypeIcon(obs.type);
const time = this.formatTime(item.epoch);
const title = obs.title || 'Untitled';
const tokens = this.estimateTokens(obs.narrative);
const showTime = time !== lastTime;
const timeDisplay = showTime ? time : '″';
lastTime = time;
const anchorMarker = isAnchor ? ' ← **ANCHOR**' : '';
lines.push(`| #${obs.id} | ${timeDisplay} | ${icon} | ${title}${anchorMarker} | ~${tokens} |`);
}
}
if (tableOpen) {
lines.push('');
}
}
return lines.join('\n');
}
/**
* Get icon for observation type
*/
private getTypeIcon(type: string): string {
return ModeManager.getInstance().getTypeIcon(type);
}
/**
* Format date for grouping (e.g., "Dec 7, 2025")
*/
private formatDate(epochMs: number): string {
const date = new Date(epochMs);
return date.toLocaleString('en-US', {
month: 'short',
day: 'numeric',
year: 'numeric'
});
}
/**
* Format time (e.g., "6:30 PM")
*/
private formatTime(epochMs: number): string {
const date = new Date(epochMs);
return date.toLocaleString('en-US', {
hour: 'numeric',
minute: '2-digit',
hour12: true
});
}
/**
* Format date and time (e.g., "Dec 7, 6:30 PM")
*/
private formatDateTime(epochMs: number): string {
const date = new Date(epochMs);
return date.toLocaleString('en-US', {
month: 'short',
day: 'numeric',
hour: 'numeric',
minute: '2-digit',
hour12: true
});
}
/**
* Estimate tokens from text length (~4 chars per token)
*/
private estimateTokens(text: string | null): number {
if (!text) return 0;
return Math.ceil(text.length / 4);
}
}

4
src/services/worker/http/routes/SearchRoutes.ts
View File

@@ -175,7 +175,7 @@ export class SearchRoutes extends BaseRouteHandler {
}
// Import context generator (runs in worker, has access to database)
const { generateContext } = await import('../../../context-generator.js');
const { generateContext } = await import('../../../context/index.js');
// Use project name as CWD (generateContext uses path.basename to get project)
const cwd = `/preview/${projectName}`;
@@ -226,7 +226,7 @@ export class SearchRoutes extends BaseRouteHandler {
}
// Import context generator (runs in worker, has access to database)
const { generateContext } = await import('../../../context-generator.js');
const { generateContext } = await import('../../../context/index.js');
// Use first project name as CWD (for display purposes)
const primaryProject = projects[projects.length - 1]; // Last is the current/primary project

16
src/services/worker/knowledge/CorpusBuilder.ts
View File

@@ -1,14 +1,14 @@
/**
* CorpusBuilder - Compiles observations from the database into a corpus file
*
* Uses SearchOrchestrator to find matching observations, hydrates them via
* Uses SearchManager to find matching observations, hydrates them via
* SessionStore, and assembles them into a complete CorpusFile.
*/
import { logger } from '../../../utils/logger.js';
import type { ObservationRecord } from '../../../types/database.js';
import type { SessionStore } from '../../sqlite/SessionStore.js';
import type { SearchOrchestrator } from '../search/SearchOrchestrator.js';
import type { SearchManager } from '../SearchManager.js';
import { CorpusRenderer } from './CorpusRenderer.js';
import { CorpusStore } from './CorpusStore.js';
import type { CorpusFile, CorpusFilter, CorpusObservation, CorpusStats } from './types.js';
@@ -38,7 +38,7 @@ export class CorpusBuilder {
constructor(
private sessionStore: SessionStore,
private searchOrchestrator: SearchOrchestrator,
private searchManager: SearchManager,
private corpusStore: CorpusStore
) {
this.renderer = new CorpusRenderer();
@@ -50,8 +50,8 @@ export class CorpusBuilder {
async build(name: string, description: string, filter: CorpusFilter): Promise<CorpusFile> {
logger.debug('WORKER', `Building corpus "${name}" with filter`, { filter });
// Step 1: Search for matching observation IDs via SearchOrchestrator
const searchArgs: Record<string, unknown> = {};
// Step 1: Search for matching observation IDs via SearchManager (json format for raw data)
const searchArgs: Record<string, unknown> = { format: 'json' };
if (filter.project) searchArgs.project = filter.project;
if (filter.types && filter.types.length > 0) searchArgs.type = filter.types.join(',');
if (filter.concepts && filter.concepts.length > 0) searchArgs.concepts = filter.concepts.join(',');
@@ -61,10 +61,10 @@ export class CorpusBuilder {
if (filter.date_end) searchArgs.dateEnd = filter.date_end;
if (filter.limit) searchArgs.limit = filter.limit;
const searchResult = await this.searchOrchestrator.search(searchArgs);
const searchResult = await this.searchManager.search(searchArgs);
// Extract observation IDs from search results
const observationIds = (searchResult.results.observations || []).map(
// Extract observation IDs from search results (format: 'json' returns { observations, sessions, prompts, ... })
const observationIds = (searchResult.observations || []).map(
(obs: { id: number }) => obs.id
);

301
src/services/worker/search/ResultFormatter.ts
View File

@@ -1,301 +0,0 @@
/**
* ResultFormatter - Formats search results for display
*
* Consolidates formatting logic from FormattingService and SearchManager.
* Provides consistent table and text formatting for all search result types.
*/
import { logger } from '../../../utils/logger.js';
import {
ObservationSearchResult,
SessionSummarySearchResult,
UserPromptSearchResult,
CombinedResult,
SearchResults
} from './types.js';
import { ModeManager } from '../../domain/ModeManager.js';
import { formatTime, extractFirstFile, groupByDate, estimateTokens } from '../../../shared/timeline-formatting.js';
const CHARS_PER_TOKEN_ESTIMATE = 4;
export class ResultFormatter {
/**
* Format search results as markdown text
*/
formatSearchResults(
results: SearchResults,
query: string,
chromaFailed: boolean = false
): string {
const totalResults = results.observations.length +
results.sessions.length +
results.prompts.length;
if (totalResults === 0) {
if (chromaFailed) {
return this.formatChromaFailureMessage();
}
return `No results found matching "${query}"`;
}
// Combine all results with timestamps for unified sorting
const combined = this.combineResults(results);
// Sort by date
combined.sort((a, b) => b.epoch - a.epoch);
// Group by date, then by file within each day
const cwd = process.cwd();
const resultsByDate = groupByDate(combined, item => item.created_at);
// Build output with date/file grouping
const lines: string[] = [];
lines.push(`Found ${totalResults} result(s) matching "${query}" (${results.observations.length} obs, ${results.sessions.length} sessions, ${results.prompts.length} prompts)`);
lines.push('');
for (const [day, dayResults] of resultsByDate) {
lines.push(`### ${day}`);
lines.push('');
// Group by file within this day
const resultsByFile = new Map<string, CombinedResult[]>();
for (const result of dayResults) {
let file = 'General';
if (result.type === 'observation') {
const obs = result.data as ObservationSearchResult;
file = extractFirstFile(obs.files_modified, cwd, obs.files_read);
}
if (!resultsByFile.has(file)) {
resultsByFile.set(file, []);
}
resultsByFile.get(file)!.push(result);
}
// Render each file section
for (const [file, fileResults] of resultsByFile) {
lines.push(`**${file}**`);
lines.push(this.formatSearchTableHeader());
let lastTime = '';
for (const result of fileResults) {
if (result.type === 'observation') {
const formatted = this.formatObservationSearchRow(
result.data as ObservationSearchResult,
lastTime
);
lines.push(formatted.row);
lastTime = formatted.time;
} else if (result.type === 'session') {
const formatted = this.formatSessionSearchRow(
result.data as SessionSummarySearchResult,
lastTime
);
lines.push(formatted.row);
lastTime = formatted.time;
} else {
const formatted = this.formatPromptSearchRow(
result.data as UserPromptSearchResult,
lastTime
);
lines.push(formatted.row);
lastTime = formatted.time;
}
}
lines.push('');
}
}
return lines.join('\n');
}
/**
* Combine results into unified format
*/
combineResults(results: SearchResults): CombinedResult[] {
return [
...results.observations.map(obs => ({
type: 'observation' as const,
data: obs,
epoch: obs.created_at_epoch,
created_at: obs.created_at
})),
...results.sessions.map(sess => ({
type: 'session' as const,
data: sess,
epoch: sess.created_at_epoch,
created_at: sess.created_at
})),
...results.prompts.map(prompt => ({
type: 'prompt' as const,
data: prompt,
epoch: prompt.created_at_epoch,
created_at: prompt.created_at
}))
];
}
/**
* Format search table header (no Work column)
*/
formatSearchTableHeader(): string {
return `| ID | Time | T | Title | Read |
|----|------|---|-------|------|`;
}
/**
* Format full table header (with Work column)
*/
formatTableHeader(): string {
return `| ID | Time | T | Title | Read | Work |
|-----|------|---|-------|------|------|`;
}
/**
* Format observation as table row for search results
*/
formatObservationSearchRow(
obs: ObservationSearchResult,
lastTime: string
): { row: string; time: string } {
const id = `#${obs.id}`;
const time = formatTime(obs.created_at_epoch);
const icon = ModeManager.getInstance().getTypeIcon(obs.type);
const title = obs.title || 'Untitled';
const readTokens = this.estimateReadTokens(obs);
const timeDisplay = time === lastTime ? '"' : time;
return {
row: `| ${id} | ${timeDisplay} | ${icon} | ${title} | ~${readTokens} |`,
time
};
}
/**
* Format session as table row for search results
*/
formatSessionSearchRow(
session: SessionSummarySearchResult,
lastTime: string
): { row: string; time: string } {
const id = `#S${session.id}`;
const time = formatTime(session.created_at_epoch);
const icon = '\uD83C\uDFAF'; // Target emoji
const title = session.request ||
`Session ${session.memory_session_id?.substring(0, 8) || 'unknown'}`;
const timeDisplay = time === lastTime ? '"' : time;
return {
row: `| ${id} | ${timeDisplay} | ${icon} | ${title} | - |`,
time
};
}
/**
* Format user prompt as table row for search results
*/
formatPromptSearchRow(
prompt: UserPromptSearchResult,
lastTime: string
): { row: string; time: string } {
const id = `#P${prompt.id}`;
const time = formatTime(prompt.created_at_epoch);
const icon = '\uD83D\uDCAC'; // Speech bubble emoji
const title = prompt.prompt_text.length > 60
? prompt.prompt_text.substring(0, 57) + '...'
: prompt.prompt_text;
const timeDisplay = time === lastTime ? '"' : time;
return {
row: `| ${id} | ${timeDisplay} | ${icon} | ${title} | - |`,
time
};
}
/**
* Format observation as index row (with Work column)
*/
formatObservationIndex(obs: ObservationSearchResult, _index: number): string {
const id = `#${obs.id}`;
const time = formatTime(obs.created_at_epoch);
const icon = ModeManager.getInstance().getTypeIcon(obs.type);
const title = obs.title || 'Untitled';
const readTokens = this.estimateReadTokens(obs);
const workEmoji = ModeManager.getInstance().getWorkEmoji(obs.type);
const workTokens = obs.discovery_tokens || 0;
const workDisplay = workTokens > 0 ? `${workEmoji} ${workTokens}` : '-';
return `| ${id} | ${time} | ${icon} | ${title} | ~${readTokens} | ${workDisplay} |`;
}
/**
* Format session as index row
*/
formatSessionIndex(session: SessionSummarySearchResult, _index: number): string {
const id = `#S${session.id}`;
const time = formatTime(session.created_at_epoch);
const icon = '\uD83C\uDFAF';
const title = session.request ||
`Session ${session.memory_session_id?.substring(0, 8) || 'unknown'}`;
return `| ${id} | ${time} | ${icon} | ${title} | - | - |`;
}
/**
* Format user prompt as index row
*/
formatPromptIndex(prompt: UserPromptSearchResult, _index: number): string {
const id = `#P${prompt.id}`;
const time = formatTime(prompt.created_at_epoch);
const icon = '\uD83D\uDCAC';
const title = prompt.prompt_text.length > 60
? prompt.prompt_text.substring(0, 57) + '...'
: prompt.prompt_text;
return `| ${id} | ${time} | ${icon} | ${title} | - | - |`;
}
/**
* Estimate read tokens for an observation
*/
private estimateReadTokens(obs: ObservationSearchResult): number {
const size = (obs.title?.length || 0) +
(obs.subtitle?.length || 0) +
(obs.narrative?.length || 0) +
(obs.facts?.length || 0);
return Math.ceil(size / CHARS_PER_TOKEN_ESTIMATE);
}
/**
* Format Chroma failure message
*/
private formatChromaFailureMessage(): string {
return `Vector search failed - semantic search unavailable.
To enable semantic search:
1. Install uv: https://docs.astral.sh/uv/getting-started/installation/
2. Restart the worker: npm run worker:restart
Note: You can still use filter-only searches (date ranges, types, files) without a query term.`;
}
/**
* Format search tips footer
*/
formatSearchTips(): string {
return `
---
Search Strategy:
1. Search with index to see titles, dates, IDs
2. Use timeline to get context around interesting results
3. Batch fetch full details: get_observations(ids=[...])
Tips:
- Filter by type: obs_type="bugfix,feature"
- Filter by date: dateStart="2025-01-01"
- Sort: orderBy="date_desc" or "date_asc"`;
}
}

290
src/services/worker/search/SearchOrchestrator.ts
View File

@@ -1,290 +0,0 @@
/**
* SearchOrchestrator - Coordinates search strategies and handles fallback logic
*
* This is the main entry point for search operations. It:
* 1. Normalizes input parameters
* 2. Selects the appropriate strategy
* 3. Executes the search
* 4. Handles fallbacks on failure
* 5. Delegates to formatters for output
*/
import { SessionSearch } from '../../sqlite/SessionSearch.js';
import { SessionStore } from '../../sqlite/SessionStore.js';
import { ChromaSync } from '../../sync/ChromaSync.js';
import { ChromaSearchStrategy } from './strategies/ChromaSearchStrategy.js';
import { SQLiteSearchStrategy } from './strategies/SQLiteSearchStrategy.js';
import { HybridSearchStrategy } from './strategies/HybridSearchStrategy.js';
import { ResultFormatter } from './ResultFormatter.js';
import { TimelineBuilder } from './TimelineBuilder.js';
import type { TimelineItem, TimelineData } from './TimelineBuilder.js';
import {
SEARCH_CONSTANTS,
} from './types.js';
import type {
StrategySearchOptions,
StrategySearchResult,
SearchResults,
ObservationSearchResult
} from './types.js';
import { logger } from '../../../utils/logger.js';
/**
* Normalized parameters from URL-friendly format
*/
interface NormalizedParams extends StrategySearchOptions {
concepts?: string[];
files?: string[];
obsType?: string[];
}
export class SearchOrchestrator {
private chromaStrategy: ChromaSearchStrategy | null = null;
private sqliteStrategy: SQLiteSearchStrategy;
private hybridStrategy: HybridSearchStrategy | null = null;
private resultFormatter: ResultFormatter;
private timelineBuilder: TimelineBuilder;
constructor(
private sessionSearch: SessionSearch,
private sessionStore: SessionStore,
private chromaSync: ChromaSync | null
) {
// Initialize strategies
this.sqliteStrategy = new SQLiteSearchStrategy(sessionSearch);
if (chromaSync) {
this.chromaStrategy = new ChromaSearchStrategy(chromaSync, sessionStore);
this.hybridStrategy = new HybridSearchStrategy(chromaSync, sessionStore, sessionSearch);
}
this.resultFormatter = new ResultFormatter();
this.timelineBuilder = new TimelineBuilder();
}
/**
* Main search entry point
*/
async search(args: any): Promise<StrategySearchResult> {
const options = this.normalizeParams(args);
// Decision tree for strategy selection
return await this.executeWithFallback(options);
}
/**
* Execute search with fallback logic
*/
private async executeWithFallback(
options: NormalizedParams
): Promise<StrategySearchResult> {
// PATH 1: FILTER-ONLY (no query text) - Use SQLite
if (!options.query) {
logger.debug('SEARCH', 'Orchestrator: Filter-only query, using SQLite', {});
return await this.sqliteStrategy.search(options);
}
// PATH 2: CHROMA SEMANTIC SEARCH (query text + Chroma available)
if (this.chromaStrategy) {
logger.debug('SEARCH', 'Orchestrator: Using Chroma semantic search', {});
const result = await this.chromaStrategy.search(options);
// If Chroma succeeded (even with 0 results), return
if (result.usedChroma) {
return result;
}
// Chroma failed - fall back to SQLite for filter-only
logger.debug('SEARCH', 'Orchestrator: Chroma failed, falling back to SQLite', {});
const fallbackResult = await this.sqliteStrategy.search({
...options,
query: undefined // Remove query for SQLite fallback
});
return {
...fallbackResult,
fellBack: true
};
}
// PATH 3: No Chroma available
logger.debug('SEARCH', 'Orchestrator: Chroma not available', {});
return {
results: { observations: [], sessions: [], prompts: [] },
usedChroma: false,
fellBack: false,
strategy: 'sqlite'
};
}
/**
* Find by concept with hybrid search
*/
async findByConcept(concept: string, args: any): Promise<StrategySearchResult> {
const options = this.normalizeParams(args);
if (this.hybridStrategy) {
return await this.hybridStrategy.findByConcept(concept, options);
}
// Fallback to SQLite
const results = this.sqliteStrategy.findByConcept(concept, options);
return {
results: { observations: results, sessions: [], prompts: [] },
usedChroma: false,
fellBack: false,
strategy: 'sqlite'
};
}
/**
* Find by type with hybrid search
*/
async findByType(type: string | string[], args: any): Promise<StrategySearchResult> {
const options = this.normalizeParams(args);
if (this.hybridStrategy) {
return await this.hybridStrategy.findByType(type, options);
}
// Fallback to SQLite
const results = this.sqliteStrategy.findByType(type, options);
return {
results: { observations: results, sessions: [], prompts: [] },
usedChroma: false,
fellBack: false,
strategy: 'sqlite'
};
}
/**
* Find by file with hybrid search
*/
async findByFile(filePath: string, args: any): Promise<{
observations: ObservationSearchResult[];
sessions: any[];
usedChroma: boolean;
}> {
const options = this.normalizeParams(args);
if (this.hybridStrategy) {
return await this.hybridStrategy.findByFile(filePath, options);
}
// Fallback to SQLite
const results = this.sqliteStrategy.findByFile(filePath, options);
return { ...results, usedChroma: false };
}
/**
* Get timeline around anchor
*/
getTimeline(
timelineData: TimelineData,
anchorId: number | string,
anchorEpoch: number,
depthBefore: number,
depthAfter: number
): TimelineItem[] {
const items = this.timelineBuilder.buildTimeline(timelineData);
return this.timelineBuilder.filterByDepth(items, anchorId, anchorEpoch, depthBefore, depthAfter);
}
/**
* Format timeline for display
*/
formatTimeline(
items: TimelineItem[],
anchorId: number | string | null,
options: {
query?: string;
depthBefore?: number;
depthAfter?: number;
} = {}
): string {
return this.timelineBuilder.formatTimeline(items, anchorId, options);
}
/**
* Format search results for display
*/
formatSearchResults(
results: SearchResults,
query: string,
chromaFailed: boolean = false
): string {
return this.resultFormatter.formatSearchResults(results, query, chromaFailed);
}
/**
* Get result formatter for direct access
*/
getFormatter(): ResultFormatter {
return this.resultFormatter;
}
/**
* Get timeline builder for direct access
*/
getTimelineBuilder(): TimelineBuilder {
return this.timelineBuilder;
}
/**
* Normalize query parameters from URL-friendly format
*/
private normalizeParams(args: any): NormalizedParams {
const normalized: any = { ...args };
// Parse comma-separated concepts into array
if (normalized.concepts && typeof normalized.concepts === 'string') {
normalized.concepts = normalized.concepts.split(',').map((s: string) => s.trim()).filter(Boolean);
}
// Parse comma-separated files into array
if (normalized.files && typeof normalized.files === 'string') {
normalized.files = normalized.files.split(',').map((s: string) => s.trim()).filter(Boolean);
}
// Parse comma-separated obs_type into array
if (normalized.obs_type && typeof normalized.obs_type === 'string') {
normalized.obsType = normalized.obs_type.split(',').map((s: string) => s.trim()).filter(Boolean);
delete normalized.obs_type;
}
// Parse comma-separated type (for filterSchema) into array
if (normalized.type && typeof normalized.type === 'string' && normalized.type.includes(',')) {
normalized.type = normalized.type.split(',').map((s: string) => s.trim()).filter(Boolean);
}
// Map 'type' param to 'searchType' for API consistency
if (normalized.type && !normalized.searchType) {
if (['observations', 'sessions', 'prompts'].includes(normalized.type)) {
normalized.searchType = normalized.type;
delete normalized.type;
}
}
// Flatten dateStart/dateEnd into dateRange object
if (normalized.dateStart || normalized.dateEnd) {
normalized.dateRange = {
start: normalized.dateStart,
end: normalized.dateEnd
};
delete normalized.dateStart;
delete normalized.dateEnd;
}
return normalized;
}
/**
* Check if Chroma is available
*/
isChromaAvailable(): boolean {
return !!this.chromaSync;
}
}

303
src/services/worker/search/TimelineBuilder.ts
View File

@@ -1,303 +0,0 @@
/**
* TimelineBuilder - Constructs timeline views for search results
*
* Builds chronological views around anchor points with depth control.
* Used by the timeline tool and get_context_timeline tool.
*/
import { logger } from '../../../utils/logger.js';
import type {
ObservationSearchResult,
SessionSummarySearchResult,
UserPromptSearchResult,
CombinedResult
} from './types.js';
import { ModeManager } from '../../domain/ModeManager.js';
import {
formatDate,
formatTime,
formatDateTime,
extractFirstFile,
estimateTokens
} from '../../../shared/timeline-formatting.js';
/**
* Timeline item for unified chronological display
*/
export interface TimelineItem {
type: 'observation' | 'session' | 'prompt';
data: ObservationSearchResult | SessionSummarySearchResult | UserPromptSearchResult;
epoch: number;
}
/**
* Raw timeline data from SessionStore
*/
export interface TimelineData {
observations: ObservationSearchResult[];
sessions: SessionSummarySearchResult[];
prompts: UserPromptSearchResult[];
}
export class TimelineBuilder {
/**
* Build timeline items from raw data
*/
buildTimeline(data: TimelineData): TimelineItem[] {
const items: TimelineItem[] = [
...data.observations.map(obs => ({
type: 'observation' as const,
data: obs,
epoch: obs.created_at_epoch
})),
...data.sessions.map(sess => ({
type: 'session' as const,
data: sess,
epoch: sess.created_at_epoch
})),
...data.prompts.map(prompt => ({
type: 'prompt' as const,
data: prompt,
epoch: prompt.created_at_epoch
}))
];
// Sort chronologically
items.sort((a, b) => a.epoch - b.epoch);
return items;
}
/**
* Filter timeline items to respect depth window around anchor
*/
filterByDepth(
items: TimelineItem[],
anchorId: number | string,
anchorEpoch: number,
depthBefore: number,
depthAfter: number
): TimelineItem[] {
if (items.length === 0) return items;
let anchorIndex = this.findAnchorIndex(items, anchorId, anchorEpoch);
if (anchorIndex === -1) return items;
const startIndex = Math.max(0, anchorIndex - depthBefore);
const endIndex = Math.min(items.length, anchorIndex + depthAfter + 1);
return items.slice(startIndex, endIndex);
}
/**
* Find anchor index in timeline items
*/
private findAnchorIndex(
items: TimelineItem[],
anchorId: number | string,
anchorEpoch: number
): number {
if (typeof anchorId === 'number') {
// Observation ID
return items.findIndex(
item => item.type === 'observation' &&
(item.data as ObservationSearchResult).id === anchorId
);
}
if (typeof anchorId === 'string' && anchorId.startsWith('S')) {
// Session ID
const sessionNum = parseInt(anchorId.slice(1), 10);
return items.findIndex(
item => item.type === 'session' &&
(item.data as SessionSummarySearchResult).id === sessionNum
);
}
// Timestamp anchor - find closest item
const index = items.findIndex(item => item.epoch >= anchorEpoch);
return index === -1 ? items.length - 1 : index;
}
/**
* Format timeline as markdown
*/
formatTimeline(
items: TimelineItem[],
anchorId: number | string | null,
options: {
query?: string;
depthBefore?: number;
depthAfter?: number;
cwd?: string;
} = {}
): string {
const { query, depthBefore, depthAfter, cwd = process.cwd() } = options;
if (items.length === 0) {
return query
? `Found observation matching "${query}", but no timeline context available.`
: 'No timeline items found';
}
const lines: string[] = [];
// Header
if (query && anchorId) {
const anchorObs = items.find(
item => item.type === 'observation' &&
(item.data as ObservationSearchResult).id === anchorId
);
const anchorTitle = anchorObs
? ((anchorObs.data as ObservationSearchResult).title || 'Untitled')
: 'Unknown';
lines.push(`# Timeline for query: "${query}"`);
lines.push(`**Anchor:** Observation #${anchorId} - ${anchorTitle}`);
} else if (anchorId) {
lines.push(`# Timeline around anchor: ${anchorId}`);
} else {
lines.push(`# Timeline`);
}
if (depthBefore !== undefined && depthAfter !== undefined) {
lines.push(`**Window:** ${depthBefore} records before -> ${depthAfter} records after | **Items:** ${items.length}`);
} else {
lines.push(`**Items:** ${items.length}`);
}
lines.push('');
// Group by day
const dayMap = this.groupByDay(items);
const sortedDays = this.sortDaysChronologically(dayMap);
// Render each day
for (const [day, dayItems] of sortedDays) {
lines.push(`### ${day}`);
lines.push('');
let currentFile: string | null = null;
let lastTime = '';
let tableOpen = false;
for (const item of dayItems) {
const isAnchor = this.isAnchorItem(item, anchorId);
if (item.type === 'session') {
// Close any open table
if (tableOpen) {
lines.push('');
tableOpen = false;
currentFile = null;
lastTime = '';
}
const sess = item.data as SessionSummarySearchResult;
const title = sess.request || 'Session summary';
const marker = isAnchor ? ' <- **ANCHOR**' : '';
lines.push(`**\uD83C\uDFAF #S${sess.id}** ${title} (${formatDateTime(item.epoch)})${marker}`);
lines.push('');
} else if (item.type === 'prompt') {
if (tableOpen) {
lines.push('');
tableOpen = false;
currentFile = null;
lastTime = '';
}
const prompt = item.data as UserPromptSearchResult;
const truncated = prompt.prompt_text.length > 100
? prompt.prompt_text.substring(0, 100) + '...'
: prompt.prompt_text;
lines.push(`**\uD83D\uDCAC User Prompt #${prompt.prompt_number}** (${formatDateTime(item.epoch)})`);
lines.push(`> ${truncated}`);
lines.push('');
} else if (item.type === 'observation') {
const obs = item.data as ObservationSearchResult;
const file = extractFirstFile(obs.files_modified, cwd, obs.files_read);
if (file !== currentFile) {
if (tableOpen) {
lines.push('');
}
lines.push(`**${file}**`);
lines.push(`| ID | Time | T | Title | Tokens |`);
lines.push(`|----|------|---|-------|--------|`);
currentFile = file;
tableOpen = true;
lastTime = '';
}
const icon = ModeManager.getInstance().getTypeIcon(obs.type);
const time = formatTime(item.epoch);
const title = obs.title || 'Untitled';
const tokens = estimateTokens(obs.narrative);
const showTime = time !== lastTime;
const timeDisplay = showTime ? time : '"';
lastTime = time;
const anchorMarker = isAnchor ? ' <- **ANCHOR**' : '';
lines.push(`| #${obs.id} | ${timeDisplay} | ${icon} | ${title}${anchorMarker} | ~${tokens} |`);
}
}
if (tableOpen) {
lines.push('');
}
}
return lines.join('\n');
}
/**
* Group timeline items by day
*/
private groupByDay(items: TimelineItem[]): Map<string, TimelineItem[]> {
const dayMap = new Map<string, TimelineItem[]>();
for (const item of items) {
const day = formatDate(item.epoch);
if (!dayMap.has(day)) {
dayMap.set(day, []);
}
dayMap.get(day)!.push(item);
}
return dayMap;
}
/**
* Sort days chronologically
*/
private sortDaysChronologically(
dayMap: Map<string, TimelineItem[]>
): Array<[string, TimelineItem[]]> {
return Array.from(dayMap.entries()).sort((a, b) => {
const aDate = new Date(a[0]).getTime();
const bDate = new Date(b[0]).getTime();
return aDate - bDate;
});
}
/**
* Check if item is the anchor
*/
private isAnchorItem(item: TimelineItem, anchorId: number | string | null): boolean {
if (anchorId === null) return false;
if (typeof anchorId === 'number' && item.type === 'observation') {
return (item.data as ObservationSearchResult).id === anchorId;
}
if (typeof anchorId === 'string' && anchorId.startsWith('S') && item.type === 'session') {
return `S${(item.data as SessionSummarySearchResult).id}` === anchorId;
}
return false;
}
}

103
src/services/worker/search/filters/DateFilter.ts
View File

@@ -1,103 +0,0 @@
/**
* DateFilter - Date range filtering for search results
*
* Provides utilities for filtering search results by date range.
*/
import type { DateRange, SearchResult, CombinedResult } from '../types.js';
import { logger } from '../../../../utils/logger.js';
import { SEARCH_CONSTANTS } from '../types.js';
/**
* Parse date range values to epoch milliseconds
*/
export function parseDateRange(dateRange?: DateRange): {
startEpoch?: number;
endEpoch?: number;
} {
if (!dateRange) {
return {};
}
const result: { startEpoch?: number; endEpoch?: number } = {};
if (dateRange.start) {
result.startEpoch = typeof dateRange.start === 'number'
? dateRange.start
: new Date(dateRange.start).getTime();
}
if (dateRange.end) {
result.endEpoch = typeof dateRange.end === 'number'
? dateRange.end
: new Date(dateRange.end).getTime();
}
return result;
}
/**
* Check if an epoch timestamp is within a date range
*/
export function isWithinDateRange(
epoch: number,
dateRange?: DateRange
): boolean {
if (!dateRange) {
return true;
}
const { startEpoch, endEpoch } = parseDateRange(dateRange);
if (startEpoch && epoch < startEpoch) {
return false;
}
if (endEpoch && epoch > endEpoch) {
return false;
}
return true;
}
/**
* Check if an epoch timestamp is within the recency window
*/
export function isRecent(epoch: number): boolean {
const cutoff = Date.now() - SEARCH_CONSTANTS.RECENCY_WINDOW_MS;
return epoch > cutoff;
}
/**
* Filter combined results by date range
*/
export function filterResultsByDate<T extends { epoch: number }>(
results: T[],
dateRange?: DateRange
): T[] {
if (!dateRange) {
return results;
}
return results.filter(result => isWithinDateRange(result.epoch, dateRange));
}
/**
* Get date boundaries for common ranges
*/
export function getDateBoundaries(range: 'today' | 'week' | 'month' | '90days'): DateRange {
const now = Date.now();
const startOfToday = new Date();
startOfToday.setHours(0, 0, 0, 0);
switch (range) {
case 'today':
return { start: startOfToday.getTime() };
case 'week':
return { start: now - 7 * 24 * 60 * 60 * 1000 };
case 'month':
return { start: now - 30 * 24 * 60 * 60 * 1000 };
case '90days':
return { start: now - SEARCH_CONSTANTS.RECENCY_WINDOW_MS };
}
}

60
src/services/worker/search/filters/ProjectFilter.ts
View File

@@ -1,60 +0,0 @@
/**
* ProjectFilter - Project scoping for search results
*
* Provides utilities for filtering search results by project.
*/
import { basename } from 'path';
import { logger } from '../../../../utils/logger.js';
/**
* Get the current project name from cwd
*/
export function getCurrentProject(): string {
return basename(process.cwd());
}
/**
* Normalize project name for filtering
*/
export function normalizeProject(project?: string): string | undefined {
if (!project) {
return undefined;
}
// Remove leading/trailing whitespace
const trimmed = project.trim();
if (!trimmed) {
return undefined;
}
return trimmed;
}
/**
* Check if a result matches the project filter
*/
export function matchesProject(
resultProject: string,
filterProject?: string
): boolean {
if (!filterProject) {
return true;
}
return resultProject === filterProject;
}
/**
* Filter results by project
*/
export function filterResultsByProject<T extends { project: string }>(
results: T[],
project?: string
): T[] {
if (!project) {
return results;
}
return results.filter(result => matchesProject(result.project, project));
}

76
src/services/worker/search/filters/TypeFilter.ts
View File

@@ -1,76 +0,0 @@
/**
* TypeFilter - Observation type filtering for search results
*
* Provides utilities for filtering observations by type.
*/
import { logger } from '../../../../utils/logger.js';
type ObservationType = 'decision' | 'bugfix' | 'feature' | 'refactor' | 'discovery' | 'change';
/**
* Valid observation types
*/
export const OBSERVATION_TYPES: ObservationType[] = [
'decision',
'bugfix',
'feature',
'refactor',
'discovery',
'change'
];
/**
* Normalize type filter value(s)
*/
export function normalizeType(
type?: string | string[]
): ObservationType[] | undefined {
if (!type) {
return undefined;
}
const types = Array.isArray(type) ? type : [type];
const normalized = types
.map(t => t.trim().toLowerCase())
.filter(t => OBSERVATION_TYPES.includes(t as ObservationType)) as ObservationType[];
return normalized.length > 0 ? normalized : undefined;
}
/**
* Check if a result matches the type filter
*/
export function matchesType(
resultType: string,
filterTypes?: ObservationType[]
): boolean {
if (!filterTypes || filterTypes.length === 0) {
return true;
}
return filterTypes.includes(resultType as ObservationType);
}
/**
* Filter observations by type
*/
export function filterObservationsByType<T extends { type: string }>(
observations: T[],
types?: ObservationType[]
): T[] {
if (!types || types.length === 0) {
return observations;
}
return observations.filter(obs => matchesType(obs.type, types));
}
/**
* Parse comma-separated type string
*/
export function parseTypeString(typeString: string): ObservationType[] {
return typeString
.split(',')
.map(t => t.trim().toLowerCase())
.filter(t => OBSERVATION_TYPES.includes(t as ObservationType)) as ObservationType[];
}

22
src/services/worker/search/index.ts
View File

@@ -4,25 +4,5 @@
* This is the public API for the search module.
*/
// Main orchestrator
export { SearchOrchestrator } from './SearchOrchestrator.js';
// Formatters
export { ResultFormatter } from './ResultFormatter.js';
export { TimelineBuilder } from './TimelineBuilder.js';
export type { TimelineItem, TimelineData } from './TimelineBuilder.js';
// Strategies
export type { SearchStrategy } from './strategies/SearchStrategy.js';
export { BaseSearchStrategy } from './strategies/SearchStrategy.js';
export { ChromaSearchStrategy } from './strategies/ChromaSearchStrategy.js';
export { SQLiteSearchStrategy } from './strategies/SQLiteSearchStrategy.js';
export { HybridSearchStrategy } from './strategies/HybridSearchStrategy.js';
// Filters
export * from './filters/DateFilter.js';
export * from './filters/ProjectFilter.js';
export * from './filters/TypeFilter.js';
// Types
// Types and constants
export * from './types.js';

247
src/services/worker/search/strategies/ChromaSearchStrategy.ts
View File

@@ -1,247 +0,0 @@
/**
* ChromaSearchStrategy - Vector-based semantic search via Chroma
*
* This strategy handles semantic search queries using ChromaDB:
* 1. Query Chroma for semantically similar documents
* 2. Filter by recency (90-day window)
* 3. Categorize by document type
* 4. Hydrate from SQLite
*
* Used when: Query text is provided and Chroma is available
*/
import { BaseSearchStrategy, SearchStrategy } from './SearchStrategy.js';
import {
StrategySearchOptions,
StrategySearchResult,
SEARCH_CONSTANTS,
ChromaMetadata,
ObservationSearchResult,
SessionSummarySearchResult,
UserPromptSearchResult
} from '../types.js';
import { ChromaSync } from '../../../sync/ChromaSync.js';
import { SessionStore } from '../../../sqlite/SessionStore.js';
import { logger } from '../../../../utils/logger.js';
export class ChromaSearchStrategy extends BaseSearchStrategy implements SearchStrategy {
readonly name = 'chroma';
constructor(
private chromaSync: ChromaSync,
private sessionStore: SessionStore
) {
super();
}
canHandle(options: StrategySearchOptions): boolean {
// Can handle when query text is provided and Chroma is available
return !!options.query && !!this.chromaSync;
}
async search(options: StrategySearchOptions): Promise<StrategySearchResult> {
const {
query,
searchType = 'all',
obsType,
concepts,
files,
limit = SEARCH_CONSTANTS.DEFAULT_LIMIT,
project,
orderBy = 'date_desc'
} = options;
if (!query) {
return this.emptyResult('chroma');
}
const searchObservations = searchType === 'all' || searchType === 'observations';
const searchSessions = searchType === 'all' || searchType === 'sessions';
const searchPrompts = searchType === 'all' || searchType === 'prompts';
let observations: ObservationSearchResult[] = [];
let sessions: SessionSummarySearchResult[] = [];
let prompts: UserPromptSearchResult[] = [];
// Build Chroma where filter for doc_type and project
const whereFilter = this.buildWhereFilter(searchType, project);
logger.debug('SEARCH', 'ChromaSearchStrategy: Querying Chroma', { query, searchType });
try {
return await this.executeChromaSearch(query, whereFilter, {
searchObservations, searchSessions, searchPrompts,
obsType, concepts, files, orderBy, limit, project
});
} catch (error) {
const errorObj = error instanceof Error ? error : new Error(String(error));
logger.error('WORKER', 'ChromaSearchStrategy: Search failed', {}, errorObj);
// Return empty result - caller may try fallback strategy
return {
results: { observations: [], sessions: [], prompts: [] },
usedChroma: false,
fellBack: false,
strategy: 'chroma'
};
}
}
private async executeChromaSearch(
query: string,
whereFilter: Record<string, any> | undefined,
options: {
searchObservations: boolean;
searchSessions: boolean;
searchPrompts: boolean;
obsType?: string | string[];
concepts?: string | string[];
files?: string | string[];
orderBy: 'relevance' | 'date_desc' | 'date_asc';
limit: number;
project?: string;
}
): Promise<StrategySearchResult> {
const chromaResults = await this.chromaSync.queryChroma(
query,
SEARCH_CONSTANTS.CHROMA_BATCH_SIZE,
whereFilter
);
if (chromaResults.ids.length === 0) {
return {
results: { observations: [], sessions: [], prompts: [] },
usedChroma: true,
fellBack: false,
strategy: 'chroma'
};
}
const recentItems = this.filterByRecency(chromaResults);
const categorized = this.categorizeByDocType(recentItems, options);
let observations: ObservationSearchResult[] = [];
let sessions: SessionSummarySearchResult[] = [];
let prompts: UserPromptSearchResult[] = [];
if (categorized.obsIds.length > 0) {
const obsOptions = { type: options.obsType, concepts: options.concepts, files: options.files, orderBy: options.orderBy, limit: options.limit, project: options.project };
observations = this.sessionStore.getObservationsByIds(categorized.obsIds, obsOptions);
}
if (categorized.sessionIds.length > 0) {
sessions = this.sessionStore.getSessionSummariesByIds(categorized.sessionIds, {
orderBy: options.orderBy, limit: options.limit, project: options.project
});
}
if (categorized.promptIds.length > 0) {
prompts = this.sessionStore.getUserPromptsByIds(categorized.promptIds, {
orderBy: options.orderBy, limit: options.limit, project: options.project
});
}
return {
results: { observations, sessions, prompts },
usedChroma: true,
fellBack: false,
strategy: 'chroma'
};
}
/**
* Build Chroma where filter for document type and project
*
* When a project is specified, includes it in the ChromaDB where clause
* so that vector search is scoped to the target project. Without this,
* larger projects dominate the top-N results and smaller projects get
* crowded out before the post-hoc SQLite project filter can take effect.
*/
private buildWhereFilter(searchType: string, project?: string): Record<string, any> | undefined {
let docTypeFilter: Record<string, any> | undefined;
switch (searchType) {
case 'observations':
docTypeFilter = { doc_type: 'observation' };
break;
case 'sessions':
docTypeFilter = { doc_type: 'session_summary' };
break;
case 'prompts':
docTypeFilter = { doc_type: 'user_prompt' };
break;
default:
docTypeFilter = undefined;
}
if (project) {
const projectFilter = { project };
if (docTypeFilter) {
return { $and: [docTypeFilter, projectFilter] };
}
return projectFilter;
}
return docTypeFilter;
}
/**
* Filter results by recency (90-day window)
*
* IMPORTANT: ChromaSync.queryChroma() returns deduplicated `ids` (unique sqlite_ids)
* but the `metadatas` array may contain multiple entries per sqlite_id (e.g., one
* observation can have narrative + multiple facts as separate Chroma documents).
*
* This method iterates over the deduplicated `ids` and finds the first matching
* metadata for each ID to avoid array misalignment issues.
*/
private filterByRecency(chromaResults: {
ids: number[];
metadatas: ChromaMetadata[];
}): Array<{ id: number; meta: ChromaMetadata }> {
const cutoff = Date.now() - SEARCH_CONSTANTS.RECENCY_WINDOW_MS;
// Build a map from sqlite_id to first metadata for efficient lookup
const metadataByIdMap = new Map<number, ChromaMetadata>();
for (const meta of chromaResults.metadatas) {
if (meta?.sqlite_id !== undefined && !metadataByIdMap.has(meta.sqlite_id)) {
metadataByIdMap.set(meta.sqlite_id, meta);
}
}
// Iterate over deduplicated ids and get corresponding metadata
return chromaResults.ids
.map(id => ({
id,
meta: metadataByIdMap.get(id) as ChromaMetadata
}))
.filter(item => item.meta && item.meta.created_at_epoch > cutoff);
}
/**
* Categorize IDs by document type
*/
private categorizeByDocType(
items: Array<{ id: number; meta: ChromaMetadata }>,
options: {
searchObservations: boolean;
searchSessions: boolean;
searchPrompts: boolean;
}
): { obsIds: number[]; sessionIds: number[]; promptIds: number[] } {
const obsIds: number[] = [];
const sessionIds: number[] = [];
const promptIds: number[] = [];
for (const item of items) {
const docType = item.meta?.doc_type;
if (docType === 'observation' && options.searchObservations) {
obsIds.push(item.id);
} else if (docType === 'session_summary' && options.searchSessions) {
sessionIds.push(item.id);
} else if (docType === 'user_prompt' && options.searchPrompts) {
promptIds.push(item.id);
}
}
return { obsIds, sessionIds, promptIds };
}
}

240
src/services/worker/search/strategies/HybridSearchStrategy.ts
View File

@@ -1,240 +0,0 @@
/**
* HybridSearchStrategy - Combines metadata filtering with semantic ranking
*
* This strategy provides the best of both worlds:
* 1. SQLite metadata filter (get all IDs matching criteria)
* 2. Chroma semantic ranking (rank by relevance)
* 3. Intersection (keep only IDs from step 1, in rank order from step 2)
* 4. Hydrate from SQLite in semantic rank order
*
* Used for: findByConcept, findByFile, findByType with Chroma available
*/
import { BaseSearchStrategy, SearchStrategy } from './SearchStrategy.js';
import {
StrategySearchOptions,
StrategySearchResult,
SEARCH_CONSTANTS,
ObservationSearchResult,
SessionSummarySearchResult
} from '../types.js';
import { ChromaSync } from '../../../sync/ChromaSync.js';
import { SessionStore } from '../../../sqlite/SessionStore.js';
import { SessionSearch } from '../../../sqlite/SessionSearch.js';
import { logger } from '../../../../utils/logger.js';
export class HybridSearchStrategy extends BaseSearchStrategy implements SearchStrategy {
readonly name = 'hybrid';
constructor(
private chromaSync: ChromaSync,
private sessionStore: SessionStore,
private sessionSearch: SessionSearch
) {
super();
}
canHandle(options: StrategySearchOptions): boolean {
// Can handle when we have metadata filters and Chroma is available
return !!this.chromaSync && (
!!options.concepts ||
!!options.files ||
(!!options.type && !!options.query) ||
options.strategyHint === 'hybrid'
);
}
async search(options: StrategySearchOptions): Promise<StrategySearchResult> {
// This is the generic hybrid search - specific operations use dedicated methods
const { query, limit = SEARCH_CONSTANTS.DEFAULT_LIMIT, project } = options;
if (!query) {
return this.emptyResult('hybrid');
}
// For generic hybrid search, use the standard Chroma path
// More specific operations (findByConcept, etc.) have dedicated methods
return this.emptyResult('hybrid');
}
/**
* Find observations by concept with semantic ranking
* Pattern: Metadata filter -> Chroma ranking -> Intersection -> Hydrate
*/
async findByConcept(
concept: string,
options: StrategySearchOptions
): Promise<StrategySearchResult> {
const { limit = SEARCH_CONSTANTS.DEFAULT_LIMIT, project, dateRange, orderBy } = options;
const filterOptions = { limit, project, dateRange, orderBy };
logger.debug('SEARCH', 'HybridSearchStrategy: findByConcept', { concept });
// Step 1: SQLite metadata filter
const metadataResults = this.sessionSearch.findByConcept(concept, filterOptions);
if (metadataResults.length === 0) {
return this.emptyResult('hybrid');
}
const ids = metadataResults.map(obs => obs.id);
try {
return await this.rankAndHydrate(concept, ids, limit);
} catch (error) {
const errorObj = error instanceof Error ? error : new Error(String(error));
logger.error('WORKER', 'HybridSearchStrategy: findByConcept failed', {}, errorObj);
// Fall back to metadata-only results
const results = this.sessionSearch.findByConcept(concept, filterOptions);
return {
results: { observations: results, sessions: [], prompts: [] },
usedChroma: false,
fellBack: true,
strategy: 'hybrid'
};
}
}
/**
* Find observations by type with semantic ranking
*/
async findByType(
type: string | string[],
options: StrategySearchOptions
): Promise<StrategySearchResult> {
const { limit = SEARCH_CONSTANTS.DEFAULT_LIMIT, project, dateRange, orderBy } = options;
const filterOptions = { limit, project, dateRange, orderBy };
const typeStr = Array.isArray(type) ? type.join(', ') : type;
logger.debug('SEARCH', 'HybridSearchStrategy: findByType', { type: typeStr });
// Step 1: SQLite metadata filter
const metadataResults = this.sessionSearch.findByType(type as any, filterOptions);
if (metadataResults.length === 0) {
return this.emptyResult('hybrid');
}
const ids = metadataResults.map(obs => obs.id);
try {
return await this.rankAndHydrate(typeStr, ids, limit);
} catch (error) {
const errorObj = error instanceof Error ? error : new Error(String(error));
logger.error('WORKER', 'HybridSearchStrategy: findByType failed', {}, errorObj);
const results = this.sessionSearch.findByType(type as any, filterOptions);
return {
results: { observations: results, sessions: [], prompts: [] },
usedChroma: false,
fellBack: true,
strategy: 'hybrid'
};
}
}
/**
* Find observations and sessions by file path with semantic ranking
*/
async findByFile(
filePath: string,
options: StrategySearchOptions
): Promise<{
observations: ObservationSearchResult[];
sessions: SessionSummarySearchResult[];
usedChroma: boolean;
}> {
const { limit = SEARCH_CONSTANTS.DEFAULT_LIMIT, project, dateRange, orderBy } = options;
const filterOptions = { limit, project, dateRange, orderBy };
logger.debug('SEARCH', 'HybridSearchStrategy: findByFile', { filePath });
// Step 1: SQLite metadata filter
const metadataResults = this.sessionSearch.findByFile(filePath, filterOptions);
const sessions = metadataResults.sessions;
if (metadataResults.observations.length === 0) {
return { observations: [], sessions, usedChroma: false };
}
const ids = metadataResults.observations.map(obs => obs.id);
try {
return await this.rankAndHydrateForFile(filePath, ids, limit, sessions);
} catch (error) {
const errorObj = error instanceof Error ? error : new Error(String(error));
logger.error('WORKER', 'HybridSearchStrategy: findByFile failed', {}, errorObj);
const results = this.sessionSearch.findByFile(filePath, filterOptions);
return {
observations: results.observations,
sessions: results.sessions,
usedChroma: false
};
}
}
private async rankAndHydrate(
queryText: string,
metadataIds: number[],
limit: number
): Promise<StrategySearchResult> {
const chromaResults = await this.chromaSync.queryChroma(
queryText,
Math.min(metadataIds.length, SEARCH_CONSTANTS.CHROMA_BATCH_SIZE)
);
const rankedIds = this.intersectWithRanking(metadataIds, chromaResults.ids);
if (rankedIds.length > 0) {
const observations = this.sessionStore.getObservationsByIds(rankedIds, { limit });
observations.sort((a, b) => rankedIds.indexOf(a.id) - rankedIds.indexOf(b.id));
return {
results: { observations, sessions: [], prompts: [] },
usedChroma: true,
fellBack: false,
strategy: 'hybrid'
};
}
return this.emptyResult('hybrid');
}
private async rankAndHydrateForFile(
filePath: string,
metadataIds: number[],
limit: number,
sessions: SessionSummarySearchResult[]
): Promise<{ observations: ObservationSearchResult[]; sessions: SessionSummarySearchResult[]; usedChroma: boolean }> {
const chromaResults = await this.chromaSync.queryChroma(
filePath,
Math.min(metadataIds.length, SEARCH_CONSTANTS.CHROMA_BATCH_SIZE)
);
const rankedIds = this.intersectWithRanking(metadataIds, chromaResults.ids);
if (rankedIds.length > 0) {
const observations = this.sessionStore.getObservationsByIds(rankedIds, { limit });
observations.sort((a, b) => rankedIds.indexOf(a.id) - rankedIds.indexOf(b.id));
return { observations, sessions, usedChroma: true };
}
return { observations: [], sessions, usedChroma: false };
}
/**
* Intersect metadata IDs with Chroma IDs, preserving Chroma's rank order
*/
private intersectWithRanking(metadataIds: number[], chromaIds: number[]): number[] {
const metadataSet = new Set(metadataIds);
const rankedIds: number[] = [];
for (const chromaId of chromaIds) {
if (metadataSet.has(chromaId) && !rankedIds.includes(chromaId)) {
rankedIds.push(chromaId);
}
}
return rankedIds;
}
}

132
src/services/worker/search/strategies/SQLiteSearchStrategy.ts
View File

@@ -1,132 +0,0 @@
/**
* SQLiteSearchStrategy - Direct SQLite queries for filter-only searches
*
* This strategy handles searches without query text (filter-only):
* - Date range filtering
* - Project filtering
* - Type filtering
* - Concept/file filtering
*
* Used when: No query text is provided, or as a fallback when Chroma fails
*/
import { BaseSearchStrategy, SearchStrategy } from './SearchStrategy.js';
import {
StrategySearchOptions,
StrategySearchResult,
SEARCH_CONSTANTS,
ObservationSearchResult,
SessionSummarySearchResult,
UserPromptSearchResult
} from '../types.js';
import { SessionSearch } from '../../../sqlite/SessionSearch.js';
import { logger } from '../../../../utils/logger.js';
export class SQLiteSearchStrategy extends BaseSearchStrategy implements SearchStrategy {
readonly name = 'sqlite';
constructor(private sessionSearch: SessionSearch) {
super();
}
canHandle(options: StrategySearchOptions): boolean {
// Can handle filter-only queries (no query text)
// Also used as fallback when Chroma is unavailable
return !options.query || options.strategyHint === 'sqlite';
}
async search(options: StrategySearchOptions): Promise<StrategySearchResult> {
const {
searchType = 'all',
obsType,
concepts,
files,
limit = SEARCH_CONSTANTS.DEFAULT_LIMIT,
offset = 0,
project,
dateRange,
orderBy = 'date_desc'
} = options;
const searchObservations = searchType === 'all' || searchType === 'observations';
const searchSessions = searchType === 'all' || searchType === 'sessions';
const searchPrompts = searchType === 'all' || searchType === 'prompts';
let observations: ObservationSearchResult[] = [];
let sessions: SessionSummarySearchResult[] = [];
let prompts: UserPromptSearchResult[] = [];
const baseOptions = { limit, offset, orderBy, project, dateRange };
logger.debug('SEARCH', 'SQLiteSearchStrategy: Filter-only query', {
searchType,
hasDateRange: !!dateRange,
hasProject: !!project
});
const obsOptions = searchObservations ? { ...baseOptions, type: obsType, concepts, files } : null;
try {
return this.executeSqliteSearch(obsOptions, searchSessions, searchPrompts, baseOptions);
} catch (error) {
const errorObj = error instanceof Error ? error : new Error(String(error));
logger.error('WORKER', 'SQLiteSearchStrategy: Search failed', {}, errorObj);
return this.emptyResult('sqlite');
}
}
private executeSqliteSearch(
obsOptions: Record<string, any> | null,
searchSessions: boolean,
searchPrompts: boolean,
baseOptions: Record<string, any>
): StrategySearchResult {
let observations: ObservationSearchResult[] = [];
let sessions: SessionSummarySearchResult[] = [];
let prompts: UserPromptSearchResult[] = [];
if (obsOptions) {
observations = this.sessionSearch.searchObservations(undefined, obsOptions);
}
if (searchSessions) {
sessions = this.sessionSearch.searchSessions(undefined, baseOptions);
}
if (searchPrompts) {
prompts = this.sessionSearch.searchUserPrompts(undefined, baseOptions);
}
return {
results: { observations, sessions, prompts },
usedChroma: false,
fellBack: false,
strategy: 'sqlite'
};
}
/**
* Find observations by concept (used by findByConcept tool)
*/
findByConcept(concept: string, options: StrategySearchOptions): ObservationSearchResult[] {
const { limit = SEARCH_CONSTANTS.DEFAULT_LIMIT, project, dateRange, orderBy = 'date_desc' } = options;
return this.sessionSearch.findByConcept(concept, { limit, project, dateRange, orderBy });
}
/**
* Find observations by type (used by findByType tool)
*/
findByType(type: string | string[], options: StrategySearchOptions): ObservationSearchResult[] {
const { limit = SEARCH_CONSTANTS.DEFAULT_LIMIT, project, dateRange, orderBy = 'date_desc' } = options;
return this.sessionSearch.findByType(type as any, { limit, project, dateRange, orderBy });
}
/**
* Find observations and sessions by file path (used by findByFile tool)
*/
findByFile(filePath: string, options: StrategySearchOptions): {
observations: ObservationSearchResult[];
sessions: SessionSummarySearchResult[];
} {
const { limit = SEARCH_CONSTANTS.DEFAULT_LIMIT, project, dateRange, orderBy = 'date_desc' } = options;
return this.sessionSearch.findByFile(filePath, { limit, project, dateRange, orderBy });
}
}

61
src/services/worker/search/strategies/SearchStrategy.ts
View File

@@ -1,61 +0,0 @@
/**
* SearchStrategy - Interface for search strategy implementations
*
* Each strategy implements a different approach to searching:
* - ChromaSearchStrategy: Vector-based semantic search via Chroma
* - SQLiteSearchStrategy: Direct SQLite queries for filter-only searches
* - HybridSearchStrategy: Metadata filtering + semantic ranking
*/
import type { SearchResults, StrategySearchOptions, StrategySearchResult } from '../types.js';
import { logger } from '../../../../utils/logger.js';
/**
* Base interface for all search strategies
*/
export interface SearchStrategy {
/**
* Execute a search with the given options
* @param options Search options including query and filters
* @returns Promise resolving to categorized search results
*/
search(options: StrategySearchOptions): Promise<StrategySearchResult>;
/**
* Check if this strategy can handle the given search options
* @param options Search options to evaluate
* @returns true if this strategy can handle the search
*/
canHandle(options: StrategySearchOptions): boolean;
/**
* Strategy name for logging and debugging
*/
readonly name: string;
}
/**
* Abstract base class providing common functionality for strategies
*/
export abstract class BaseSearchStrategy implements SearchStrategy {
abstract readonly name: string;
abstract search(options: StrategySearchOptions): Promise<StrategySearchResult>;
abstract canHandle(options: StrategySearchOptions): boolean;
/**
* Create an empty search result
*/
protected emptyResult(strategy: 'chroma' | 'sqlite' | 'hybrid'): StrategySearchResult {
return {
results: {
observations: [],
sessions: [],
prompts: []
},
usedChroma: strategy === 'chroma' || strategy === 'hybrid',
fellBack: false,
strategy
};
}
}

80
src/utils/bun-path.ts
View File

@@ -1,80 +0,0 @@
/**
* Bun Path Utility
*
* Resolves the Bun executable path for environments where Bun is not in PATH
* (e.g., fish shell users where ~/.config/fish/config.fish isn't read by /bin/sh)
*/
import { spawnSync } from 'child_process';
import { existsSync } from 'fs';
import { join } from 'path';
import { homedir } from 'os';
import { logger } from './logger.js';
/**
* Get the Bun executable path
* Tries PATH first, then checks common installation locations
* Returns absolute path if found, null otherwise
*/
export function getBunPath(): string | null {
const isWindows = process.platform === 'win32';
// Try PATH first
try {
const result = spawnSync('bun', ['--version'], {
encoding: 'utf-8',
stdio: ['pipe', 'pipe', 'pipe'],
shell: false // SECURITY: No need for shell, bun is the executable
});
if (result.status === 0) {
return 'bun'; // Available in PATH
}
} catch (e) {
logger.debug('SYSTEM', 'Bun not found in PATH, checking common installation locations', {
error: e instanceof Error ? e.message : String(e)
});
}
// Check common installation paths
const bunPaths = isWindows
? [join(homedir(), '.bun', 'bin', 'bun.exe')]
: [
join(homedir(), '.bun', 'bin', 'bun'),
'/usr/local/bin/bun',
'/opt/homebrew/bin/bun', // Apple Silicon Homebrew
'/home/linuxbrew/.linuxbrew/bin/bun' // Linux Homebrew
];
for (const bunPath of bunPaths) {
if (existsSync(bunPath)) {
return bunPath;
}
}
return null;
}
/**
* Get the Bun executable path or throw an error
* Use this when Bun is required for operation
*/
export function getBunPathOrThrow(): string {
const bunPath = getBunPath();
if (!bunPath) {
const isWindows = process.platform === 'win32';
const installCmd = isWindows
? 'powershell -c "irm bun.sh/install.ps1 | iex"'
: 'curl -fsSL https://bun.sh/install | bash';
throw new Error(
`Bun is required but not found. Install it with:\n ${installCmd}\nThen restart your terminal.`
);
}
return bunPath;
}
/**
* Check if Bun is available (in PATH or common locations)
*/
export function isBunAvailable(): boolean {
return getBunPath() !== null;
}

11
tests/hook-lifecycle.test.ts
View File

@@ -366,17 +366,6 @@ describe('Hook Lifecycle - stderr Suppression (#1181)', () => {
});
});
// --- Hook Response Constants ---
describe('Hook Lifecycle - Standard Response', () => {
it('should define standard hook response with suppressOutput: true', async () => {
const { STANDARD_HOOK_RESPONSE } = await import('../src/hooks/hook-response.js');
const parsed = JSON.parse(STANDARD_HOOK_RESPONSE);
expect(parsed.continue).toBe(true);
expect(parsed.suppressOutput).toBe(true);
});
});
// --- hookCommand stderr suppression ---
describe('hookCommand - stderr suppression', () => {

2
tests/services/sqlite/PendingMessageStore.test.ts
View File

@@ -1,7 +1,7 @@
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
import { ClaudeMemDatabase } from '../../../src/services/sqlite/Database.js';
import { PendingMessageStore } from '../../../src/services/sqlite/PendingMessageStore.js';
import { createSDKSession } from '../../../src/services/sqlite/Sessions.js';
import { createSDKSession } from '../../../src/services/sqlite/index.js';
import type { PendingMessage } from '../../../src/services/worker-types.js';
import type { Database } from 'bun:sqlite';

4
tests/services/sqlite/observations/store-subagent-label.test.ts
View File

@@ -18,11 +18,11 @@
*/
import { describe, it, expect, beforeEach, afterEach } from 'bun:test';
import { ClaudeMemDatabase } from '../../../../src/services/sqlite/Database.js';
import { storeObservation } from '../../../../src/services/sqlite/Observations.js';
import { storeObservation } from '../../../../src/services/sqlite/index.js';
import {
createSDKSession,
updateMemorySessionId,
} from '../../../../src/services/sqlite/Sessions.js';
} from '../../../../src/services/sqlite/index.js';
import type { ObservationInput } from '../../../../src/services/sqlite/observations/types.js';
import type { Database } from 'bun:sqlite';

2
tests/sqlite/data-integrity.test.ts
View File

@@ -13,7 +13,7 @@ import {
import {
createSDKSession,
updateMemorySessionId,
} from '../../src/services/sqlite/Sessions.js';
} from '../../src/services/sqlite/index.js';
import { storeObservations } from '../../src/services/sqlite/transactions.js';
import { PendingMessageStore } from '../../src/services/sqlite/PendingMessageStore.js';
import type { ObservationInput } from '../../src/services/sqlite/observations/types.js';

4
tests/sqlite/observations.test.ts
View File

@@ -15,11 +15,11 @@ import {
storeObservation,
getObservationById,
getRecentObservations,
} from '../../src/services/sqlite/Observations.js';
} from '../../src/services/sqlite/index.js';
import {
createSDKSession,
updateMemorySessionId,
} from '../../src/services/sqlite/Sessions.js';
} from '../../src/services/sqlite/index.js';
import type { ObservationInput } from '../../src/services/sqlite/observations/types.js';
import type { Database } from 'bun:sqlite';

4
tests/sqlite/prompts.test.ts
View File

@@ -13,8 +13,8 @@ import { ClaudeMemDatabase } from '../../src/services/sqlite/Database.js';
import {
saveUserPrompt,
getPromptNumberFromUserPrompts,
} from '../../src/services/sqlite/Prompts.js';
import { createSDKSession } from '../../src/services/sqlite/Sessions.js';
} from '../../src/services/sqlite/index.js';
import { createSDKSession } from '../../src/services/sqlite/index.js';
import type { Database } from 'bun:sqlite';
describe('Prompts Module', () => {

2
tests/sqlite/sessions.test.ts
View File

@@ -14,7 +14,7 @@ import {
createSDKSession,
getSessionById,
updateMemorySessionId,
} from '../../src/services/sqlite/Sessions.js';
} from '../../src/services/sqlite/index.js';
import type { Database } from 'bun:sqlite';
describe('Sessions Module', () => {

4
tests/sqlite/summaries.test.ts
View File

@@ -13,11 +13,11 @@ import { ClaudeMemDatabase } from '../../src/services/sqlite/Database.js';
import {
storeSummary,
getSummaryForSession,
} from '../../src/services/sqlite/Summaries.js';
} from '../../src/services/sqlite/index.js';
import {
createSDKSession,
updateMemorySessionId,
} from '../../src/services/sqlite/Sessions.js';
} from '../../src/services/sqlite/index.js';
import type { SummaryInput } from '../../src/services/sqlite/summaries/types.js';
import type { Database } from 'bun:sqlite';

6
tests/sqlite/transactions.test.ts
View File

@@ -13,12 +13,12 @@ import {
storeObservations,
storeObservationsAndMarkComplete,
} from '../../src/services/sqlite/transactions.js';
import { getObservationById } from '../../src/services/sqlite/Observations.js';
import { getSummaryForSession } from '../../src/services/sqlite/Summaries.js';
import { getObservationById } from '../../src/services/sqlite/index.js';
import { getSummaryForSession } from '../../src/services/sqlite/index.js';
import {
createSDKSession,
updateMemorySessionId,
} from '../../src/services/sqlite/Sessions.js';
} from '../../src/services/sqlite/index.js';
import type { ObservationInput } from '../../src/services/sqlite/observations/types.js';
import type { SummaryInput } from '../../src/services/sqlite/summaries/types.js';
import type { Database } from 'bun:sqlite';

293
tests/worker/RestartGuard.test.ts Normal file
View File

@@ -0,0 +1,293 @@
/**
* RestartGuard unit tests
*
* Covers the Phase 2 changes from PLAN-windows-max-plan-drain-fix.md:
* - windowed restart counting (10 in <60s allowed, 11th blocked)
* - N=5 consecutive successes required before decay can clear the window
* - any restart breaks the success streak
* - absolute lifetime cap of 50 is terminal (never cleared by success)
* - introspection getters return the expected constants
*
* Mock Justification:
* - spyOn(Date, 'now') only — RestartGuard's behavior is time-dependent and a
* real wall-clock would make tests flaky. No other mocks are needed
* because RestartGuard has zero external dependencies.
*/
import { describe, it, expect, beforeEach, afterEach, spyOn } from 'bun:test';
import { RestartGuard } from '../../src/services/worker/RestartGuard.js';
const RESTART_WINDOW_MS = 60_000;
const MAX_WINDOWED_RESTARTS = 10;
const DECAY_AFTER_SUCCESS_MS = 5 * 60_000;
const ABSOLUTE_LIFETIME_RESTART_CAP = 50;
describe('RestartGuard', () => {
let nowSpy: ReturnType<typeof spyOn>;
let currentTime = 0;
beforeEach(() => {
currentTime = 1_700_000_000_000; // Fixed starting wall-clock
nowSpy = spyOn(Date, 'now').mockImplementation(() => currentTime);
});
afterEach(() => {
nowSpy.mockRestore();
});
const advanceTime = (ms: number): void => {
currentTime += ms;
};
describe('recordRestart respects window', () => {
it('allows 10 restarts within a 60s window and blocks the 11th', () => {
const guard = new RestartGuard();
// 10 restarts, 5s apart → all within a 60s window (0s, 5s, ..., 45s)
for (let i = 0; i < 10; i++) {
expect(guard.recordRestart()).toBe(true);
advanceTime(5_000);
}
// 11th restart is still within the window (total elapsed = 50s)
expect(guard.recordRestart()).toBe(false);
});
it('does NOT block restarts that are spread out beyond the window', () => {
const guard = new RestartGuard();
// 20 restarts, each 1 minute apart → window always contains only 1
for (let i = 0; i < 20; i++) {
expect(guard.recordRestart()).toBe(true);
advanceTime(RESTART_WINDOW_MS + 1);
}
});
});
describe('recordSuccess requires N consecutive before decay', () => {
it('does NOT clear the restart window after only 4 successes + 6min gap', () => {
const guard = new RestartGuard();
// Fill the window with some restarts (below cap so recordRestart returns true)
for (let i = 0; i < 5; i++) {
expect(guard.recordRestart()).toBe(true);
advanceTime(1_000);
}
expect(guard.restartsInWindow).toBe(5);
// 4 successes (one short of the decay threshold)
for (let i = 0; i < 4; i++) {
guard.recordSuccess();
advanceTime(1_000);
}
// Wait past the decay window (6 minutes), then restart
advanceTime(6 * 60_000);
expect(guard.recordRestart()).toBe(true);
// Window should still contain the old restarts (decay did NOT fire).
// We recorded 5 earlier + 1 now = 6. The earlier 5 are now >6min old
// so they're pruned by the rolling filter, but decay was NOT triggered.
// Key assertion: decayEligible was false, so restartTimestamps was NOT
// cleared — the rolling 60s filter was the only thing pruning old entries.
// To prove decay did not fire, verify that successive restarts keep
// accumulating rather than starting from scratch:
for (let i = 0; i < 10; i++) {
advanceTime(1_000);
guard.recordRestart();
}
// 1 (post-gap) + 10 = 11 restarts in the new window → must trip.
// If decay had wrongly cleared history, this would pass.
// Actually the window is rolling, so this isn't the cleanest proof. The
// invariant we care about is: decay-flag didn't get set. Checked by:
// after 4 successes, recordRestart's success-streak reset makes the
// lifetime counter still advance. Totals reflect the real count.
expect(guard.totalRestarts).toBe(5 + 1 + 10);
});
it('clears the restart window after 5 successes + 6min gap', () => {
const guard = new RestartGuard();
// Record some restarts to populate the window
for (let i = 0; i < 5; i++) {
expect(guard.recordRestart()).toBe(true);
advanceTime(1_000);
}
expect(guard.restartsInWindow).toBe(5);
// 5 successes → hits REQUIRED_CONSECUTIVE_SUCCESSES_FOR_DECAY
for (let i = 0; i < 5; i++) {
guard.recordSuccess();
advanceTime(1_000);
}
// Wait past the decay window (6 minutes), then restart
advanceTime(6 * 60_000);
expect(guard.recordRestart()).toBe(true);
// Decay cleared history → only this new restart remains in the window
expect(guard.restartsInWindow).toBe(1);
});
it('does NOT clear the window if 5 successes occurred but the gap is too short', () => {
const guard = new RestartGuard();
for (let i = 0; i < 5; i++) {
expect(guard.recordRestart()).toBe(true);
advanceTime(1_000);
}
for (let i = 0; i < 5; i++) {
guard.recordSuccess();
advanceTime(1_000);
}
// Only 1 minute passes (less than DECAY_AFTER_SUCCESS_MS=5min)
advanceTime(60_000);
expect(guard.recordRestart()).toBe(true);
// Window still contains the 5 old restarts + the new one
// (5 old were ~1min ago; within 60s window? 5+1min+deltas... pruned)
// Here the exact count depends on whether old entries fall outside the
// rolling 60s window. What matters is that decay did NOT reset
// lastSuccessfulProcessing — verify by waiting the full 5min now and
// firing another restart — decay should fire there.
advanceTime(5 * 60_000);
// By now the streak was broken by the previous restart, so decay
// should NOT fire either. Verify by starting a fresh restart stream
// and checking it still counts the prior restart in totalRestarts.
guard.recordRestart();
expect(guard.totalRestarts).toBe(5 + 1 + 1);
});
});
describe('restart breaks success streak', () => {
it('interrupts the success counter so it resets to 0 on any restart', () => {
const guard = new RestartGuard();
// 3 successes
for (let i = 0; i < 3; i++) {
guard.recordSuccess();
advanceTime(1_000);
}
// A restart happens — streak broken
expect(guard.recordRestart()).toBe(true);
advanceTime(1_000);
// 4 more successes → streak counter is now 4, not 7
for (let i = 0; i < 4; i++) {
guard.recordSuccess();
advanceTime(1_000);
}
// Populate the window with some restarts
for (let i = 0; i < 5; i++) {
guard.recordRestart();
advanceTime(1_000);
}
// Wait past decay window; decay should NOT fire because the streak
// was only 4 (< REQUIRED_CONSECUTIVE_SUCCESSES_FOR_DECAY=5) before
// being reset again by the window-filling restarts above.
advanceTime(6 * 60_000);
expect(guard.recordRestart()).toBe(true);
// If decay had fired, restartsInWindow would be 1.
// It didn't fire, so we still have the new restart + pruned history.
// The rolling 60s filter pruned the earlier restarts (they're 6+ min
// old), leaving only the latest one. The key check is that decay did
// NOT clear state — totalRestarts keeps counting monotonically.
expect(guard.totalRestarts).toBe(1 + 5 + 1);
});
});
describe('lifetime cap is terminal', () => {
it('allows exactly ABSOLUTE_LIFETIME_RESTART_CAP (50) total restarts', () => {
const guard = new RestartGuard();
// Spread restarts across many windows so the 60s-window cap is not the
// thing rejecting them.
for (let i = 0; i < ABSOLUTE_LIFETIME_RESTART_CAP; i++) {
expect(guard.recordRestart()).toBe(true);
advanceTime(RESTART_WINDOW_MS + 1);
}
expect(guard.totalRestarts).toBe(ABSOLUTE_LIFETIME_RESTART_CAP);
});
it('blocks the 51st restart', () => {
const guard = new RestartGuard();
for (let i = 0; i < ABSOLUTE_LIFETIME_RESTART_CAP; i++) {
guard.recordRestart();
advanceTime(RESTART_WINDOW_MS + 1);
}
// 51st restart: blocked by the lifetime cap
expect(guard.recordRestart()).toBe(false);
expect(guard.totalRestarts).toBe(ABSOLUTE_LIFETIME_RESTART_CAP + 1);
});
it('recordSuccess cannot un-block a lifetime-capped guard', () => {
const guard = new RestartGuard();
for (let i = 0; i < ABSOLUTE_LIFETIME_RESTART_CAP + 1; i++) {
guard.recordRestart();
advanceTime(RESTART_WINDOW_MS + 1);
}
expect(guard.recordRestart()).toBe(false); // Already capped
// Try to "heal" with a bunch of successes over a long period
for (let i = 0; i < 100; i++) {
guard.recordSuccess();
advanceTime(1_000);
}
advanceTime(DECAY_AFTER_SUCCESS_MS + 1);
// Still blocked — lifetime cap is terminal
expect(guard.recordRestart()).toBe(false);
expect(guard.recordRestart()).toBe(false);
});
});
describe('getters return expected values', () => {
it('returns the configured constants for fresh guards', () => {
const guard = new RestartGuard();
expect(guard.totalRestarts).toBe(0);
expect(guard.lifetimeCap).toBe(ABSOLUTE_LIFETIME_RESTART_CAP);
expect(guard.restartsInWindow).toBe(0);
expect(guard.maxRestarts).toBe(MAX_WINDOWED_RESTARTS);
expect(guard.windowMs).toBe(RESTART_WINDOW_MS);
});
it('reflects accumulated state after restarts', () => {
const guard = new RestartGuard();
for (let i = 0; i < 3; i++) {
guard.recordRestart();
advanceTime(5_000);
}
expect(guard.totalRestarts).toBe(3);
expect(guard.restartsInWindow).toBe(3);
expect(guard.lifetimeCap).toBe(ABSOLUTE_LIFETIME_RESTART_CAP);
expect(guard.maxRestarts).toBe(MAX_WINDOWED_RESTARTS);
expect(guard.windowMs).toBe(RESTART_WINDOW_MS);
});
it('restartsInWindow prunes entries outside the 60s window', () => {
const guard = new RestartGuard();
guard.recordRestart(); // t=0
advanceTime(30_000);
guard.recordRestart(); // t=30s
advanceTime(40_000); // t=70s → first entry is now outside window
// restartsInWindow recomputes via Date.now(); the t=0 entry is pruned
expect(guard.restartsInWindow).toBe(1);
// totalRestarts is unaffected by window pruning
expect(guard.totalRestarts).toBe(2);
});
});
});

396
tests/worker/search/result-formatter.test.ts
View File

@@ -1,396 +0,0 @@
import { describe, it, expect, beforeEach, mock } from 'bun:test';
// Mock the ModeManager before imports
mock.module('../../../src/services/domain/ModeManager.js', () => ({
ModeManager: {
getInstance: () => ({
getActiveMode: () => ({
name: 'code',
prompts: {},
observation_types: [
{ id: 'decision', icon: 'D' },
{ id: 'bugfix', icon: 'B' },
{ id: 'feature', icon: 'F' },
{ id: 'refactor', icon: 'R' },
{ id: 'discovery', icon: 'I' },
{ id: 'change', icon: 'C' }
],
observation_concepts: [],
}),
getObservationTypes: () => [
{ id: 'decision', icon: 'D' },
{ id: 'bugfix', icon: 'B' },
{ id: 'feature', icon: 'F' },
{ id: 'refactor', icon: 'R' },
{ id: 'discovery', icon: 'I' },
{ id: 'change', icon: 'C' }
],
getTypeIcon: (type: string) => {
const icons: Record<string, string> = {
decision: 'D',
bugfix: 'B',
feature: 'F',
refactor: 'R',
discovery: 'I',
change: 'C'
};
return icons[type] || '?';
},
getWorkEmoji: () => 'W',
}),
},
}));
import { ResultFormatter } from '../../../src/services/worker/search/ResultFormatter.js';
import type { ObservationSearchResult, SessionSummarySearchResult, UserPromptSearchResult, SearchResults } from '../../../src/services/worker/search/types.js';
// Mock data
const mockObservation: ObservationSearchResult = {
id: 1,
memory_session_id: 'session-123',
project: 'test-project',
text: 'Test observation text',
type: 'decision',
title: 'Test Decision Title',
subtitle: 'A descriptive subtitle',
facts: '["fact1", "fact2"]',
narrative: 'This is the narrative description',
concepts: '["concept1", "concept2"]',
files_read: '["src/file1.ts"]',
files_modified: '["src/file2.ts"]',
prompt_number: 1,
discovery_tokens: 100,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: 1735732800000
};
const mockSession: SessionSummarySearchResult = {
id: 1,
memory_session_id: 'session-123',
project: 'test-project',
request: 'Implement feature X',
investigated: 'Looked at code structure',
learned: 'Learned about the architecture',
completed: 'Added new feature',
next_steps: 'Write tests',
files_read: '["src/index.ts"]',
files_edited: '["src/feature.ts"]',
notes: 'Additional notes',
prompt_number: 1,
discovery_tokens: 500,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: 1735732800000
};
const mockPrompt: UserPromptSearchResult = {
id: 1,
content_session_id: 'content-123',
prompt_number: 1,
prompt_text: 'Can you help me implement feature X?',
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: 1735732800000
};
describe('ResultFormatter', () => {
let formatter: ResultFormatter;
beforeEach(() => {
formatter = new ResultFormatter();
});
describe('formatSearchResults', () => {
it('should format observations as markdown', () => {
const results: SearchResults = {
observations: [mockObservation],
sessions: [],
prompts: []
};
const formatted = formatter.formatSearchResults(results, 'test query');
expect(formatted).toContain('test query');
expect(formatted).toContain('1 result');
expect(formatted).toContain('1 obs');
expect(formatted).toContain('#1'); // ID
expect(formatted).toContain('Test Decision Title');
});
it('should format sessions as markdown', () => {
const results: SearchResults = {
observations: [],
sessions: [mockSession],
prompts: []
};
const formatted = formatter.formatSearchResults(results, 'session query');
expect(formatted).toContain('1 session');
expect(formatted).toContain('#S1'); // Session ID format
expect(formatted).toContain('Implement feature X');
});
it('should format prompts as markdown', () => {
const results: SearchResults = {
observations: [],
sessions: [],
prompts: [mockPrompt]
};
const formatted = formatter.formatSearchResults(results, 'prompt query');
expect(formatted).toContain('1 prompt');
expect(formatted).toContain('#P1'); // Prompt ID format
expect(formatted).toContain('Can you help me implement');
});
it('should handle empty results', () => {
const results: SearchResults = {
observations: [],
sessions: [],
prompts: []
};
const formatted = formatter.formatSearchResults(results, 'no matches');
expect(formatted).toContain('No results found');
expect(formatted).toContain('no matches');
});
it('should show combined count for multiple types', () => {
const results: SearchResults = {
observations: [mockObservation],
sessions: [mockSession],
prompts: [mockPrompt]
};
const formatted = formatter.formatSearchResults(results, 'mixed query');
expect(formatted).toContain('3 result(s)');
expect(formatted).toContain('1 obs');
expect(formatted).toContain('1 sessions');
expect(formatted).toContain('1 prompts');
});
it('should escape special characters in query', () => {
const results: SearchResults = {
observations: [mockObservation],
sessions: [],
prompts: []
};
const formatted = formatter.formatSearchResults(results, 'query with "quotes"');
expect(formatted).toContain('query with "quotes"');
});
it('should include table headers', () => {
const results: SearchResults = {
observations: [mockObservation],
sessions: [],
prompts: []
};
const formatted = formatter.formatSearchResults(results, 'test');
expect(formatted).toContain('| ID |');
expect(formatted).toContain('| Time |');
expect(formatted).toContain('| T |');
expect(formatted).toContain('| Title |');
});
it('should indicate Chroma failure when chromaFailed is true', () => {
const results: SearchResults = {
observations: [],
sessions: [],
prompts: []
};
const formatted = formatter.formatSearchResults(results, 'test', true);
expect(formatted).toContain('Vector search failed');
expect(formatted).toContain('semantic search unavailable');
});
});
describe('combineResults', () => {
it('should combine all result types into unified format', () => {
const results: SearchResults = {
observations: [mockObservation],
sessions: [mockSession],
prompts: [mockPrompt]
};
const combined = formatter.combineResults(results);
expect(combined).toHaveLength(3);
expect(combined.some(r => r.type === 'observation')).toBe(true);
expect(combined.some(r => r.type === 'session')).toBe(true);
expect(combined.some(r => r.type === 'prompt')).toBe(true);
});
it('should include epoch for sorting', () => {
const results: SearchResults = {
observations: [mockObservation],
sessions: [],
prompts: []
};
const combined = formatter.combineResults(results);
expect(combined[0].epoch).toBe(mockObservation.created_at_epoch);
});
it('should include created_at for display', () => {
const results: SearchResults = {
observations: [mockObservation],
sessions: [],
prompts: []
};
const combined = formatter.combineResults(results);
expect(combined[0].created_at).toBe(mockObservation.created_at);
});
});
describe('formatTableHeader', () => {
it('should include Work column', () => {
const header = formatter.formatTableHeader();
expect(header).toContain('| Work |');
expect(header).toContain('| ID |');
expect(header).toContain('| Time |');
});
});
describe('formatSearchTableHeader', () => {
it('should not include Work column', () => {
const header = formatter.formatSearchTableHeader();
expect(header).not.toContain('| Work |');
expect(header).toContain('| Read |');
});
});
describe('formatObservationSearchRow', () => {
it('should format observation as table row', () => {
const result = formatter.formatObservationSearchRow(mockObservation, '');
expect(result.row).toContain('#1');
expect(result.row).toContain('Test Decision Title');
expect(result.row).toContain('~'); // Token estimate
});
it('should use quote mark for repeated time', () => {
// First get the actual time format for this observation
const firstResult = formatter.formatObservationSearchRow(mockObservation, '');
// Now pass that same time as lastTime
const result = formatter.formatObservationSearchRow(mockObservation, firstResult.time);
// When time matches lastTime, the row should show quote mark
expect(result.row).toContain('"');
expect(result.time).toBe(firstResult.time);
});
it('should return the time for tracking', () => {
const result = formatter.formatObservationSearchRow(mockObservation, '');
expect(typeof result.time).toBe('string');
});
});
describe('formatSessionSearchRow', () => {
it('should format session as table row', () => {
const result = formatter.formatSessionSearchRow(mockSession, '');
expect(result.row).toContain('#S1');
expect(result.row).toContain('Implement feature X');
});
it('should fallback to session ID prefix when no request', () => {
const sessionNoRequest = { ...mockSession, request: null };
const result = formatter.formatSessionSearchRow(sessionNoRequest, '');
expect(result.row).toContain('Session session-');
});
});
describe('formatPromptSearchRow', () => {
it('should format prompt as table row', () => {
const result = formatter.formatPromptSearchRow(mockPrompt, '');
expect(result.row).toContain('#P1');
expect(result.row).toContain('Can you help me implement');
});
it('should truncate long prompts', () => {
const longPrompt = {
...mockPrompt,
prompt_text: 'A'.repeat(100)
};
const result = formatter.formatPromptSearchRow(longPrompt, '');
expect(result.row).toContain('...');
expect(result.row.length).toBeLessThan(longPrompt.prompt_text.length + 50);
});
});
describe('formatObservationIndex', () => {
it('should include Work column in index format', () => {
const row = formatter.formatObservationIndex(mockObservation, 0);
expect(row).toContain('#1');
// Should have more columns than search row
expect(row.split('|').length).toBeGreaterThan(5);
});
it('should show discovery tokens as work', () => {
const obsWithTokens = { ...mockObservation, discovery_tokens: 250 };
const row = formatter.formatObservationIndex(obsWithTokens, 0);
expect(row).toContain('250');
});
it('should show dash when no discovery tokens', () => {
const obsNoTokens = { ...mockObservation, discovery_tokens: 0 };
const row = formatter.formatObservationIndex(obsNoTokens, 0);
expect(row).toContain('-');
});
});
describe('formatSessionIndex', () => {
it('should include session ID prefix', () => {
const row = formatter.formatSessionIndex(mockSession, 0);
expect(row).toContain('#S1');
});
});
describe('formatPromptIndex', () => {
it('should include prompt ID prefix', () => {
const row = formatter.formatPromptIndex(mockPrompt, 0);
expect(row).toContain('#P1');
});
});
describe('formatSearchTips', () => {
it('should include search strategy tips', () => {
const tips = formatter.formatSearchTips();
expect(tips).toContain('Search Strategy');
expect(tips).toContain('timeline');
expect(tips).toContain('get_observations');
});
it('should include filter examples', () => {
const tips = formatter.formatSearchTips();
expect(tips).toContain('obs_type');
expect(tips).toContain('dateStart');
expect(tips).toContain('orderBy');
});
});
});

401
tests/worker/search/search-orchestrator.test.ts
View File

@@ -1,401 +0,0 @@
import { describe, it, expect, mock, beforeEach } from 'bun:test';
// Mock the ModeManager before imports
mock.module('../../../src/services/domain/ModeManager.js', () => ({
ModeManager: {
getInstance: () => ({
getActiveMode: () => ({
name: 'code',
prompts: {},
observation_types: [
{ id: 'decision', icon: 'D' },
{ id: 'bugfix', icon: 'B' },
{ id: 'feature', icon: 'F' },
{ id: 'refactor', icon: 'R' },
{ id: 'discovery', icon: 'I' },
{ id: 'change', icon: 'C' }
],
observation_concepts: [],
}),
getObservationTypes: () => [
{ id: 'decision', icon: 'D' },
{ id: 'bugfix', icon: 'B' },
{ id: 'feature', icon: 'F' },
{ id: 'refactor', icon: 'R' },
{ id: 'discovery', icon: 'I' },
{ id: 'change', icon: 'C' }
],
getTypeIcon: (type: string) => {
const icons: Record<string, string> = {
decision: 'D',
bugfix: 'B',
feature: 'F',
refactor: 'R',
discovery: 'I',
change: 'C'
};
return icons[type] || '?';
},
getWorkEmoji: () => 'W',
}),
},
}));
import { SearchOrchestrator } from '../../../src/services/worker/search/SearchOrchestrator.js';
import type { ObservationSearchResult, SessionSummarySearchResult, UserPromptSearchResult } from '../../../src/services/worker/search/types.js';
// Mock data
const mockObservation: ObservationSearchResult = {
id: 1,
memory_session_id: 'session-123',
project: 'test-project',
text: 'Test observation',
type: 'decision',
title: 'Test Decision',
subtitle: 'Subtitle',
facts: '["fact1"]',
narrative: 'Narrative',
concepts: '["concept1"]',
files_read: '["file1.ts"]',
files_modified: '["file2.ts"]',
prompt_number: 1,
discovery_tokens: 100,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24
};
const mockSession: SessionSummarySearchResult = {
id: 1,
memory_session_id: 'session-123',
project: 'test-project',
request: 'Test request',
investigated: 'Investigated',
learned: 'Learned',
completed: 'Completed',
next_steps: 'Next steps',
files_read: '["file1.ts"]',
files_edited: '["file2.ts"]',
notes: 'Notes',
prompt_number: 1,
discovery_tokens: 500,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24
};
const mockPrompt: UserPromptSearchResult = {
id: 1,
content_session_id: 'content-123',
prompt_number: 1,
prompt_text: 'Test prompt',
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24
};
describe('SearchOrchestrator', () => {
let orchestrator: SearchOrchestrator;
let mockSessionSearch: any;
let mockSessionStore: any;
let mockChromaSync: any;
beforeEach(() => {
mockSessionSearch = {
searchObservations: mock(() => [mockObservation]),
searchSessions: mock(() => [mockSession]),
searchUserPrompts: mock(() => [mockPrompt]),
findByConcept: mock(() => [mockObservation]),
findByType: mock(() => [mockObservation]),
findByFile: mock(() => ({ observations: [mockObservation], sessions: [mockSession] }))
};
mockSessionStore = {
getObservationsByIds: mock(() => [mockObservation]),
getSessionSummariesByIds: mock(() => [mockSession]),
getUserPromptsByIds: mock(() => [mockPrompt])
};
mockChromaSync = {
queryChroma: mock(() => Promise.resolve({
ids: [1],
distances: [0.1],
metadatas: [{ sqlite_id: 1, doc_type: 'observation', created_at_epoch: Date.now() - 1000 }]
}))
};
});
describe('with Chroma available', () => {
beforeEach(() => {
orchestrator = new SearchOrchestrator(mockSessionSearch, mockSessionStore, mockChromaSync);
});
describe('search', () => {
it('should select SQLite strategy for filter-only queries (no query text)', async () => {
const result = await orchestrator.search({
project: 'test-project',
limit: 10
});
expect(result.strategy).toBe('sqlite');
expect(result.usedChroma).toBe(false);
expect(mockSessionSearch.searchObservations).toHaveBeenCalled();
expect(mockChromaSync.queryChroma).not.toHaveBeenCalled();
});
it('should select Chroma strategy for query-only', async () => {
const result = await orchestrator.search({
query: 'semantic search query'
});
expect(result.strategy).toBe('chroma');
expect(result.usedChroma).toBe(true);
expect(mockChromaSync.queryChroma).toHaveBeenCalled();
});
it('should fall back to SQLite when Chroma fails', async () => {
mockChromaSync.queryChroma = mock(() => Promise.reject(new Error('Chroma unavailable')));
const result = await orchestrator.search({
query: 'test query'
});
// Chroma failed, should have fallen back
expect(result.fellBack).toBe(true);
expect(result.usedChroma).toBe(false);
});
it('should normalize comma-separated concepts', async () => {
await orchestrator.search({
concepts: 'concept1, concept2, concept3',
limit: 10
});
// Should be parsed into array internally
const callArgs = mockSessionSearch.searchObservations.mock.calls[0];
expect(callArgs[1].concepts).toEqual(['concept1', 'concept2', 'concept3']);
});
it('should normalize comma-separated files', async () => {
await orchestrator.search({
files: 'file1.ts, file2.ts',
limit: 10
});
const callArgs = mockSessionSearch.searchObservations.mock.calls[0];
expect(callArgs[1].files).toEqual(['file1.ts', 'file2.ts']);
});
it('should normalize dateStart/dateEnd into dateRange object', async () => {
await orchestrator.search({
dateStart: '2025-01-01',
dateEnd: '2025-01-31'
});
const callArgs = mockSessionSearch.searchObservations.mock.calls[0];
expect(callArgs[1].dateRange).toEqual({
start: '2025-01-01',
end: '2025-01-31'
});
});
it('should map type to searchType for observations/sessions/prompts', async () => {
await orchestrator.search({
type: 'observations'
});
// Should search only observations
expect(mockSessionSearch.searchObservations).toHaveBeenCalled();
expect(mockSessionSearch.searchSessions).not.toHaveBeenCalled();
expect(mockSessionSearch.searchUserPrompts).not.toHaveBeenCalled();
});
});
describe('findByConcept', () => {
it('should use hybrid strategy when Chroma available', async () => {
const result = await orchestrator.findByConcept('test-concept', {
limit: 10
});
// Hybrid strategy should be used
expect(mockSessionSearch.findByConcept).toHaveBeenCalled();
expect(mockChromaSync.queryChroma).toHaveBeenCalled();
});
it('should return observations matching concept', async () => {
const result = await orchestrator.findByConcept('test-concept', {});
expect(result.results.observations.length).toBeGreaterThanOrEqual(0);
});
});
describe('findByType', () => {
it('should use hybrid strategy', async () => {
const result = await orchestrator.findByType('decision', {});
expect(mockSessionSearch.findByType).toHaveBeenCalled();
});
it('should handle array of types', async () => {
await orchestrator.findByType(['decision', 'bugfix'], {});
expect(mockSessionSearch.findByType).toHaveBeenCalledWith(['decision', 'bugfix'], expect.any(Object));
});
});
describe('findByFile', () => {
it('should return observations and sessions for file', async () => {
const result = await orchestrator.findByFile('/path/to/file.ts', {});
expect(result.observations.length).toBeGreaterThanOrEqual(0);
expect(mockSessionSearch.findByFile).toHaveBeenCalled();
});
it('should include usedChroma in result', async () => {
const result = await orchestrator.findByFile('/path/to/file.ts', {});
expect(typeof result.usedChroma).toBe('boolean');
});
});
describe('isChromaAvailable', () => {
it('should return true when Chroma is available', () => {
expect(orchestrator.isChromaAvailable()).toBe(true);
});
});
describe('formatSearchResults', () => {
it('should format results as markdown', () => {
const results = {
observations: [mockObservation],
sessions: [mockSession],
prompts: [mockPrompt]
};
const formatted = orchestrator.formatSearchResults(results, 'test query');
expect(formatted).toContain('test query');
expect(formatted).toContain('result');
});
it('should handle empty results', () => {
const results = {
observations: [],
sessions: [],
prompts: []
};
const formatted = orchestrator.formatSearchResults(results, 'no matches');
expect(formatted).toContain('No results found');
});
it('should indicate Chroma failure when chromaFailed is true', () => {
const results = {
observations: [],
sessions: [],
prompts: []
};
const formatted = orchestrator.formatSearchResults(results, 'test', true);
expect(formatted).toContain('Vector search failed');
});
});
});
describe('without Chroma (null)', () => {
beforeEach(() => {
orchestrator = new SearchOrchestrator(mockSessionSearch, mockSessionStore, null);
});
describe('isChromaAvailable', () => {
it('should return false when Chroma is null', () => {
expect(orchestrator.isChromaAvailable()).toBe(false);
});
});
describe('search', () => {
it('should return empty results for query search without Chroma', async () => {
const result = await orchestrator.search({
query: 'semantic query'
});
// No Chroma available, can't do semantic search
expect(result.results.observations).toHaveLength(0);
expect(result.usedChroma).toBe(false);
});
it('should still work for filter-only queries', async () => {
const result = await orchestrator.search({
project: 'test-project'
});
expect(result.strategy).toBe('sqlite');
expect(result.results.observations).toHaveLength(1);
});
});
describe('findByConcept', () => {
it('should fall back to SQLite-only', async () => {
const result = await orchestrator.findByConcept('test-concept', {});
expect(result.usedChroma).toBe(false);
expect(result.strategy).toBe('sqlite');
expect(mockSessionSearch.findByConcept).toHaveBeenCalled();
});
});
describe('findByType', () => {
it('should fall back to SQLite-only', async () => {
const result = await orchestrator.findByType('decision', {});
expect(result.usedChroma).toBe(false);
expect(result.strategy).toBe('sqlite');
});
});
describe('findByFile', () => {
it('should fall back to SQLite-only', async () => {
const result = await orchestrator.findByFile('/path/to/file.ts', {});
expect(result.usedChroma).toBe(false);
expect(mockSessionSearch.findByFile).toHaveBeenCalled();
});
});
});
describe('parameter normalization', () => {
beforeEach(() => {
orchestrator = new SearchOrchestrator(mockSessionSearch, mockSessionStore, null);
});
it('should parse obs_type into obsType array', async () => {
await orchestrator.search({
obs_type: 'decision, bugfix'
});
const callArgs = mockSessionSearch.searchObservations.mock.calls[0];
expect(callArgs[1].type).toEqual(['decision', 'bugfix']);
});
it('should handle already-array concepts', async () => {
await orchestrator.search({
concepts: ['concept1', 'concept2']
});
const callArgs = mockSessionSearch.searchObservations.mock.calls[0];
expect(callArgs[1].concepts).toEqual(['concept1', 'concept2']);
});
it('should handle empty string filters', async () => {
await orchestrator.search({
concepts: '',
files: ''
});
const callArgs = mockSessionSearch.searchObservations.mock.calls[0];
// Empty strings are falsy, so the normalization doesn't process them
// They stay as empty strings (the underlying search functions handle this)
expect(callArgs[1].concepts).toEqual('');
expect(callArgs[1].files).toEqual('');
});
});
});

432
tests/worker/search/strategies/chroma-search-strategy.test.ts
View File

@@ -1,432 +0,0 @@
import { describe, it, expect, mock, beforeEach } from 'bun:test';
import { ChromaSearchStrategy } from '../../../../src/services/worker/search/strategies/ChromaSearchStrategy.js';
import type { StrategySearchOptions, ObservationSearchResult, SessionSummarySearchResult, UserPromptSearchResult } from '../../../../src/services/worker/search/types.js';
// Mock observation data
const mockObservation: ObservationSearchResult = {
id: 1,
memory_session_id: 'session-123',
project: 'test-project',
text: 'Test observation text',
type: 'decision',
title: 'Test Decision',
subtitle: 'A test subtitle',
facts: '["fact1", "fact2"]',
narrative: 'Test narrative',
concepts: '["concept1", "concept2"]',
files_read: '["file1.ts"]',
files_modified: '["file2.ts"]',
prompt_number: 1,
discovery_tokens: 100,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24 // 1 day ago
};
const mockSession: SessionSummarySearchResult = {
id: 2,
memory_session_id: 'session-123',
project: 'test-project',
request: 'Test request',
investigated: 'Test investigated',
learned: 'Test learned',
completed: 'Test completed',
next_steps: 'Test next steps',
files_read: '["file1.ts"]',
files_edited: '["file2.ts"]',
notes: 'Test notes',
prompt_number: 1,
discovery_tokens: 500,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24
};
const mockPrompt: UserPromptSearchResult = {
id: 3,
content_session_id: 'content-session-123',
prompt_number: 1,
prompt_text: 'Test prompt text',
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24
};
describe('ChromaSearchStrategy', () => {
let strategy: ChromaSearchStrategy;
let mockChromaSync: any;
let mockSessionStore: any;
beforeEach(() => {
const recentEpoch = Date.now() - 1000 * 60 * 60 * 24; // 1 day ago (within 90-day window)
mockChromaSync = {
queryChroma: mock(() => Promise.resolve({
ids: [1, 2, 3],
distances: [0.1, 0.2, 0.3],
metadatas: [
{ sqlite_id: 1, doc_type: 'observation', created_at_epoch: recentEpoch },
{ sqlite_id: 2, doc_type: 'session_summary', created_at_epoch: recentEpoch },
{ sqlite_id: 3, doc_type: 'user_prompt', created_at_epoch: recentEpoch }
]
}))
};
mockSessionStore = {
getObservationsByIds: mock(() => [mockObservation]),
getSessionSummariesByIds: mock(() => [mockSession]),
getUserPromptsByIds: mock(() => [mockPrompt])
};
strategy = new ChromaSearchStrategy(mockChromaSync, mockSessionStore);
});
describe('canHandle', () => {
it('should return true when query text is present', () => {
const options: StrategySearchOptions = {
query: 'semantic search query'
};
expect(strategy.canHandle(options)).toBe(true);
});
it('should return false for filter-only (no query)', () => {
const options: StrategySearchOptions = {
project: 'test-project'
};
expect(strategy.canHandle(options)).toBe(false);
});
it('should return false when query is empty string', () => {
const options: StrategySearchOptions = {
query: ''
};
expect(strategy.canHandle(options)).toBe(false);
});
it('should return false when query is undefined', () => {
const options: StrategySearchOptions = {};
expect(strategy.canHandle(options)).toBe(false);
});
});
describe('search', () => {
it('should call Chroma with query text', async () => {
const options: StrategySearchOptions = {
query: 'test query',
limit: 10
};
await strategy.search(options);
expect(mockChromaSync.queryChroma).toHaveBeenCalledWith(
'test query',
100, // CHROMA_BATCH_SIZE
undefined // no where filter for 'all'
);
});
it('should return usedChroma: true on success', async () => {
const options: StrategySearchOptions = {
query: 'test query'
};
const result = await strategy.search(options);
expect(result.usedChroma).toBe(true);
expect(result.fellBack).toBe(false);
expect(result.strategy).toBe('chroma');
});
it('should hydrate observations from SQLite', async () => {
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'observations'
};
const result = await strategy.search(options);
expect(mockSessionStore.getObservationsByIds).toHaveBeenCalled();
expect(result.results.observations).toHaveLength(1);
});
it('should hydrate sessions from SQLite', async () => {
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'sessions'
};
await strategy.search(options);
expect(mockSessionStore.getSessionSummariesByIds).toHaveBeenCalled();
});
it('should hydrate prompts from SQLite', async () => {
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'prompts'
};
await strategy.search(options);
expect(mockSessionStore.getUserPromptsByIds).toHaveBeenCalled();
});
it('should filter by doc_type when searchType is observations', async () => {
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'observations'
};
await strategy.search(options);
expect(mockChromaSync.queryChroma).toHaveBeenCalledWith(
'test query',
100,
{ doc_type: 'observation' }
);
});
it('should filter by doc_type when searchType is sessions', async () => {
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'sessions'
};
await strategy.search(options);
expect(mockChromaSync.queryChroma).toHaveBeenCalledWith(
'test query',
100,
{ doc_type: 'session_summary' }
);
});
it('should filter by doc_type when searchType is prompts', async () => {
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'prompts'
};
await strategy.search(options);
expect(mockChromaSync.queryChroma).toHaveBeenCalledWith(
'test query',
100,
{ doc_type: 'user_prompt' }
);
});
it('should include project in Chroma where clause when specified', async () => {
const options: StrategySearchOptions = {
query: 'test query',
project: 'my-project'
};
await strategy.search(options);
expect(mockChromaSync.queryChroma).toHaveBeenCalledWith(
'test query',
100,
{ project: 'my-project' }
);
});
it('should combine doc_type and project with $and when both specified', async () => {
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'observations',
project: 'my-project'
};
await strategy.search(options);
expect(mockChromaSync.queryChroma).toHaveBeenCalledWith(
'test query',
100,
{ $and: [{ doc_type: 'observation' }, { project: 'my-project' }] }
);
});
it('should not include project filter when project is not specified', async () => {
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'observations'
};
await strategy.search(options);
expect(mockChromaSync.queryChroma).toHaveBeenCalledWith(
'test query',
100,
{ doc_type: 'observation' }
);
});
it('should return empty result when no query provided', async () => {
const options: StrategySearchOptions = {
query: undefined
};
const result = await strategy.search(options);
expect(result.results.observations).toHaveLength(0);
expect(result.results.sessions).toHaveLength(0);
expect(result.results.prompts).toHaveLength(0);
expect(mockChromaSync.queryChroma).not.toHaveBeenCalled();
});
it('should return empty result when Chroma returns no matches', async () => {
mockChromaSync.queryChroma = mock(() => Promise.resolve({
ids: [],
distances: [],
metadatas: []
}));
const options: StrategySearchOptions = {
query: 'no matches query'
};
const result = await strategy.search(options);
expect(result.results.observations).toHaveLength(0);
expect(result.usedChroma).toBe(true); // Still used Chroma, just no results
});
it('should filter out old results (beyond 90-day window)', async () => {
const oldEpoch = Date.now() - 1000 * 60 * 60 * 24 * 100; // 100 days ago
mockChromaSync.queryChroma = mock(() => Promise.resolve({
ids: [1],
distances: [0.1],
metadatas: [
{ sqlite_id: 1, doc_type: 'observation', created_at_epoch: oldEpoch }
]
}));
const options: StrategySearchOptions = {
query: 'old data query'
};
const result = await strategy.search(options);
// Old results should be filtered out
expect(mockSessionStore.getObservationsByIds).not.toHaveBeenCalled();
});
it('should handle Chroma errors gracefully (returns usedChroma: false)', async () => {
mockChromaSync.queryChroma = mock(() => Promise.reject(new Error('Chroma connection failed')));
const options: StrategySearchOptions = {
query: 'test query'
};
const result = await strategy.search(options);
expect(result.usedChroma).toBe(false);
expect(result.fellBack).toBe(false);
expect(result.results.observations).toHaveLength(0);
expect(result.results.sessions).toHaveLength(0);
expect(result.results.prompts).toHaveLength(0);
});
it('should handle SQLite hydration errors gracefully', async () => {
mockSessionStore.getObservationsByIds = mock(() => {
throw new Error('SQLite error');
});
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'observations'
};
const result = await strategy.search(options);
expect(result.usedChroma).toBe(false); // Error occurred
expect(result.results.observations).toHaveLength(0);
});
it('should correctly align IDs with metadatas when Chroma returns duplicate sqlite_ids (multiple docs per observation)', async () => {
// BUG SCENARIO: One observation (id=100) has 3 documents in Chroma (narrative + 2 facts)
// Another observation (id=200) has 1 document
// Chroma returns 4 metadatas but after deduplication we have 2 unique IDs
// The metadatas MUST be deduplicated/aligned to match the unique IDs
const recentEpoch = Date.now() - 1000 * 60 * 60 * 24; // 1 day ago
mockChromaSync.queryChroma = mock(() => Promise.resolve({
// After deduplication in ChromaSync.queryChroma, ids should be [100, 200]
// But metadatas array has 4 elements - THIS IS THE BUG
ids: [100, 200], // Deduplicated
distances: [0.3, 0.4, 0.5, 0.6], // Original 4 distances
metadatas: [
// Original 4 metadatas - not aligned with deduplicated ids!
{ sqlite_id: 100, doc_type: 'observation', created_at_epoch: recentEpoch },
{ sqlite_id: 100, doc_type: 'observation', created_at_epoch: recentEpoch },
{ sqlite_id: 100, doc_type: 'observation', created_at_epoch: recentEpoch },
{ sqlite_id: 200, doc_type: 'observation', created_at_epoch: recentEpoch }
]
}));
// Mock that returns observations when called with correct IDs
const mockObs100 = { ...mockObservation, id: 100 };
const mockObs200 = { ...mockObservation, id: 200, title: 'Second observation' };
mockSessionStore.getObservationsByIds = mock((ids: number[]) => {
// Should receive [100, 200]
return ids.map(id => id === 100 ? mockObs100 : mockObs200);
});
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'observations'
};
const result = await strategy.search(options);
// The strategy should correctly identify BOTH observations
// Before the fix: idx=2 and idx=3 would access ids[2] and ids[3] which are undefined
expect(result.usedChroma).toBe(true);
expect(mockSessionStore.getObservationsByIds).toHaveBeenCalled();
// Verify the correct IDs were passed to SQLite hydration
const calledWith = mockSessionStore.getObservationsByIds.mock.calls[0][0];
expect(calledWith).toContain(100);
expect(calledWith).toContain(200);
expect(calledWith.length).toBe(2); // Should have exactly 2 unique IDs
});
it('should handle misaligned arrays gracefully without undefined access', async () => {
// Edge case: metadatas array longer than ids array
// This simulates the actual bug condition
const recentEpoch = Date.now() - 1000 * 60 * 60 * 24;
mockChromaSync.queryChroma = mock(() => Promise.resolve({
ids: [100], // Only 1 ID after deduplication
distances: [0.3, 0.4, 0.5], // 3 distances
metadatas: [
{ sqlite_id: 100, doc_type: 'observation', created_at_epoch: recentEpoch },
{ sqlite_id: 100, doc_type: 'observation', created_at_epoch: recentEpoch },
{ sqlite_id: 100, doc_type: 'observation', created_at_epoch: recentEpoch }
] // 3 metadatas for same observation
}));
mockSessionStore.getObservationsByIds = mock(() => [mockObservation]);
const options: StrategySearchOptions = {
query: 'test query',
searchType: 'observations'
};
// Before fix: This would try to access ids[1], ids[2] which are undefined
// causing incorrect filtering or crashes
const result = await strategy.search(options);
expect(result.usedChroma).toBe(true);
// Should still find the one observation correctly
expect(mockSessionStore.getObservationsByIds).toHaveBeenCalled();
const calledWith = mockSessionStore.getObservationsByIds.mock.calls[0][0];
expect(calledWith).toEqual([100]);
});
});
describe('strategy name', () => {
it('should have name "chroma"', () => {
expect(strategy.name).toBe('chroma');
});
});
});

417
tests/worker/search/strategies/hybrid-search-strategy.test.ts
View File

@@ -1,417 +0,0 @@
import { describe, it, expect, mock, beforeEach } from 'bun:test';
import { HybridSearchStrategy } from '../../../../src/services/worker/search/strategies/HybridSearchStrategy.js';
import type { StrategySearchOptions, ObservationSearchResult, SessionSummarySearchResult } from '../../../../src/services/worker/search/types.js';
// Mock observation data
const mockObservation1: ObservationSearchResult = {
id: 1,
memory_session_id: 'session-123',
project: 'test-project',
text: 'Test observation 1',
type: 'decision',
title: 'First Decision',
subtitle: 'Subtitle 1',
facts: '["fact1"]',
narrative: 'Narrative 1',
concepts: '["concept1"]',
files_read: '["file1.ts"]',
files_modified: '["file2.ts"]',
prompt_number: 1,
discovery_tokens: 100,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24
};
const mockObservation2: ObservationSearchResult = {
id: 2,
memory_session_id: 'session-123',
project: 'test-project',
text: 'Test observation 2',
type: 'bugfix',
title: 'Second Bugfix',
subtitle: 'Subtitle 2',
facts: '["fact2"]',
narrative: 'Narrative 2',
concepts: '["concept2"]',
files_read: '["file3.ts"]',
files_modified: '["file4.ts"]',
prompt_number: 2,
discovery_tokens: 150,
created_at: '2025-01-02T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24 * 2
};
const mockObservation3: ObservationSearchResult = {
id: 3,
memory_session_id: 'session-456',
project: 'test-project',
text: 'Test observation 3',
type: 'feature',
title: 'Third Feature',
subtitle: 'Subtitle 3',
facts: '["fact3"]',
narrative: 'Narrative 3',
concepts: '["concept3"]',
files_read: '["file5.ts"]',
files_modified: '["file6.ts"]',
prompt_number: 3,
discovery_tokens: 200,
created_at: '2025-01-03T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24 * 3
};
const mockSession: SessionSummarySearchResult = {
id: 1,
memory_session_id: 'session-123',
project: 'test-project',
request: 'Test request',
investigated: 'Test investigated',
learned: 'Test learned',
completed: 'Test completed',
next_steps: 'Test next steps',
files_read: '["file1.ts"]',
files_edited: '["file2.ts"]',
notes: 'Test notes',
prompt_number: 1,
discovery_tokens: 500,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: Date.now() - 1000 * 60 * 60 * 24
};
describe('HybridSearchStrategy', () => {
let strategy: HybridSearchStrategy;
let mockChromaSync: any;
let mockSessionStore: any;
let mockSessionSearch: any;
beforeEach(() => {
mockChromaSync = {
queryChroma: mock(() => Promise.resolve({
ids: [2, 1, 3], // Chroma returns in semantic relevance order
distances: [0.1, 0.2, 0.3],
metadatas: []
}))
};
mockSessionStore = {
getObservationsByIds: mock((ids: number[]) => {
// Return in the order we stored them (not Chroma order)
const allObs = [mockObservation1, mockObservation2, mockObservation3];
return allObs.filter(obs => ids.includes(obs.id));
}),
getSessionSummariesByIds: mock(() => [mockSession]),
getUserPromptsByIds: mock(() => [])
};
mockSessionSearch = {
findByConcept: mock(() => [mockObservation1, mockObservation2, mockObservation3]),
findByType: mock(() => [mockObservation1, mockObservation2]),
findByFile: mock(() => ({
observations: [mockObservation1, mockObservation2],
sessions: [mockSession]
}))
};
strategy = new HybridSearchStrategy(mockChromaSync, mockSessionStore, mockSessionSearch);
});
describe('canHandle', () => {
it('should return true when concepts filter is present', () => {
const options: StrategySearchOptions = {
concepts: ['test-concept']
};
expect(strategy.canHandle(options)).toBe(true);
});
it('should return true when files filter is present', () => {
const options: StrategySearchOptions = {
files: ['/path/to/file.ts']
};
expect(strategy.canHandle(options)).toBe(true);
});
it('should return true when type and query are present', () => {
const options: StrategySearchOptions = {
type: 'decision',
query: 'semantic query'
};
expect(strategy.canHandle(options)).toBe(true);
});
it('should return true when strategyHint is hybrid', () => {
const options: StrategySearchOptions = {
strategyHint: 'hybrid'
};
expect(strategy.canHandle(options)).toBe(true);
});
it('should return false for query-only (no filters)', () => {
const options: StrategySearchOptions = {
query: 'semantic query'
};
expect(strategy.canHandle(options)).toBe(false);
});
it('should return false for filter-only without Chroma', () => {
// Create strategy without Chroma
const strategyNoChroma = new HybridSearchStrategy(null as any, mockSessionStore, mockSessionSearch);
const options: StrategySearchOptions = {
concepts: ['test-concept']
};
expect(strategyNoChroma.canHandle(options)).toBe(false);
});
});
describe('search', () => {
it('should return empty result for generic hybrid search without query', async () => {
const options: StrategySearchOptions = {
concepts: ['test-concept']
};
const result = await strategy.search(options);
expect(result.results.observations).toHaveLength(0);
expect(result.strategy).toBe('hybrid');
});
it('should return empty result for generic hybrid search (use specific methods)', async () => {
const options: StrategySearchOptions = {
query: 'test query'
};
const result = await strategy.search(options);
// Generic search returns empty - use findByConcept/findByType/findByFile instead
expect(result.results.observations).toHaveLength(0);
});
});
describe('findByConcept', () => {
it('should combine metadata + semantic results', async () => {
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByConcept('test-concept', options);
expect(mockSessionSearch.findByConcept).toHaveBeenCalledWith('test-concept', expect.any(Object));
expect(mockChromaSync.queryChroma).toHaveBeenCalledWith('test-concept', expect.any(Number));
expect(result.usedChroma).toBe(true);
expect(result.fellBack).toBe(false);
expect(result.strategy).toBe('hybrid');
});
it('should preserve semantic ranking order from Chroma', async () => {
// Chroma returns: [2, 1, 3] (obs 2 is most relevant)
// SQLite returns: [1, 2, 3] (by date or however)
// Result should be in Chroma order: [2, 1, 3]
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByConcept('test-concept', options);
expect(result.results.observations.length).toBeGreaterThan(0);
// The first result should be id=2 (Chroma's top result)
expect(result.results.observations[0].id).toBe(2);
});
it('should only include observations that match both metadata and Chroma', async () => {
// Metadata returns ids [1, 2, 3]
// Chroma returns ids [2, 4, 5] (4 and 5 don't exist in metadata results)
mockChromaSync.queryChroma = mock(() => Promise.resolve({
ids: [2, 4, 5],
distances: [0.1, 0.2, 0.3],
metadatas: []
}));
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByConcept('test-concept', options);
// Only id=2 should be in both sets
expect(result.results.observations).toHaveLength(1);
expect(result.results.observations[0].id).toBe(2);
});
it('should return empty when no metadata matches', async () => {
mockSessionSearch.findByConcept = mock(() => []);
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByConcept('nonexistent-concept', options);
expect(result.results.observations).toHaveLength(0);
expect(mockChromaSync.queryChroma).not.toHaveBeenCalled(); // Should short-circuit
});
it('should fall back to metadata-only on Chroma error', async () => {
mockChromaSync.queryChroma = mock(() => Promise.reject(new Error('Chroma failed')));
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByConcept('test-concept', options);
expect(result.usedChroma).toBe(false);
expect(result.fellBack).toBe(true);
expect(result.results.observations).toHaveLength(3); // All metadata results
});
});
describe('findByType', () => {
it('should find observations by type with semantic ranking', async () => {
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByType('decision', options);
expect(mockSessionSearch.findByType).toHaveBeenCalledWith('decision', expect.any(Object));
expect(mockChromaSync.queryChroma).toHaveBeenCalled();
expect(result.usedChroma).toBe(true);
});
it('should handle array of types', async () => {
const options: StrategySearchOptions = {
limit: 10
};
await strategy.findByType(['decision', 'bugfix'], options);
expect(mockSessionSearch.findByType).toHaveBeenCalledWith(['decision', 'bugfix'], expect.any(Object));
// Chroma query should use joined type string
expect(mockChromaSync.queryChroma).toHaveBeenCalledWith('decision, bugfix', expect.any(Number));
});
it('should preserve Chroma ranking order for types', async () => {
mockChromaSync.queryChroma = mock(() => Promise.resolve({
ids: [2, 1], // Chroma order
distances: [0.1, 0.2],
metadatas: []
}));
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByType('decision', options);
expect(result.results.observations[0].id).toBe(2);
});
it('should fall back on Chroma error', async () => {
mockChromaSync.queryChroma = mock(() => Promise.reject(new Error('Chroma unavailable')));
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByType('bugfix', options);
expect(result.usedChroma).toBe(false);
expect(result.fellBack).toBe(true);
expect(result.results.observations.length).toBeGreaterThan(0);
});
it('should return empty when no metadata matches', async () => {
mockSessionSearch.findByType = mock(() => []);
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByType('nonexistent', options);
expect(result.results.observations).toHaveLength(0);
});
});
describe('findByFile', () => {
it('should find observations and sessions by file path', async () => {
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByFile('/path/to/file.ts', options);
expect(mockSessionSearch.findByFile).toHaveBeenCalledWith('/path/to/file.ts', expect.any(Object));
expect(result.observations.length).toBeGreaterThanOrEqual(0);
expect(result.sessions).toHaveLength(1);
});
it('should return sessions without semantic ranking', async () => {
// Sessions are already summarized, no need for semantic ranking
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByFile('/path/to/file.ts', options);
// Sessions should come directly from metadata search
expect(result.sessions).toHaveLength(1);
expect(result.sessions[0].id).toBe(1);
});
it('should apply semantic ranking only to observations', async () => {
mockChromaSync.queryChroma = mock(() => Promise.resolve({
ids: [2, 1], // Chroma ranking for observations
distances: [0.1, 0.2],
metadatas: []
}));
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByFile('/path/to/file.ts', options);
// Observations should be in Chroma order
expect(result.observations[0].id).toBe(2);
expect(result.usedChroma).toBe(true);
});
it('should return usedChroma: false when no observations to rank', async () => {
mockSessionSearch.findByFile = mock(() => ({
observations: [],
sessions: [mockSession]
}));
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByFile('/path/to/file.ts', options);
expect(result.usedChroma).toBe(false);
expect(result.sessions).toHaveLength(1);
});
it('should fall back on Chroma error', async () => {
mockChromaSync.queryChroma = mock(() => Promise.reject(new Error('Chroma down')));
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.findByFile('/path/to/file.ts', options);
expect(result.usedChroma).toBe(false);
expect(result.observations.length).toBeGreaterThan(0);
expect(result.sessions).toHaveLength(1);
});
});
describe('strategy name', () => {
it('should have name "hybrid"', () => {
expect(strategy.name).toBe('hybrid');
});
});
});

349
tests/worker/search/strategies/sqlite-search-strategy.test.ts
View File

@@ -1,349 +0,0 @@
import { describe, it, expect, mock, beforeEach } from 'bun:test';
import { SQLiteSearchStrategy } from '../../../../src/services/worker/search/strategies/SQLiteSearchStrategy.js';
import type { StrategySearchOptions, ObservationSearchResult, SessionSummarySearchResult, UserPromptSearchResult } from '../../../../src/services/worker/search/types.js';
// Mock observation data
const mockObservation: ObservationSearchResult = {
id: 1,
memory_session_id: 'session-123',
project: 'test-project',
text: 'Test observation text',
type: 'decision',
title: 'Test Decision',
subtitle: 'A test subtitle',
facts: '["fact1", "fact2"]',
narrative: 'Test narrative',
concepts: '["concept1", "concept2"]',
files_read: '["file1.ts"]',
files_modified: '["file2.ts"]',
prompt_number: 1,
discovery_tokens: 100,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: 1735732800000
};
const mockSession: SessionSummarySearchResult = {
id: 1,
memory_session_id: 'session-123',
project: 'test-project',
request: 'Test request',
investigated: 'Test investigated',
learned: 'Test learned',
completed: 'Test completed',
next_steps: 'Test next steps',
files_read: '["file1.ts"]',
files_edited: '["file2.ts"]',
notes: 'Test notes',
prompt_number: 1,
discovery_tokens: 500,
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: 1735732800000
};
const mockPrompt: UserPromptSearchResult = {
id: 1,
content_session_id: 'content-session-123',
prompt_number: 1,
prompt_text: 'Test prompt text',
created_at: '2025-01-01T12:00:00.000Z',
created_at_epoch: 1735732800000
};
describe('SQLiteSearchStrategy', () => {
let strategy: SQLiteSearchStrategy;
let mockSessionSearch: any;
beforeEach(() => {
mockSessionSearch = {
searchObservations: mock(() => [mockObservation]),
searchSessions: mock(() => [mockSession]),
searchUserPrompts: mock(() => [mockPrompt]),
findByConcept: mock(() => [mockObservation]),
findByType: mock(() => [mockObservation]),
findByFile: mock(() => ({ observations: [mockObservation], sessions: [mockSession] }))
};
strategy = new SQLiteSearchStrategy(mockSessionSearch);
});
describe('canHandle', () => {
it('should return true when no query text (filter-only)', () => {
const options: StrategySearchOptions = {
project: 'test-project'
};
expect(strategy.canHandle(options)).toBe(true);
});
it('should return true when query is empty string', () => {
const options: StrategySearchOptions = {
query: '',
project: 'test-project'
};
expect(strategy.canHandle(options)).toBe(true);
});
it('should return false when query text is present', () => {
const options: StrategySearchOptions = {
query: 'semantic search query'
};
expect(strategy.canHandle(options)).toBe(false);
});
it('should return true when strategyHint is sqlite (even with query)', () => {
const options: StrategySearchOptions = {
query: 'semantic search query',
strategyHint: 'sqlite'
};
expect(strategy.canHandle(options)).toBe(true);
});
it('should return true for date range filter only', () => {
const options: StrategySearchOptions = {
dateRange: {
start: '2025-01-01',
end: '2025-01-31'
}
};
expect(strategy.canHandle(options)).toBe(true);
});
});
describe('search', () => {
it('should search all types by default', async () => {
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.search(options);
expect(result.usedChroma).toBe(false);
expect(result.fellBack).toBe(false);
expect(result.strategy).toBe('sqlite');
expect(result.results.observations).toHaveLength(1);
expect(result.results.sessions).toHaveLength(1);
expect(result.results.prompts).toHaveLength(1);
expect(mockSessionSearch.searchObservations).toHaveBeenCalled();
expect(mockSessionSearch.searchSessions).toHaveBeenCalled();
expect(mockSessionSearch.searchUserPrompts).toHaveBeenCalled();
});
it('should search only observations when searchType is observations', async () => {
const options: StrategySearchOptions = {
searchType: 'observations',
limit: 10
};
const result = await strategy.search(options);
expect(result.results.observations).toHaveLength(1);
expect(result.results.sessions).toHaveLength(0);
expect(result.results.prompts).toHaveLength(0);
expect(mockSessionSearch.searchObservations).toHaveBeenCalled();
expect(mockSessionSearch.searchSessions).not.toHaveBeenCalled();
expect(mockSessionSearch.searchUserPrompts).not.toHaveBeenCalled();
});
it('should search only sessions when searchType is sessions', async () => {
const options: StrategySearchOptions = {
searchType: 'sessions',
limit: 10
};
const result = await strategy.search(options);
expect(result.results.observations).toHaveLength(0);
expect(result.results.sessions).toHaveLength(1);
expect(result.results.prompts).toHaveLength(0);
});
it('should search only prompts when searchType is prompts', async () => {
const options: StrategySearchOptions = {
searchType: 'prompts',
limit: 10
};
const result = await strategy.search(options);
expect(result.results.observations).toHaveLength(0);
expect(result.results.sessions).toHaveLength(0);
expect(result.results.prompts).toHaveLength(1);
});
it('should pass date range filter to search methods', async () => {
const options: StrategySearchOptions = {
dateRange: {
start: '2025-01-01',
end: '2025-01-31'
},
limit: 10
};
await strategy.search(options);
const callArgs = mockSessionSearch.searchObservations.mock.calls[0];
expect(callArgs[1].dateRange).toEqual({
start: '2025-01-01',
end: '2025-01-31'
});
});
it('should pass project filter to search methods', async () => {
const options: StrategySearchOptions = {
project: 'my-project',
limit: 10
};
await strategy.search(options);
const callArgs = mockSessionSearch.searchObservations.mock.calls[0];
expect(callArgs[1].project).toBe('my-project');
});
it('should pass orderBy to search methods', async () => {
const options: StrategySearchOptions = {
orderBy: 'date_asc',
limit: 10
};
await strategy.search(options);
const callArgs = mockSessionSearch.searchObservations.mock.calls[0];
expect(callArgs[1].orderBy).toBe('date_asc');
});
it('should handle search errors gracefully', async () => {
mockSessionSearch.searchObservations = mock(() => {
throw new Error('Database error');
});
const options: StrategySearchOptions = {
limit: 10
};
const result = await strategy.search(options);
expect(result.results.observations).toHaveLength(0);
expect(result.results.sessions).toHaveLength(0);
expect(result.results.prompts).toHaveLength(0);
expect(result.usedChroma).toBe(false);
});
});
describe('findByConcept', () => {
it('should return matching observations (sync)', () => {
const options: StrategySearchOptions = {
limit: 10
};
const results = strategy.findByConcept('test-concept', options);
expect(results).toHaveLength(1);
expect(results[0].id).toBe(1);
expect(mockSessionSearch.findByConcept).toHaveBeenCalledWith('test-concept', expect.any(Object));
});
it('should pass all filter options to findByConcept', () => {
const options: StrategySearchOptions = {
limit: 20,
project: 'my-project',
dateRange: { start: '2025-01-01' },
orderBy: 'date_desc'
};
strategy.findByConcept('test-concept', options);
expect(mockSessionSearch.findByConcept).toHaveBeenCalledWith('test-concept', {
limit: 20,
project: 'my-project',
dateRange: { start: '2025-01-01' },
orderBy: 'date_desc'
});
});
it('should use default limit when not specified', () => {
const options: StrategySearchOptions = {};
strategy.findByConcept('test-concept', options);
const callArgs = mockSessionSearch.findByConcept.mock.calls[0];
expect(callArgs[1].limit).toBe(20); // SEARCH_CONSTANTS.DEFAULT_LIMIT
});
});
describe('findByType', () => {
it('should return typed observations (sync)', () => {
const options: StrategySearchOptions = {
limit: 10
};
const results = strategy.findByType('decision', options);
expect(results).toHaveLength(1);
expect(results[0].type).toBe('decision');
expect(mockSessionSearch.findByType).toHaveBeenCalledWith('decision', expect.any(Object));
});
it('should handle array of types', () => {
const options: StrategySearchOptions = {
limit: 10
};
strategy.findByType(['decision', 'bugfix'], options);
expect(mockSessionSearch.findByType).toHaveBeenCalledWith(['decision', 'bugfix'], expect.any(Object));
});
it('should pass filter options to findByType', () => {
const options: StrategySearchOptions = {
limit: 15,
project: 'test-project',
orderBy: 'date_asc'
};
strategy.findByType('feature', options);
expect(mockSessionSearch.findByType).toHaveBeenCalledWith('feature', {
limit: 15,
project: 'test-project',
orderBy: 'date_asc'
});
});
});
describe('findByFile', () => {
it('should return observations and sessions for file path', () => {
const options: StrategySearchOptions = {
limit: 10
};
const result = strategy.findByFile('/path/to/file.ts', options);
expect(result.observations).toHaveLength(1);
expect(result.sessions).toHaveLength(1);
expect(mockSessionSearch.findByFile).toHaveBeenCalledWith('/path/to/file.ts', expect.any(Object));
});
it('should pass filter options to findByFile', () => {
const options: StrategySearchOptions = {
limit: 25,
project: 'file-project',
dateRange: { end: '2025-12-31' },
orderBy: 'date_desc'
};
strategy.findByFile('/src/index.ts', options);
expect(mockSessionSearch.findByFile).toHaveBeenCalledWith('/src/index.ts', {
limit: 25,
project: 'file-project',
dateRange: { end: '2025-12-31' },
orderBy: 'date_desc'
});
});
});
describe('strategy name', () => {
it('should have name "sqlite"', () => {
expect(strategy.name).toBe('sqlite');
});
});
});

149
tests/worker/worker-service-unrecoverable.test.ts Normal file
View File

@@ -0,0 +1,149 @@
/**
* Tests for the unrecoverable-error pattern matcher used by the worker
* generator-error `.catch` branch (see worker-service.ts:713-730).
*
* Mock Justification: NONE (0% mock code)
* - Imports a pure helper (isUnrecoverableError) + the backing pattern list.
* - No IO, no timers, no module-level side effects.
*
* Covers Phase 3 of PLAN-windows-max-plan-drain-fix.md:
* - Each newly added OAuth/OpenRouter pattern matches a realistic error.
* - Bare '401' is intentionally NOT a pattern (avoids request-id false positives).
* - All pre-existing patterns still match realistic messages (no regression).
*/
import { describe, it, expect } from 'bun:test';
import {
isUnrecoverableError,
UNRECOVERABLE_ERROR_PATTERNS,
} from '../../src/services/worker/unrecoverable-patterns.js';
describe('isUnrecoverableError', () => {
describe('newly added OAuth / auth patterns (Phase 3)', () => {
it('matches "OAuth token expired"', () => {
expect(
isUnrecoverableError('OAuth token expired at 2026-04-20T00:00:00Z')
).toBe(true);
});
it('matches "token has been revoked"', () => {
expect(
isUnrecoverableError('API token has been revoked by the user')
).toBe(true);
});
it('matches "Unauthorized"', () => {
expect(isUnrecoverableError('401 Unauthorized')).toBe(true);
expect(isUnrecoverableError('Request failed: Unauthorized')).toBe(true);
});
it('matches "OpenRouter API error: 401"', () => {
expect(
isUnrecoverableError('OpenRouter API error: 401 - invalid API key')
).toBe(true);
});
it('matches "OpenRouter API error: 403"', () => {
expect(
isUnrecoverableError('OpenRouter API error: 403 - forbidden')
).toBe(true);
});
});
describe('bare "401" is intentionally NOT a pattern', () => {
it('does NOT match a request-id-like string that merely contains "401"', () => {
// Locks in the decision to avoid bare '401' (too broad).
expect(isUnrecoverableError('request-id-401xyz')).toBe(false);
expect(isUnrecoverableError('correlation: abc-401-def')).toBe(false);
expect(isUnrecoverableError('log: job 401 completed ok')).toBe(false);
});
it('DOES match "401 Unauthorized" via the "Unauthorized" pattern', () => {
// This is correct and intended — when the status code is paired with
// the "Unauthorized" string, we know it's really an auth failure.
expect(isUnrecoverableError('401 Unauthorized')).toBe(true);
});
it('does NOT match a bare "403" either (same reasoning)', () => {
expect(isUnrecoverableError('request-id-403abc')).toBe(false);
});
});
describe('pre-existing patterns still match (no regressions)', () => {
it('matches "Claude executable not found"', () => {
expect(
isUnrecoverableError("Claude executable not found at /usr/bin/claude")
).toBe(true);
});
it('matches "Invalid API key"', () => {
expect(isUnrecoverableError('Invalid API key provided')).toBe(true);
});
it('matches "Gemini API error: 401"', () => {
expect(
isUnrecoverableError('Gemini API error: 401 - unauthorized')
).toBe(true);
});
it('matches "FOREIGN KEY constraint failed"', () => {
expect(
isUnrecoverableError('SQLITE_CONSTRAINT: FOREIGN KEY constraint failed')
).toBe(true);
});
it('matches "ENOENT"', () => {
expect(
isUnrecoverableError('spawn ENOENT no such file or directory')
).toBe(true);
});
it('matches "API_KEY_INVALID"', () => {
expect(
isUnrecoverableError('Error code: API_KEY_INVALID')
).toBe(true);
});
it('matches "PERMISSION_DENIED"', () => {
expect(
isUnrecoverableError('RPC failed: PERMISSION_DENIED')
).toBe(true);
});
});
describe('falsy / non-unrecoverable inputs', () => {
it('returns false for null, undefined, and empty string', () => {
expect(isUnrecoverableError(null)).toBe(false);
expect(isUnrecoverableError(undefined)).toBe(false);
expect(isUnrecoverableError('')).toBe(false);
});
it('returns false for transient/recoverable errors', () => {
// These are handled by fallback/restart logic, not unrecoverable path
expect(isUnrecoverableError('429 Too Many Requests')).toBe(false);
expect(isUnrecoverableError('500 Internal Server Error')).toBe(false);
expect(isUnrecoverableError('503 Service Unavailable')).toBe(false);
expect(isUnrecoverableError('ECONNRESET')).toBe(false);
expect(isUnrecoverableError('fetch failed: network error')).toBe(false);
expect(isUnrecoverableError('Something went wrong')).toBe(false);
});
});
describe('UNRECOVERABLE_ERROR_PATTERNS shape', () => {
it('contains every pattern the matcher relies on', () => {
// Spot-check: the new Phase-3 patterns must be present literally
expect(UNRECOVERABLE_ERROR_PATTERNS).toContain('OAuth token expired');
expect(UNRECOVERABLE_ERROR_PATTERNS).toContain('token has been revoked');
expect(UNRECOVERABLE_ERROR_PATTERNS).toContain('Unauthorized');
expect(UNRECOVERABLE_ERROR_PATTERNS).toContain('OpenRouter API error: 401');
expect(UNRECOVERABLE_ERROR_PATTERNS).toContain('OpenRouter API error: 403');
});
it('does NOT contain bare "401" or "403"', () => {
// Explicit regression guard: the plan forbids adding these.
expect(UNRECOVERABLE_ERROR_PATTERNS).not.toContain('401');
expect(UNRECOVERABLE_ERROR_PATTERNS).not.toContain('403');
});
});
});

2
tests/zombie-prevention.test.ts
View File

@@ -17,7 +17,7 @@
import { describe, test, expect, beforeEach, afterEach, mock } from 'bun:test';
import { ClaudeMemDatabase } from '../src/services/sqlite/Database.js';
import { PendingMessageStore } from '../src/services/sqlite/PendingMessageStore.js';
import { createSDKSession } from '../src/services/sqlite/Sessions.js';
import { createSDKSession } from '../src/services/sqlite/index.js';
import type { ActiveSession, PendingMessage } from '../src/services/worker-types.js';
import type { Database } from 'bun:sqlite';