refactor: Clean up search architecture, remove experimental contextualize endpoint (#133)

* Refactor code structure for improved readability and maintainability

* Add test results for search API and related functionalities

- Created test result files for various search-related functionalities, including:
  - test-11-search-server-changes.json
  - test-12-context-hook-changes.json
  - test-13-worker-service-changes.json
  - test-14-patterns.json
  - test-15-gotchas.json
  - test-16-discoveries.json
  - test-17-all-bugfixes.json
  - test-18-all-features.json
  - test-19-all-decisions.json
  - test-20-session-search.json
  - test-21-prompt-search.json
  - test-22-decisions-endpoint.json
  - test-23-changes-endpoint.json
  - test-24-how-it-works-endpoint.json
  - test-25-contextualize-endpoint.json
  - test-26-timeline-around-observation.json
  - test-27-multi-param-combo.json
  - test-28-file-type-combo.json

- Each test result file captures specific search failures or outcomes, including issues with undefined properties and successful execution of search queries.
- Enhanced documentation of search architecture and testing strategies, ensuring compliance with established guidelines and improving overall search functionality.

* feat: Enhance unified search API with catch-all parameters and backward compatibility

- Implemented a unified search API at /api/search that accepts catch-all parameters for filtering by type, observation type, concepts, and files.
- Maintained backward compatibility by keeping granular endpoints functional while routing through the same infrastructure.
- Completed comprehensive testing of search capabilities with real-world query scenarios.

fix: Address missing debug output in search API query tests

- Flushed PM2 logs and executed search queries to verify functionality.
- Diagnosed absence of "Raw Chroma" debug messages in worker logs, indicating potential issues with logging or query processing.

refactor: Improve build and deployment pipeline for claude-mem plugin

- Successfully built and synced all hooks and services to the marketplace directory.
- Ensured all dependencies are installed and up-to-date in the deployment location.

feat: Implement hybrid search filters with 90-day recency window

- Enhanced search server to apply a 90-day recency filter to Chroma results before categorizing by document type.

fix: Correct parameter handling in searchUserPrompts method

- Added support for filter-only queries and improved dual-path logic for clarity.

refactor: Rename FTS5 method to clarify fallback status

- Renamed escapeFTS5 to escapeFTS5_fallback_when_chroma_unavailable to indicate its temporary usage.

feat: Introduce contextualize tool for comprehensive project overview

- Added a new tool to fetch recent observations, sessions, and user prompts, providing a quick project overview.

feat: Add semantic shortcut tools for common search patterns

- Implemented 'decisions', 'changes', and 'how_it_works' tools for convenient access to frequently searched observation categories.

feat: Unified timeline tool supports anchor and query modes

- Combined get_context_timeline and get_timeline_by_query into a single interface for timeline exploration.

feat: Unified search tool added to MCP server

- New tool queries all memory types simultaneously, providing combined chronological results for improved search efficiency.

* Refactor search functionality to clarify FTS5 fallback usage

- Updated `worker-service.cjs` to replace FTS5 fallback function with a more descriptive name and improved error handling.
- Enhanced documentation in `SKILL.md` to specify the unified API endpoint and clarify the behavior of the search engine, including the conditions under which FTS5 is used.
- Modified `search-server.ts` to provide clearer logging and descriptions regarding the fallback to FTS5 when UVX/Python is unavailable.
- Renamed and updated the `SessionSearch.ts` methods to reflect the conditions for using FTS5, emphasizing the lack of semantic understanding in fallback scenarios.

* feat: Add ID-based fetch endpoints and simplify mem-search skill

**Problem:**
- Search returns IDs but no way to fetch by ID
- Skill documentation was bloated with too many options
- Claude wasn't using IDs because we didn't tell it how

**Solution:**
1. Added three new HTTP endpoints:
   - GET /api/observation/:id
   - GET /api/session/:id
   - GET /api/prompt/:id

2. Completely rewrote SKILL.md:
   - Stripped complexity down to essentials
   - Clear 3-step prescriptive workflow: Search → Review IDs → Fetch by ID
   - Emphasized ID usage: "The IDs are there for a reason - USE THEM"
   - Removed confusing multi-endpoint documentation
   - Kept only unified search with filters

**Impact:**
- Token efficiency: Claude can now fetch full details only for relevant IDs
- Clarity: One clear workflow instead of 10+ options to choose from
- Usability: IDs are no longer wasted context - they're actionable

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: Move internal docs to private directory

Moved POSTMORTEM and planning docs to ./private to exclude from PR reviews.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove experimental contextualize endpoint

- Removed contextualize MCP tool from search-server (saves ~4KB)
- Disabled FTS5 fallback paths in SessionSearch (now vector-first)
- Cleaned up CLAUDE.md documentation
- Removed contextualize-rewrite-plan.md doc

Rationale:
- Contextualize is better suited as a skill (LLM-powered) than an endpoint
- Search API already provides vector search with configurable limits
- Created issue #132 to track future contextualize skill implementation

Changes:
- src/servers/search-server.ts: Removed contextualize tool definition
- src/services/sqlite/SessionSearch.ts: Disabled FTS5 fallback, added deprecation warnings
- CLAUDE.md: Cleaned up outdated skill documentation
- docs/: Removed contextualize plan document

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Complete FTS5 cleanup - remove all deprecated search code

This completes the FTS5 cleanup work by removing all commented-out
FTS5 search code while preserving database tables for backward compatibility.

Changes:
- Removed 200+ lines of commented FTS5 search code from SessionSearch.ts
- Removed deprecated degraded_search_query__when_uvx_unavailable method
- Updated all method documentation to clarify vector-first architecture
- Updated class documentation to reflect filter-only query support
- Updated CLAUDE.md to remove FTS5 search references
- Clarified that FTS5 tables exist for backward compatibility only
- Updated "Why SQLite FTS5" section to "Why Vector-First Search"

Database impact: NONE - FTS5 tables remain intact for existing installations

Search architecture:
- ChromaDB: All text-based vector search queries
- SQLite: Filter-only queries (date ranges, metadata, no query text)
- FTS5 tables: Maintained but unused (backward compatibility)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Remove all FTS5 fallback execution code from search-server

Completes the FTS5 cleanup by removing all fallback execution paths
that attempted to use FTS5 when ChromaDB was unavailable.

Changes:
- Removed all FTS5 fallback code execution paths
- When ChromaDB fails or is unavailable, return empty results with helpful error messages
- Updated all deprecated tool descriptions (search_observations, search_sessions, search_user_prompts)
- Changed error messages to indicate FTS5 fallback has been removed
- Added installation instructions for UVX/Python when vector search is unavailable
- Updated comments from "hybrid search" to "vector-first search"
- Removed ~100 lines of dead FTS5 fallback code

Database impact: NONE - FTS5 tables remain intact (backward compatibility)

Search behavior when ChromaDB unavailable:
- Text queries: Return empty results with error explaining ChromaDB is required
- Filter-only queries (no text): Continue to work via direct SQLite

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Address PR 133 review feedback

Critical fixes:
- Remove contextualize endpoint from worker-service (route + handler)
- Fix build script logging to show correct .cjs extension (was .mjs)

Documentation improvements:
- Add comprehensive FTS5 retention rationale documentation
- Include v7.0.0 removal TODO for future cleanup

Testing:
- Build succeeds with correct output logging
- Worker restarts successfully (30th restart)
- Contextualize endpoint properly removed (404 response)
- Search endpoint verified working

This addresses all critical review feedback from PR 133.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

CLAUDE.md

@@ -10,10 +10,6 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
 
 **Current Version**: 6.0.9
 
-## IMPORTANT: Skills Are Auto-Invoked
-
-**There is no `/skill` command.** Skills auto-invoke based on description metadata matching user queries. Don't document manual invocation (e.g., "Run `/skill troubleshoot`"). Instead: "The troubleshoot skill auto-activates when issues are detected."
-
 ## Critical Architecture Knowledge
 
 ### The Lifecycle Flow
@@ -25,7 +21,7 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
 
 2. **UserPromptSubmit** → `new-hook.ts` runs
    - Creates session record in SQLite
-   - Saves raw user prompt for FTS5 search
+   - Saves raw user prompt for vector search
 
 3. **PostToolUse** → `save-hook.ts` runs
    - Captures your tool executions
@@ -55,20 +51,17 @@ Claude-mem is a Claude Code plugin providing persistent memory across sessions.
 **Database** (`src/services/sqlite/`)
 - SQLite3 with better-sqlite3 (NOT bun:sqlite - that's legacy)
 - Location: `~/.claude-mem/claude-mem.db`
-- FTS5 virtual tables for full-text search
-- `SessionStore` = CRUD, `SessionSearch` = FTS5 queries
+- FTS5 virtual tables maintained for backward compatibility (not used for search)
+- `SessionStore` = CRUD, `SessionSearch` = filter-only queries
 
-**Search Skill** (`plugin/skills/mem-search/SKILL.md`)
+**Mem-Search Skill** (`plugin/skills/mem-search/SKILL.md`)
 - Provides access to all search functionality via HTTP API + skill
-- Auto-invoked when users ask about past work, decisions, or history
-- Uses HTTP endpoints instead of MCP tools (~2,250 token savings per session)
-- 10 search operations: observations, sessions, prompts, by-type, by-file, by-concept, timelines, etc.
 
 **Chroma Vector Database** (`src/services/sync/ChromaSync.ts`)
-- Hybrid semantic + keyword search architecture
+- Vector-first semantic search architecture
 - Automatic vector embedding synchronization
 - 90-day recency filtering for relevant results
-- Combined with SQLite FTS5 for optimal search performance
+- SQLite provides filter-only queries for date ranges and metadata filtering
 
 **Viewer UI** (`src/ui/viewer/`)
 - React + TypeScript web interface accessible at http://localhost:37777
@@ -187,8 +180,8 @@ When investigations fail persistently, use Task agents for comprehensive file an
 ### Why PM2 Instead of Direct Process
 Hooks have strict timeout limits. PM2 manages a persistent background worker, allowing AI processing to continue after hooks complete.
 
-### Why SQLite FTS5
-Enables instant full-text search across thousands of observations without external dependencies. Automatic sync triggers keep FTS5 tables synchronized.
+### Why Vector-First Search (ChromaDB)
+Enables semantic search that understands meaning, not just keywords. ChromaDB handles all text queries. SQLite provides filter-only queries for date ranges and metadata that vector databases can't handle efficiently. FTS5 tables exist for backward compatibility but are no longer used.
 
 ### Why Graceful Cleanup
 Changed from aggressive DELETE requests to marking sessions complete. Prevents interrupting summary generation and other async operations.

package.json

@@ -39,7 +39,8 @@
     "worker:start": "pm2 start ecosystem.config.cjs",
     "worker:stop": "pm2 stop claude-mem-worker",
     "worker:restart": "pm2 restart claude-mem-worker",
-    "worker:logs": "pm2 logs claude-mem-worker",
+    "worker:logs": "pm2 flush claude-mem-worker && pm2 logs claude-mem-worker --lines 100 --nostream",
+    "worker:logs:no-flush": "pm2 logs claude-mem-worker --lines 100 --nostream",
     "changelog:generate": "node scripts/generate-changelog.js",
     "usage:analyze": "node scripts/analyze-usage.js",
     "usage:today": "node scripts/analyze-usage.js $(date +%Y-%m-%d)"

plugin/skills/mem-search/SKILL.md

@@ -1,196 +1,123 @@
 ---
 name: mem-search
-description: Search claude-mem's persistent cross-session memory database to find work from previous conversations days, weeks, or months ago. Access past session summaries, bug fixes, feature implementations, and decisions that are NOT in the current conversation context. Use when user asks "did we already solve this?", "how did we do X last time?", "what happened in last week's session?", or needs information from previous sessions stored in the PM2-managed database. Searches observations, session summaries, and user prompts across entire project history.
+description: Search claude-mem's persistent cross-session memory database. Use when user asks "did we already solve this?", "how did we do X last time?", or needs work from previous sessions.
 ---
 
 # Memory Search
 
-Access claude-mem's persistent cross-session memory through HTTP API. Find past work, understand context across sessions, and learn from previous decisions.
+Search past work across all sessions. Simple workflow: search → get IDs → fetch details by ID.
 
-## When to Use This Skill
+## When to Use
 
-**Use when users ask about work from PREVIOUS sessions** (not current conversation):
+Use when users ask about PREVIOUS sessions (not current conversation):
+- "Did we already fix this?"
+- "How did we solve X last time?"
+- "What happened last week?"
 
-### Temporal Triggers (Key Indicators)
-- "Did we **already** fix this bug?" or "Have we seen this error **before**?"
-- "How did we solve X **last time**?" or "What approach did we take **previously**?"
-- "What did we do in **yesterday's/last week's/last month's** session?"
-- "**When** did we last work on this?" or "What's the **history** of this file?"
+## The Workflow
 
-### Cross-Session Queries
-- "Show me all authentication-related changes **across all sessions**"
-- "What features did we add **last month**?" (not "today" or "this session")
-- "Why did we choose this approach **before**?" (decisions from past sessions)
-- "What files did we modify **when we added X**?" (historical context)
+**ALWAYS follow this exact flow:**
 
-**Do NOT use** for current session work, future planning, or questions Claude can answer from current conversation context.
+1. **Search** - Get an index of results with IDs
+2. **Review** - Look at titles/dates, pick relevant IDs
+3. **Fetch** - Get full details ONLY for those IDs
 
-## Common Trigger Phrases
+### Step 1: Search Everything
 
-This skill activates when detecting phrases about **cross-session history**:
+```bash
+curl "http://localhost:37777/api/search?query=authentication&format=index&limit=5"
+```
 
-- "Did we already solve this?" / "Have we done this before?"
-- "How did we implement X last time?"
-- "What did we work on yesterday/last week/last month?"
-- "Show me the history of [file/feature/decision]"
-- "When did we fix/add/change X?"
-- "What was happening around [date/time]?"
-- "Catch me up on recent sessions" / "What have we been doing?"
-- "What changes to [filename] across all sessions?"
+**Required parameters:**
+- `query` - Search term
+- `format=index` - ALWAYS start with index (lightweight)
+- `limit=5` - Start small (3-5 results)
 
-**Unique identifiers:** claude-mem, persistent memory, cross-session database, session history, PM2-managed database
+**Returns:**
+```
+1. [feature] Added JWT authentication
+   Date: 11/17/2025, 3:48:45 PM
+   ID: 11131
 
-## Available Operations
+2. [bugfix] Fixed auth token expiration
+   Date: 11/16/2025, 2:15:22 PM
+   ID: 10942
+```
 
-### Full-Text Search
-- **observations** - Search all observations by keyword (bugs, features, decisions, discoveries, changes)
-  - Use when: "How did we implement X?" or "What bugs did we fix?"
-  - Example: Search for "authentication JWT" to find auth-related work
+### Step 2: Pick IDs
 
-- **sessions** - Search session summaries to find what was accomplished when
-  - Use when: "What did we accomplish last time?" or "What was the goal of that session?"
-  - Example: Find sessions where "added login feature"
+Review the index results. Identify which IDs are actually relevant. Discard the rest.
 
-- **prompts** - Find what users have asked about in previous sessions
-  - Use when: "Did I ask about this before?" or "What did I request last week?"
-  - Example: Search for "database migration" in past user prompts
+### Step 3: Fetch by ID
 
-### Filtered Search
-- **by-type** - Filter observations by type (bugfix, feature, refactor, decision, discovery, change)
-  - Use when: "Show me all bug fixes" or "List features we added"
-  - Example: Get all observations with type=bugfix from last month
+For each relevant ID, fetch full details:
 
-- **by-concept** - Find observations tagged with specific concepts (problem-solution, how-it-works, gotcha)
-  - Use when: "What patterns did we discover?" or "Show me gotchas"
-  - Example: Find all observations tagged with concept "gotcha"
+```bash
+# Fetch observation
+curl "http://localhost:37777/api/observation/11131"
 
-- **by-file** - Find all work related to a specific file path across all sessions
-  - Use when: "What changes to auth.ts?" or "History of this file"
-  - Example: Get all work related to "src/auth/login.ts"
+# Fetch session
+curl "http://localhost:37777/api/session/2005"
 
-### Timeline & Context
-- **recent-context** - Get last N sessions with summaries and observations
-  - Use when: "What's been happening?" or "Catch me up on recent work"
-  - Example: Get last 3 sessions with limit=3
+# Fetch prompt
+curl "http://localhost:37777/api/prompt/5421"
+```
 
-- **timeline** - Get chronological context around a specific point in time (before/after window)
-  - Use when: "What was happening around [date]?" or "Show me context from that time"
-  - Example: Timeline around session 123 with depth 5 before and after
+**ID formats:**
+- Observations: Just the number (11131)
+- Sessions: Just the number (2005) from "S2005"
+- Prompts: Just the number (5421)
 
-- **timeline-by-query** - Search first, then get timeline context around best match
-  - Use when: "When did we implement auth?" combined with "show me context around that time"
-  - Example: Search for "OAuth implementation" then get surrounding timeline
+## Search Parameters
 
-For detailed instructions on any operation, read the corresponding file in [operations/](operations/).
+**Basic:**
+- `query` - What to search for (required)
+- `format` - "index" or "full" (always use "index" first)
+- `limit` - How many results (default 5, max 100)
 
-## Quick Decision Guide
+**Filters (optional):**
+- `type` - Filter to "observations", "sessions", or "prompts"
+- `project` - Filter by project name
+- `dateRange[start]` - Start date (YYYY-MM-DD)
+- `dateRange[end]` - End date (YYYY-MM-DD)
+- `obs_type` - Filter observations by: bugfix, feature, decision, discovery, change
 
-**What is the user asking about?**
+## Examples
 
-1. **Recent work** (last 3-5 sessions) → Use **recent-context** with limit=3-5
-2. **Specific topic/keyword** (bugs, features, decisions) → Use **observations** search
-3. **Specific file history** (changes to a file) → Use **by-file** search
-4. **Timeline/chronology** (what happened when) → Use **timeline** or **timeline-by-query**
-5. **Type-specific** (all bug fixes, all features) → Use **by-type** filter
+**Find recent bug fixes:**
+```bash
+curl "http://localhost:37777/api/search?query=bug&type=observations&obs_type=bugfix&format=index&limit=5"
+```
 
-**Most common:** Use **observations** search for general "how did we..." questions.
+**Find what happened last week:**
+```bash
+curl "http://localhost:37777/api/search?query=&type=observations&dateRange[start]=2025-11-11&format=index&limit=10"
+```
 
-## Progressive Disclosure Workflow (Token Efficiency)
+**Search everything:**
+```bash
+curl "http://localhost:37777/api/search?query=database+migration&format=index&limit=5"
+```
 
-**Core Principle**: Find high-signal items in index format FIRST (~50-100 tokens each), then request full details ONLY for relevant items (~500-1000 tokens each).
+## Why This Workflow?
 
-**4-Step Workflow:**
+**Token efficiency:**
+- Index format: ~50-100 tokens per result
+- Full format: ~500-1000 tokens per result
+- **10x difference** - only fetch full when you know it's relevant
 
-1. **Start with Index Format**
-   - Always use `format=index` initially
-   - Set `limit=3-5` (not 10-20)
-   - Review titles and dates to assess relevance
-   - Token cost: ~50-100 per result
-
-2. **Identify Relevant Items**
-   - Scan index results
-   - Discard irrelevant items from list
-   - Keep only 1-3 most relevant
-
-3. **Request Full Details Selectively**
-   - Use `format=full` ONLY for specific relevant items
-   - Token cost: ~500-1000 per result
-   - **10x cost difference** - be selective
-
-4. **Refine with Filters**
-   - Use type, dateRange, concepts, files filters
-   - Paginate with offset if needed
-   - Narrow scope before expanding limits
-
-**DO:**
-- ✅ Start with `format=index` and `limit=3-5`
-- ✅ Use filters (type, dateRange, concepts, files) to narrow results
-- ✅ Request `format=full` ONLY for specific relevant items
-- ✅ Use offset for pagination instead of large limits
-
-**DON'T:**
-- ❌ Jump straight to `format=full`
-- ❌ Request `limit=20` without assessing index results first
-- ❌ Load full details for all results upfront
-- ❌ Skip index format to "save a step" (costs 10x more tokens)
-
-See [principles/progressive-disclosure.md](principles/progressive-disclosure.md) for complete workflow with examples and token calculations.
-
-## Quick Reference Table
-
-| Need | Operation | Key Parameters |
-|------|-----------|----------------|
-| Recent context | recent-context | limit=3-5 |
-| Search observations | observations | query, format=index, limit=5 |
-| Search sessions | sessions | query, format=index, limit=5 |
-| Find by type | by-type | type=(bugfix\|feature\|decision), format=index |
-| Find by file | by-file | filePath, format=index |
-| Timeline around event | timeline | anchor=(sessionDbId), depth_before=5, depth_after=5 |
-| Search + timeline | timeline-by-query | query, mode=auto |
-
-## Common Workflows
-
-For step-by-step guides on typical user requests, see [operations/common-workflows.md](operations/common-workflows.md):
-- Understanding past work from previous sessions
-- Finding specific bug fixes from history
-- Understanding file history across sessions
-- Timeline investigation workflows
-- Search composition patterns
-
-## Response Formatting
-
-For guidelines on presenting search results to users, see [operations/formatting.md](operations/formatting.md):
-- Index format responses (compact lists with titles/dates)
-- Full format responses (complete observation details)
-- Timeline responses (chronologically grouped)
-- Error handling and user-friendly messages
-
-## Technical Notes
-
-- **Port:** Default 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
-- **Response format:** Always JSON
-- **Search engine:** ChromaDB semantic search (primary ranking) + SQLite FTS5 (fallback) + 90-day recency filter + temporal ordering (hybrid architecture)
-- **All operations:** HTTP GET with query parameters
-- **Worker:** PM2-managed background process
+**Clarity:**
+- See everything first
+- Pick what matters
+- Get details only for what you need
 
 ## Error Handling
 
-If HTTP request fails:
-1. Inform user the claude-mem search service isn't available
-2. Suggest checking if worker is running: `pm2 list` or `pm2 status claude-mem-worker`
-3. Offer to help troubleshoot using the troubleshoot skill
-
-## Resources
-
-**Principles:**
-- [principles/progressive-disclosure.md](principles/progressive-disclosure.md) - Complete 4-step workflow with token calculations
-- [principles/anti-patterns.md](principles/anti-patterns.md) - 5 anti-patterns to avoid with LLM behavior insights
-
-**Operations:**
-- [operations/](operations/) - Detailed instructions for each operation (9 operations + help)
-- [operations/common-workflows.md](operations/common-workflows.md) - Step-by-step workflow guides
-- [operations/formatting.md](operations/formatting.md) - Response formatting templates
+If search fails, tell the user the worker isn't available and suggest:
+```bash
+pm2 list  # Check if worker is running
+```
 
 ---
 
-**Remember:** This skill searches **cross-session persistent memory**, NOT current conversation. Start with index format for token efficiency. Use temporal triggers to differentiate from native Claude memory.
+**Remember:** ALWAYS search with format=index first. ALWAYS fetch by ID for details. The IDs are there for a reason - USE THEM.

plugin/skills/mem-search/operations/observations.md

@@ -17,10 +17,16 @@ curl -s "http://localhost:37777/api/search/observations?query=authentication&for
 
 ## Parameters
 
-- **query** (required): Natural language search query - uses semantic search (ChromaDB) for ranking with SQLite FTS5 fallback (e.g., "authentication", "bug fix", "database migration")
+- **query** (optional): Natural language search query - uses semantic search (ChromaDB) for ranking with SQLite FTS5 fallback (e.g., "authentication", "bug fix", "database migration"). Can be omitted for filter-only searches.
 - **format**: "index" (summary) or "full" (complete details). Default: "full"
 - **limit**: Number of results (default: 20, max: 100)
 - **project**: Filter by project name (optional)
+- **dateRange**: Filter by date range (optional) - `dateRange[start]` and/or `dateRange[end]`
+- **obs_type**: Filter by observation type: bugfix, feature, refactor, decision, discovery, change (optional)
+- **concepts**: Filter by concept tags (optional)
+- **files**: Filter by file paths (optional)
+
+**Important**: When omitting `query`, you MUST provide at least one filter (project, dateRange, obs_type, concepts, or files)
 
 ## When to Use Each Format
 
@@ -76,13 +82,28 @@ Found 5 results for "authentication":
 
 For complete formatting guidelines, see formatting.md (documentation coming soon).
 
+## Filter-Only Examples
+
+Search without query text (direct SQLite filtering):
+
+```bash
+# Get all observations from November 2025
+curl -s "http://localhost:37777/api/search?type=observations&dateRange[start]=2025-11-01&format=index"
+
+# Get all bug fixes from a specific project
+curl -s "http://localhost:37777/api/search?type=observations&obs_type=bugfix&project=api-server&format=index"
+
+# Get all observations from last 7 days
+curl -s "http://localhost:37777/api/search?type=observations&dateRange[start]=2025-11-11&format=index"
+```
+
 ## Error Handling
 
-**Missing query parameter:**
+**Missing query and filters:**
 ```json
-{"error": "Missing required parameter: query"}
+{"error": "Either query or filters required for search"}
 ```
-Fix: Add the query parameter
+Fix: Provide either a query parameter OR at least one filter (project, dateRange, obs_type, concepts, files)
 
 **No results found:**
 ```json

private/POSTMORTEM-worker-debug-failure.md

@@ -0,0 +1,109 @@
+# Postmortem: Worker Debug Failure - 2025-11-17
+
+## Incident Summary
+Attempted to fix broken worker service. Worker was in crash loop with 225 restarts, failing with "MCP error -32000: Connection closed". Debug attempt failed and changes were reverted.
+
+## What Went Wrong
+
+### 1. **Jumped to Symptoms, Not Root Cause**
+- Saw "MCP connection failed" errors in logs
+- Immediately focused on MCP/Chroma connection code
+- Never asked: "Why is this suddenly broken when it worked before?"
+- Classic symptom chasing instead of root cause analysis
+
+### 2. **Ignored the Build Pipeline**
+- Worker file wasn't in the expected location (`plugin/worker-service.cjs` vs `plugin/scripts/worker-service.cjs`)
+- Build output existed but search server was producing corrupted/error output
+- Never investigated: "Is the build system broken?"
+- Should have compared built artifacts between main and current branch
+
+### 3. **Tried to Fix by Disabling Instead of Understanding**
+- Final approach: comment out Chroma, comment out search server
+- This is the opposite of debugging - it's just making things "work" by removing functionality
+- User called this out as "duct tape around 5 things unrelated to the problem"
+- Violated YAGNI/KISS by adding defensive complexity instead of fixing the actual issue
+
+### 4. **Didn't Compare Working vs Broken State**
+- User specifically said "we fixed this before"
+- Should have immediately: `git diff main src/services/worker-service.ts`
+- Did this eventually but didn't follow through on the findings
+- The diff showed only search-everything additions - the core worker code was UNCHANGED
+- This should have been a huge red flag: "If the code is the same, why is it broken?"
+
+### 5. **Overcomplicated the Investigation**
+- Started reading through ChromaSync implementation
+- Traced through MCP connection code
+- Analyzed startup sequences
+- All of this was unnecessary if the root cause was a build issue
+
+## What Should Have Happened
+
+### Correct Debug Sequence:
+1. ✅ Check worker status (`pm2 list`) - DONE
+2. ✅ Check error logs - DONE
+3. ❌ **Compare current code to main branch** - SKIPPED INITIALLY
+4. ❌ **Check if built files are correct** - SKIPPED
+5. ❌ **Test the build pipeline** - NEVER DONE
+6. ❌ **Verify dependencies are installed** - NEVER CHECKED
+
+### The Real Questions:
+- Is this a code change or a build issue?
+- What changed between working state and broken state?
+- Are the built artifacts corrupted?
+- Is the search server build actually valid?
+- Are there missing dependencies in plugin/scripts/node_modules?
+
+## Likely Root Causes (Untested)
+
+Based on evidence:
+1. **Build artifacts are corrupted** - search-server.mjs threw syntax errors when run
+2. **Node modules missing/outdated** - plugin/scripts/node_modules may be stale
+3. **ESM/CJS bundling issue** - esbuild may have produced invalid output
+4. **search-everything branch has broken build config** - scripts/build-hooks.js may have issues
+
+## Key Lessons
+
+### KISS/DRY/YAGNI Violations
+- Added complexity (disabling features) instead of removing it
+- Tried to work around symptoms instead of fixing root cause
+- Ignored the principle: "If it worked before and code is same, it's environment/build"
+
+### Debugging Anti-Patterns
+1. **Symptom Chasing**: Following error messages down rabbit holes
+2. **Defensive Coding**: Commenting out "broken" features instead of fixing them
+3. **Ignoring History**: Not comparing working vs broken states
+4. **Build Blindness**: Assuming built artifacts are correct without verification
+
+### What Good Debugging Looks Like
+1. Compare working state (main) vs broken state (current branch)
+2. Identify what actually changed (code? deps? build?)
+3. Test the simplest hypothesis first (build issue vs code issue)
+4. Never disable features to "fix" things - that's not fixing
+
+## Action Items for Next Attempt
+
+### Before Writing Any Code:
+- [ ] `git diff main` for all modified files
+- [ ] Check if `plugin/scripts/` artifacts are valid JavaScript
+- [ ] Compare build process: `npm run build` output on main vs current branch
+- [ ] Verify `plugin/scripts/node_modules` exists and is current
+- [ ] Test search-server.mjs in isolation: `node plugin/scripts/search-server.mjs`
+
+### If Build is Broken:
+- [ ] Check scripts/build-hooks.js for recent changes
+- [ ] Verify esbuild configuration
+- [ ] Test build on main branch, then on current branch
+- [ ] Don't modify source code until build is proven working
+
+### If Code is Broken:
+- [ ] Create minimal repro (which specific change broke it?)
+- [ ] Fix the actual bug, don't add workarounds
+- [ ] Test the fix in isolation
+
+## Conclusion
+
+This failure exemplifies "debugging by making changes" instead of "debugging by understanding". The instinct to fix symptoms (MCP errors) instead of investigating root cause (why is it broken now?) led to wasted effort and ultimately no solution.
+
+The user's frustration was justified - I was adding defensive duct tape instead of finding and fixing the real problem. This is exactly what KISS/DRY/YAGNI principles are meant to prevent.
+
+**Next time: Compare, verify, understand, THEN fix. Never disable features to make errors go away.**

private/docs/context/search-plan-nov-17.md

@@ -0,0 +1,119 @@
+ Unified Search API Consolidation Plan
+
+ Overview
+
+ Consolidate 10 search endpoints into 6 powerful, semantic endpoints with intelligent aliasing for backward compatibility.
+
+ New Endpoint Structure
+
+ 1. /search - Unified Cross-Type Search
+
+ - Searches all record types (observations + sessions + prompts) via Chroma multi-collection search
+ - Optional type filter to narrow down
+ - Replaces: search_observations, search_sessions, search_user_prompts
+ - Params: query, response=[index|full], type, project, dateRange, limit, offset, orderBy
+
+ 2. /timeline - Unified Timeline Tool
+
+ - Supports both anchor-based and query-based modes via params
+ - If anchor → direct timeline lookup (like get_context_timeline)
+ - If query → search-first then timeline (like get_timeline_by_query)
+ - Params: anchor OR query, depth_before, depth_after, response, mode, project
+
+ 3. /decisions - Decision Observations
+
+ - Metadata-first search for type=decision observations
+ - Uses specialized search logic for precision
+ - Params: response, project, dateRange, limit, offset, orderBy
+
+ 4. /changes - Change Observations
+
+ - Metadata-first search for type=change observations
+ - Same pattern as /decisions
+
+ 5. /how-it-works - How-It-Works Concept
+
+ - Metadata-first search for concept=how-it-works observations
+ - Same pattern as concept endpoints
+
+ 6. /contextualize - Intelligent Context Builder
+
+ - Complex hybrid endpoint:
+   a. Get 7 latest decisions + 7 latest changes + 3 latest how-it-works
+   b. Find newest date across all results
+   c. Get timeline (7 before + 7 after) around that date
+   d. Merge & re-sort into single timeline (newest → oldest)
+   e. Return timeline + narratives of each concept's latest result
+ - Params: query (for contextualization), project
+
+ Implementation Phases
+
+ Phase 1: Core Unified Search (search-server.ts)
+
+ - Create search tool with Chroma multi-collection query
+ - Add type filtering support
+ - Alias old tools: search_observations → search(type=['observations'])
+
+ Phase 2: Unified Timeline (search-server.ts)
+
+ - Merge get_context_timeline + get_timeline_by_query logic
+ - Support both anchor and query params (mutually exclusive)
+ - Alias old timeline tools to new unified implementation
+
+ Phase 3: Specialized Concept Endpoints (search-server.ts)
+
+ - Create decisions, changes, how_it_works tools
+ - Use metadata-first search strategy
+ - Update find_by_type and find_by_concept to call these internally
+
+ Phase 4: Contextualize Endpoint (search-server.ts)
+
+ - Implement parallel fetching (7 decisions, 7 changes, 3 how-it-works)
+ - Find newest date, get timeline around it
+ - Merge, re-sort, extract narratives
+ - Return structured response with timeline + narratives
+
+ Phase 5: HTTP API Routes (worker-service.ts)
+
+ - Add 6 new routes: /api/search, /api/timeline, /api/decisions, /api/changes, /api/how-it-works, /api/contextualize
+ - Update old routes to alias new implementations
+ - Maintain backward compatibility
+
+ Phase 6: Chroma Multi-Collection Search (ChromaSync.ts)
+
+ - Add searchAll() method to query all collections in parallel
+ - Include source collection metadata in results
+ - Merge and rank by similarity score
+
+ Phase 7: SQLite Fallback (SessionSearch.ts)
+
+ - Add searchAll() for FTS5 fallback when Chroma unavailable
+ - Merge results from all three FTS5 tables
+
+ Phase 8: Documentation & Skill Updates
+
+ - Update mem-search skill with new endpoints
+ - Update CLAUDE.md and README.md
+ - Add examples and migration guide
+
+ Phase 9: Testing & Deployment
+
+ - Unit tests for all new tools
+ - Integration tests for aliasing
+ - Manual testing via mem-search skill
+ - Build → sync → worker restart
+
+ Key Design Decisions
+
+ ✅ Aliasing Strategy: Old endpoints call new implementations internally (zero breaking changes)
+ ✅ Unified Search: Chroma multi-collection search for true cross-type queries
+ ✅ Flexible Timeline: Single tool supports both direct and query-based modes
+ ✅ Specialized Shortcuts: /decisions, /changes, /how-it-works for common queries
+ ✅ Intelligent Context: /contextualize auto-builds rich context with narratives
+
+ Migration Impact
+
+ - Users: Zero breaking changes, old endpoints work via aliasing
+ - Codebase: Simplified from 10 conceptual endpoints to 6
+ - Performance: Improved via Chroma multi-collection search
+ - Developer UX: Cleaner, more semantic API

scripts/build-hooks.js

@@ -26,12 +26,10 @@ const WORKER_SERVICE = {
   source: 'src/services/worker-service.ts'
 };
 
-// DEPRECATED: MCP search server replaced by skill-based search
-// Keeping source file for reference: src/servers/search-server.ts
-// const SEARCH_SERVER = {
-//   name: 'search-server',
-//   source: 'src/servers/search-server.ts'
-// };
+const SEARCH_SERVER = {
+  name: 'search-server',
+  source: 'src/servers/search-server.ts'
+};
 
 async function buildHooks() {
   console.log('🔨 Building claude-mem hooks and worker service...\n');
@@ -94,6 +92,31 @@ async function buildHooks() {
     const workerStats = fs.statSync(`${hooksDir}/${WORKER_SERVICE.name}.cjs`);
     console.log(`✓ worker-service built (${(workerStats.size / 1024).toFixed(2)} KB)`);
 
+    // Build search server
+    console.log(`\n🔧 Building search server...`);
+    await build({
+      entryPoints: [SEARCH_SERVER.source],
+      bundle: true,
+      platform: 'node',
+      target: 'node18',
+      format: 'cjs',
+      outfile: `${hooksDir}/${SEARCH_SERVER.name}.cjs`,
+      minify: true,
+      logLevel: 'error',
+      external: ['better-sqlite3'],
+      define: {
+        '__DEFAULT_PACKAGE_VERSION__': `"${version}"`
+      },
+      banner: {
+        js: '#!/usr/bin/env node'
+      }
+    });
+
+    // Make search server executable
+    fs.chmodSync(`${hooksDir}/${SEARCH_SERVER.name}.cjs`, 0o755);
+    const searchServerStats = fs.statSync(`${hooksDir}/${SEARCH_SERVER.name}.cjs`);
+    console.log(`✓ search-server built (${(searchServerStats.size / 1024).toFixed(2)} KB)`);
+
     // Build each hook
     for (const hook of HOOKS) {
       console.log(`\n🔧 Building ${hook.name}...`);
@@ -126,14 +149,11 @@ async function buildHooks() {
       console.log(`✓ ${hook.name} built (${sizeInKB} KB)`);
     }
 
-    // DEPRECATED: MCP search server no longer built (replaced by skill-based search)
-    // Search functionality now provided via HTTP API + search skill
-    // Source file kept for reference: src/servers/search-server.ts
-
-    console.log('\n✅ All hooks and worker service built successfully!');
+    console.log('\n✅ All hooks, worker service, and search server built successfully!');
     console.log(`   Output: ${hooksDir}/`);
     console.log(`   - Hooks: *-hook.js`);
     console.log(`   - Worker: worker-service.cjs`);
+    console.log(`   - Search Server: search-server.cjs`);
     console.log(`   - Skills: plugin/skills/`);
     console.log('\n💡 Note: Dependencies will be auto-installed on first hook execution');
 

src/services/sqlite/SessionSearch.ts

@@ -13,7 +13,8 @@ import {
 
 /**
  * Search interface for session-based memory
- * Provides FTS5 full-text search and structured queries for sessions, observations, and summaries
+ * Provides filter-only structured queries for sessions, observations, and user prompts
+ * Vector search is handled by ChromaDB - this class only supports filtering without query text
  */
 export class SessionSearch {
   private db: Database.Database;
@@ -31,7 +32,18 @@ export class SessionSearch {
   }
 
   /**
-   * Ensure FTS5 tables exist (inline migration)
+   * Ensure FTS5 tables exist (backward compatibility only - no longer used for search)
+   *
+   * FTS5 tables are maintained for backward compatibility but not used for search.
+   * Vector search (Chroma) is now the primary search mechanism.
+   *
+   * Retention Rationale:
+   * - Prevents breaking existing installations with FTS5 tables
+   * - Allows graceful migration path for users
+   * - Tables maintained but search paths removed
+   * - Triggers still fire to keep tables synchronized
+   *
+   * TODO: Remove FTS5 infrastructure in future major version (v7.0.0)
    */
   private ensureFTSTables(): void {
     try {
@@ -134,22 +146,6 @@ export class SessionSearch {
     }
   }
 
-  /**
-   * Escape FTS5 special characters in user input
-   * 
-   * FTS5 uses double quotes for phrase searches and treats certain characters
-   * as operators (*, AND, OR, NOT, parentheses, etc.). To prevent injection,
-   * we wrap user input in double quotes and escape internal quotes by doubling them.
-   * This converts any user input into a safe phrase search.
-   * 
-   * @param text - User input to escape for FTS5 MATCH queries
-   * @returns Safely escaped FTS5 query string
-   */
-  private escapeFTS5(text: string): string {
-    // Escape internal double quotes by doubling them (FTS5 standard)
-    // Then wrap the entire string in double quotes for phrase search
-    return `"${text.replace(/"/g, '""')}"`;
-  }
 
   /**
    * Build WHERE clause for structured filters
@@ -243,118 +239,78 @@ export class SessionSearch {
   }
 
   /**
-   * Search observations using FTS5 full-text search
+   * Search observations using filter-only direct SQLite query.
+   * Vector search is handled by ChromaDB - this only supports filtering without query text.
    */
-  searchObservations(query: string, options: SearchOptions = {}): ObservationSearchResult[] {
+  searchObservations(query: string | undefined, options: SearchOptions = {}): ObservationSearchResult[] {
     const params: any[] = [];
     const { limit = 50, offset = 0, orderBy = 'relevance', ...filters } = options;
 
-    // Build FTS5 match query
-    const ftsQuery = this.escapeFTS5(query);
-    params.push(ftsQuery);
+    // FILTER-ONLY PATH: When no query text, query table directly
+    // This enables date filtering which Chroma cannot do (requires direct SQLite access)
+    if (!query) {
+      const filterClause = this.buildFilterClause(filters, params, 'o');
+      if (!filterClause) {
+        throw new Error('Either query or filters required for search');
+      }
 
-    // Build filter conditions
-    const filterClause = this.buildFilterClause(filters, params, 'o');
-    const whereClause = filterClause ? `AND ${filterClause}` : '';
+      const orderClause = this.buildOrderClause(orderBy, false);
 
-    // Build ORDER BY
-    const orderClause = this.buildOrderClause(orderBy, true);
+      const sql = `
+        SELECT o.*, o.discovery_tokens
+        FROM observations o
+        WHERE ${filterClause}
+        ${orderClause}
+        LIMIT ? OFFSET ?
+      `;
 
-    // Main query with FTS5
-    const sql = `
-      SELECT
-        o.*,
-        o.discovery_tokens,
-        observations_fts.rank as rank
-      FROM observations o
-      JOIN observations_fts ON o.id = observations_fts.rowid
-      WHERE observations_fts MATCH ?
-      ${whereClause}
-      ${orderClause}
-      LIMIT ? OFFSET ?
-    `;
-
-    params.push(limit, offset);
-
-    const results = this.db.prepare(sql).all(...params) as ObservationSearchResult[];
-
-    // Normalize rank to score (0-1, higher is better)
-    if (results.length > 0) {
-      const minRank = Math.min(...results.map(r => r.rank || 0));
-      const maxRank = Math.max(...results.map(r => r.rank || 0));
-      const range = maxRank - minRank || 1;
-
-      results.forEach(r => {
-        if (r.rank !== undefined) {
-          // Invert rank (lower rank = better match) and normalize to 0-1
-          r.score = 1 - ((r.rank - minRank) / range);
-        }
-      });
+      params.push(limit, offset);
+      return this.db.prepare(sql).all(...params) as ObservationSearchResult[];
     }
 
-    return results;
+    // Vector search with query text should be handled by ChromaDB
+    // This method only supports filter-only queries (query=undefined)
+    console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
+    return [];
   }
 
   /**
-   * Search session summaries using FTS5 full-text search
+   * Search session summaries using filter-only direct SQLite query.
+   * Vector search is handled by ChromaDB - this only supports filtering without query text.
    */
-  searchSessions(query: string, options: SearchOptions = {}): SessionSummarySearchResult[] {
+  searchSessions(query: string | undefined, options: SearchOptions = {}): SessionSummarySearchResult[] {
     const params: any[] = [];
     const { limit = 50, offset = 0, orderBy = 'relevance', ...filters } = options;
 
-    // Build FTS5 match query
-    const ftsQuery = this.escapeFTS5(query);
-    params.push(ftsQuery);
+    // FILTER-ONLY PATH: When no query text, query session_summaries table directly
+    if (!query) {
+      const filterOptions = { ...filters };
+      delete filterOptions.type;
+      const filterClause = this.buildFilterClause(filterOptions, params, 's');
+      if (!filterClause) {
+        throw new Error('Either query or filters required for search');
+      }
 
-    // Build filter conditions (without type filter - not applicable to summaries)
-    const filterOptions = { ...filters };
-    delete filterOptions.type;
-    const filterClause = this.buildFilterClause(filterOptions, params, 's');
-    const whereClause = filterClause ? `AND ${filterClause}` : '';
+      const orderClause = orderBy === 'date_asc'
+        ? 'ORDER BY s.created_at_epoch ASC'
+        : 'ORDER BY s.created_at_epoch DESC';
 
-    // Note: session_summaries don't have files_read/files_modified in the same way
-    // We'll need to adjust the filter clause
-    const adjustedWhereClause = whereClause.replace(/files_read/g, 'files_read').replace(/files_modified/g, 'files_edited');
+      const sql = `
+        SELECT s.*, s.discovery_tokens
+        FROM session_summaries s
+        WHERE ${filterClause}
+        ${orderClause}
+        LIMIT ? OFFSET ?
+      `;
 
-    // Build ORDER BY
-    const orderClause = orderBy === 'relevance'
-      ? 'ORDER BY session_summaries_fts.rank ASC'
-      : orderBy === 'date_asc'
-      ? 'ORDER BY s.created_at_epoch ASC'
-      : 'ORDER BY s.created_at_epoch DESC';
-
-    // Main query with FTS5
-    const sql = `
-      SELECT
-        s.*,
-        s.discovery_tokens,
-        session_summaries_fts.rank as rank
-      FROM session_summaries s
-      JOIN session_summaries_fts ON s.id = session_summaries_fts.rowid
-      WHERE session_summaries_fts MATCH ?
-      ${adjustedWhereClause}
-      ${orderClause}
-      LIMIT ? OFFSET ?
-    `;
-
-    params.push(limit, offset);
-
-    const results = this.db.prepare(sql).all(...params) as SessionSummarySearchResult[];
-
-    // Normalize rank to score
-    if (results.length > 0) {
-      const minRank = Math.min(...results.map(r => r.rank || 0));
-      const maxRank = Math.max(...results.map(r => r.rank || 0));
-      const range = maxRank - minRank || 1;
-
-      results.forEach(r => {
-        if (r.rank !== undefined) {
-          r.score = 1 - ((r.rank - minRank) / range);
-        }
-      });
+      params.push(limit, offset);
+      return this.db.prepare(sql).all(...params) as SessionSummarySearchResult[];
     }
 
-    return results;
+    // Vector search with query text should be handled by ChromaDB
+    // This method only supports filter-only queries (query=undefined)
+    console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
+    return [];
   }
 
   /**
@@ -485,16 +441,13 @@ export class SessionSearch {
   }
 
   /**
-   * Search user prompts with full-text search
+   * Search user prompts using filter-only direct SQLite query.
+   * Vector search is handled by ChromaDB - this only supports filtering without query text.
    */
-  searchUserPrompts(query: string, options: SearchOptions = {}): UserPromptSearchResult[] {
+  searchUserPrompts(query: string | undefined, options: SearchOptions = {}): UserPromptSearchResult[] {
     const params: any[] = [];
     const { limit = 20, offset = 0, orderBy = 'relevance', ...filters } = options;
 
-    // Build FTS5 match query
-    const ftsQuery = this.escapeFTS5(query);
-    params.push(ftsQuery);
-
     // Build filter conditions (join with sdk_sessions for project filtering)
     const baseConditions: string[] = [];
     if (filters.project) {
@@ -516,47 +469,34 @@ export class SessionSearch {
       }
     }
 
-    const whereClause = baseConditions.length > 0 ? `AND ${baseConditions.join(' AND ')}` : '';
+    // FILTER-ONLY PATH: When no query text, query user_prompts table directly
+    if (!query) {
+      if (baseConditions.length === 0) {
+        throw new Error('Either query or filters required for search');
+      }
 
-    // Build ORDER BY
-    const orderClause = orderBy === 'relevance'
-      ? 'ORDER BY user_prompts_fts.rank ASC'
-      : orderBy === 'date_asc'
-      ? 'ORDER BY up.created_at_epoch ASC'
-      : 'ORDER BY up.created_at_epoch DESC';
+      const whereClause = `WHERE ${baseConditions.join(' AND ')}`;
+      const orderClause = orderBy === 'date_asc'
+        ? 'ORDER BY up.created_at_epoch ASC'
+        : 'ORDER BY up.created_at_epoch DESC';
 
-    // Main query with FTS5 (join sdk_sessions for project filtering)
-    const sql = `
-      SELECT
-        up.*,
-        user_prompts_fts.rank as rank
-      FROM user_prompts up
-      JOIN user_prompts_fts ON up.id = user_prompts_fts.rowid
-      JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
-      WHERE user_prompts_fts MATCH ?
-      ${whereClause}
-      ${orderClause}
-      LIMIT ? OFFSET ?
-    `;
+      const sql = `
+        SELECT up.*
+        FROM user_prompts up
+        JOIN sdk_sessions s ON up.claude_session_id = s.claude_session_id
+        ${whereClause}
+        ${orderClause}
+        LIMIT ? OFFSET ?
+      `;
 
-    params.push(limit, offset);
-
-    const results = this.db.prepare(sql).all(...params) as UserPromptSearchResult[];
-
-    // Normalize rank to score
-    if (results.length > 0) {
-      const minRank = Math.min(...results.map(r => r.rank || 0));
-      const maxRank = Math.max(...results.map(r => r.rank || 0));
-      const range = maxRank - minRank || 1;
-
-      results.forEach(r => {
-        if (r.rank !== undefined) {
-          r.score = 1 - ((r.rank - minRank) / range);
-        }
-      });
+      params.push(limit, offset);
+      return this.db.prepare(sql).all(...params) as UserPromptSearchResult[];
     }
 
-    return results;
+    // Vector search with query text should be handled by ChromaDB
+    // This method only supports filter-only queries (query=undefined)
+    console.warn('[SessionSearch] Text search not supported - use ChromaDB for vector search');
+    return [];
   }
 
   /**

src/services/worker-service.ts

@@ -155,6 +155,12 @@ export class WorkerService {
     this.app.get('/api/observations', this.handleGetObservations.bind(this));
     this.app.get('/api/summaries', this.handleGetSummaries.bind(this));
     this.app.get('/api/prompts', this.handleGetPrompts.bind(this));
+
+    // Fetch by ID
+    this.app.get('/api/observation/:id', this.handleGetObservationById.bind(this));
+    this.app.get('/api/session/:id', this.handleGetSessionById.bind(this));
+    this.app.get('/api/prompt/:id', this.handleGetPromptById.bind(this));
+
     this.app.get('/api/stats', this.handleGetStats.bind(this));
     this.app.get('/api/processing-status', this.handleGetProcessingStatus.bind(this));
     this.app.post('/api/processing', this.handleSetProcessing.bind(this));
@@ -168,6 +174,14 @@ export class WorkerService {
     this.app.post('/api/mcp/toggle', this.handleToggleMcp.bind(this));
 
     // Search API endpoints (for skill-based search)
+    // Unified endpoints (new consolidated API)
+    this.app.get('/api/search', this.handleUnifiedSearch.bind(this));
+    this.app.get('/api/timeline', this.handleUnifiedTimeline.bind(this));
+    this.app.get('/api/decisions', this.handleDecisions.bind(this));
+    this.app.get('/api/changes', this.handleChanges.bind(this));
+    this.app.get('/api/how-it-works', this.handleHowItWorks.bind(this));
+
+    // Backward compatibility endpoints (use /api/search with type param instead)
     this.app.get('/api/search/observations', this.handleSearchObservations.bind(this));
     this.app.get('/api/search/sessions', this.handleSearchSessions.bind(this));
     this.app.get('/api/search/prompts', this.handleSearchPrompts.bind(this));
@@ -223,7 +237,7 @@ export class WorkerService {
     await this.dbManager.initialize();
 
     // Connect to MCP search server
-    const searchServerPath = path.join(__dirname, '..', '..', 'plugin', 'scripts', 'search-server.mjs');
+    const searchServerPath = path.join(__dirname, '..', '..', 'plugin', 'scripts', 'search-server.cjs');
     const transport = new StdioClientTransport({
       command: 'node',
       args: [searchServerPath],
@@ -659,6 +673,87 @@ export class WorkerService {
     }
   }
 
+  /**
+   * Get observation by ID
+   * GET /api/observation/:id
+   */
+  private handleGetObservationById(req: Request, res: Response): void {
+    try {
+      const id = parseInt(req.params.id, 10);
+      if (isNaN(id)) {
+        res.status(400).json({ error: 'Invalid observation ID' });
+        return;
+      }
+
+      const store = this.dbManager.getSessionStore();
+      const observation = store.getObservationById(id);
+
+      if (!observation) {
+        res.status(404).json({ error: `Observation #${id} not found` });
+        return;
+      }
+
+      res.json(observation);
+    } catch (error) {
+      logger.failure('WORKER', 'Get observation by ID failed', {}, error as Error);
+      res.status(500).json({ error: (error as Error).message });
+    }
+  }
+
+  /**
+   * Get session by ID
+   * GET /api/session/:id
+   */
+  private handleGetSessionById(req: Request, res: Response): void {
+    try {
+      const id = parseInt(req.params.id, 10);
+      if (isNaN(id)) {
+        res.status(400).json({ error: 'Invalid session ID' });
+        return;
+      }
+
+      const store = this.dbManager.getSessionStore();
+      const sessions = store.getSessionSummariesByIds([id]);
+
+      if (sessions.length === 0) {
+        res.status(404).json({ error: `Session #${id} not found` });
+        return;
+      }
+
+      res.json(sessions[0]);
+    } catch (error) {
+      logger.failure('WORKER', 'Get session by ID failed', {}, error as Error);
+      res.status(500).json({ error: (error as Error).message });
+    }
+  }
+
+  /**
+   * Get user prompt by ID
+   * GET /api/prompt/:id
+   */
+  private handleGetPromptById(req: Request, res: Response): void {
+    try {
+      const id = parseInt(req.params.id, 10);
+      if (isNaN(id)) {
+        res.status(400).json({ error: 'Invalid prompt ID' });
+        return;
+      }
+
+      const store = this.dbManager.getSessionStore();
+      const prompts = store.getUserPromptsByIds([id]);
+
+      if (prompts.length === 0) {
+        res.status(404).json({ error: `Prompt #${id} not found` });
+        return;
+      }
+
+      res.json(prompts[0]);
+    } catch (error) {
+      logger.failure('WORKER', 'Get prompt by ID failed', {}, error as Error);
+      res.status(500).json({ error: (error as Error).message });
+    }
+  }
+
   /**
    * Get database statistics (with worker metadata)
    */
@@ -944,8 +1039,102 @@ export class WorkerService {
   // Search API Handlers (for skill-based search)
   // ============================================================================
 
+  // ============================================================================
+  // Unified Search API Handlers (New Consolidated API)
+  // ============================================================================
+
   /**
-   * Search observations
+   * Unified search across all memory types (observations, sessions, prompts)
+   * GET /api/search?query=...&format=index&limit=20
+   */
+  private async handleUnifiedSearch(req: Request, res: Response): Promise<void> {
+    try {
+      const result = await this.mcpClient.callTool({
+        name: 'search',
+        arguments: req.query
+      });
+      res.json(result.content);
+    } catch (error) {
+      logger.failure('WORKER', 'Unified search failed', {}, error as Error);
+      res.status(500).json({ error: (error as Error).message });
+    }
+  }
+
+  /**
+   * Unified timeline (anchor or query-based)
+   * GET /api/timeline?anchor=123 OR GET /api/timeline?query=...
+   */
+  private async handleUnifiedTimeline(req: Request, res: Response): Promise<void> {
+    try {
+      const result = await this.mcpClient.callTool({
+        name: 'timeline',
+        arguments: req.query
+      });
+      res.json(result.content);
+    } catch (error) {
+      logger.failure('WORKER', 'Unified timeline failed', {}, error as Error);
+      res.status(500).json({ error: (error as Error).message });
+    }
+  }
+
+  /**
+   * Semantic shortcut for finding decision observations
+   * GET /api/decisions?format=index&limit=20
+   */
+  private async handleDecisions(req: Request, res: Response): Promise<void> {
+    try {
+      const result = await this.mcpClient.callTool({
+        name: 'decisions',
+        arguments: req.query
+      });
+      res.json(result.content);
+    } catch (error) {
+      logger.failure('WORKER', 'Decisions search failed', {}, error as Error);
+      res.status(500).json({ error: (error as Error).message });
+    }
+  }
+
+  /**
+   * Semantic shortcut for finding change-related observations
+   * GET /api/changes?format=index&limit=20
+   */
+  private async handleChanges(req: Request, res: Response): Promise<void> {
+    try {
+      const result = await this.mcpClient.callTool({
+        name: 'changes',
+        arguments: req.query
+      });
+      res.json(result.content);
+    } catch (error) {
+      logger.failure('WORKER', 'Changes search failed', {}, error as Error);
+      res.status(500).json({ error: (error as Error).message });
+    }
+  }
+
+  /**
+   * Semantic shortcut for finding "how it works" explanations
+   * GET /api/how-it-works?format=index&limit=20
+   */
+  private async handleHowItWorks(req: Request, res: Response): Promise<void> {
+    try {
+      const result = await this.mcpClient.callTool({
+        name: 'how_it_works',
+        arguments: req.query
+      });
+      res.json(result.content);
+    } catch (error) {
+      logger.failure('WORKER', 'How it works search failed', {}, error as Error);
+      res.status(500).json({ error: (error as Error).message });
+    }
+  }
+
+  // ============================================================================
+  // Backward Compatibility API Handlers
+  // All functionality available via /api/search with type/obs_type/concepts/files params
+  // ============================================================================
+
+  /**
+   * Search observations (use /api/search?type=observations instead)
    * GET /api/search/observations?query=...&format=index&limit=20&project=...
    */
   private async handleSearchObservations(req: Request, res: Response): Promise<void> {

test-results/run-search-tests.sh

@@ -0,0 +1,86 @@
+#!/bin/bash
+
+# Comprehensive Search API Test Suite
+# Tests all endpoints and parameter combinations
+
+API_URL="http://localhost:37777"
+RESULTS_DIR="test-results"
+
+echo "🔍 Starting comprehensive search API tests..."
+echo ""
+
+# SEMANTIC QUERIES - Understanding how things work
+echo "📚 Running semantic queries..."
+curl -s "$API_URL/api/search?type=observations&query=worker%20service%20startup&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-01-worker-service-startup.json"
+curl -s "$API_URL/api/search?type=observations&query=SQLite%20FTS5%20implementation&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-02-sqlite-fts5-implementation.json"
+curl -s "$API_URL/api/search?type=observations&query=hook%20lifecycle%20flow&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-03-hook-lifecycle-flow.json"
+curl -s "$API_URL/api/search?type=observations&query=build%20pipeline%20process&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-04-build-pipeline-process.json"
+echo "✅ Semantic queries complete (4 tests)"
+
+# DECISION QUERIES - Architectural choices
+echo "⚖️  Running decision queries..."
+curl -s "$API_URL/api/search?type=observations&obs_type=decision&query=PM2%20instead%20of%20direct%20process&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-05-pm2-decision.json"
+curl -s "$API_URL/api/search?type=observations&obs_type=decision&query=search%20architecture%20guidelines&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-06-search-architecture-decision.json"
+curl -s "$API_URL/api/search?type=observations&obs_type=decision&query=MCP%20as%20DRY%20source&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-07-mcp-dry-decision.json"
+echo "✅ Decision queries complete (3 tests)"
+
+# TROUBLESHOOTING QUERIES - Finding bugfixes
+echo "🔴 Running troubleshooting queries..."
+curl -s "$API_URL/api/search?type=observations&obs_type=bugfix&query=worker%20service%20debugging&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-08-worker-debugging.json"
+curl -s "$API_URL/api/search?type=observations&query=hook%20timeout%20problems&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-09-hook-timeout.json"
+curl -s "$API_URL/api/search?type=observations&query=database%20migration%20issues&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-10-database-migration.json"
+echo "✅ Troubleshooting queries complete (3 tests)"
+
+# FILE-SPECIFIC QUERIES - Tracking file changes
+echo "📁 Running file-specific queries..."
+curl -s "$API_URL/api/search?type=observations&files=search-server.ts&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-11-search-server-changes.json"
+curl -s "$API_URL/api/search?type=observations&files=context-hook&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-12-context-hook-changes.json"
+curl -s "$API_URL/api/search?type=observations&files=worker-service&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-13-worker-service-changes.json"
+echo "✅ File-specific queries complete (3 tests)"
+
+# CONCEPT-BASED QUERIES - Patterns, gotchas, discoveries
+echo "🏷️  Running concept-based queries..."
+curl -s "$API_URL/api/search?type=observations&concepts=pattern&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-14-patterns.json"
+curl -s "$API_URL/api/search?type=observations&concepts=gotcha&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-15-gotchas.json"
+curl -s "$API_URL/api/search?type=observations&concepts=discovery&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-16-discoveries.json"
+echo "✅ Concept-based queries complete (3 tests)"
+
+# TYPE-FILTERED QUERIES - Bugfixes, features, decisions
+echo "🔖 Running type-filtered queries..."
+curl -s "$API_URL/api/search?type=observations&obs_type=bugfix&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-17-all-bugfixes.json"
+curl -s "$API_URL/api/search?type=observations&obs_type=feature&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-18-all-features.json"
+curl -s "$API_URL/api/search?type=observations&obs_type=decision&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-19-all-decisions.json"
+echo "✅ Type-filtered queries complete (3 tests)"
+
+# SESSION QUERIES - Testing session search
+echo "📝 Running session queries..."
+curl -s "$API_URL/api/search?type=sessions&query=search%20architecture&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-20-session-search.json"
+echo "✅ Session queries complete (1 test)"
+
+# USER PROMPT QUERIES - Testing prompt search
+echo "💬 Running user prompt queries..."
+curl -s "$API_URL/api/search?type=prompts&query=build%20and%20deploy&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-21-prompt-search.json"
+echo "✅ User prompt queries complete (1 test)"
+
+# DEDICATED ENDPOINTS - Timeline and semantic shortcuts
+echo "🎯 Running dedicated endpoint tests..."
+curl -s "$API_URL/api/decisions?format=full&limit=5" > "$RESULTS_DIR/test-22-decisions-endpoint.json"
+curl -s "$API_URL/api/changes?format=full&limit=5" > "$RESULTS_DIR/test-23-changes-endpoint.json"
+curl -s "$API_URL/api/how-it-works?format=full&limit=5" > "$RESULTS_DIR/test-24-how-it-works-endpoint.json"
+curl -s "$API_URL/api/contextualize?format=full" > "$RESULTS_DIR/test-25-contextualize-endpoint.json"
+echo "✅ Dedicated endpoint tests complete (4 tests)"
+
+# TIMELINE QUERY - Get context around a specific observation
+echo "⏱️  Running timeline query..."
+curl -s "$API_URL/api/timeline?anchor=10630&depth_before=3&depth_after=3&format=full" > "$RESULTS_DIR/test-26-timeline-around-observation.json"
+echo "✅ Timeline query complete (1 test)"
+
+# MULTI-PARAMETER COMBO - Test complex query combinations
+echo "🎛️  Running multi-parameter combination tests..."
+curl -s "$API_URL/api/search?type=observations&obs_type=decision&concepts=pattern&query=search&format=full&limit=5&orderBy=relevance" > "$RESULTS_DIR/test-27-multi-param-combo.json"
+curl -s "$API_URL/api/search?type=observations&files=search-server&obs_type=feature&format=full&limit=5&orderBy=date_desc" > "$RESULTS_DIR/test-28-file-type-combo.json"
+echo "✅ Multi-parameter tests complete (2 tests)"
+
+echo ""
+echo "✨ All tests complete! 28 total queries executed."
+echo "📊 Results saved to $RESULTS_DIR/"

test-results/test-21-prompt-search.json

@@ -0,0 +1 @@
+[{"type":"text","text":"## User Prompt #5\n*Source: claude-mem://user-prompt/2735*\n\nAre you ok? You seem a little excited and it's hard to do this project. Can you tell me what's going on?\n\n---\nDate: 11/17/2025, 9:38:06 PM\n\n---\n\n## User Prompt #2\n*Source: claude-mem://user-prompt/2719*\n\nedit the build script, we need it to always do this.\n\n---\nDate: 11/17/2025, 7:49:17 PM\n\n---\n\n## User Prompt #3\n*Source: claude-mem://user-prompt/2662*\n\nthose built files, the js, the cjs, if the source files don't have merge issues, then these are just way outdated build files.... is that the case?\n\n---\nDate: 11/17/2025, 3:03:13 PM\n\n---\n\n## User Prompt #12\n*Source: claude-mem://user-prompt/2651*\n\nI FUCKING DID BUILD AND SYNC AND DELETE. I would not have told you to fucking check pm2 info had I not done that\n\n---\nDate: 11/17/2025, 2:25:18 PM\n\n---\n\n## User Prompt #7\n*Source: claude-mem://user-prompt/2646*\n\nCan you fix it so that it runs from the marketplace folder so it doesn't break on EVERYONES SYSTEM WHO DOESNT FUCKING BUILD MANUALLY\n\n---\nDate: 11/17/2025, 2:19:29 PM"}]

test-results/test-25-contextualize-endpoint.json

@@ -0,0 +1 @@
+[{"type":"text","text":"# Project Context: claude-mem\n\nNo activity found for this project."}]

test-results/test-26-timeline-around-observation.json

@@ -0,0 +1 @@
+[{"type":"text","text":"# Timeline around anchor: 10630\n**Window:** 3 records before → 3 records after | **Items:** 3 (3 obs, 0 sessions, 0 prompts)\n\n**Legend:** 🎯 session-request | 🔴 bugfix | 🟣 feature | 🔄 refactor | ✅ change | 🔵 discovery | 🧠 decision\n\n### Nov 17, 2025\n\n**General**\n| ID | Time | T | Title | Tokens |\n|----|------|---|-------|--------|\n| #10756 | 11:50 PM | 🔴 | Fixed Incorrect Parameter Array in searchUserPrompts FTS5 Path | ~213 |\n| #10757 | ″ | 🔵 | Unified search handler implements Chroma-first with FTS5 fallback on zero results | ~224 |\n| #10758 | 11:51 PM | ✅ | Build and sync claude-mem plugin to marketplace location | ~189 |\n"}]