Files
claude-mem/scripts/investigate-timestamps.ts
Alex Newman 266c746d50 feat: Fix observation timestamps, refactor session management, and enhance worker reliability (#437)
* Refactor worker version checks and increase timeout settings

- Updated the default hook timeout from 5000ms to 120000ms for improved stability.
- Modified the worker version check to log a warning instead of restarting the worker on version mismatch.
- Removed legacy PM2 cleanup and worker start logic, simplifying the ensureWorkerRunning function.
- Enhanced polling mechanism for worker readiness with increased retries and reduced interval.

* feat: implement worker queue polling to ensure processing completion before proceeding

* refactor: change worker command from start to restart in hooks configuration

* refactor: remove session management complexity

- Simplify createSDKSession to pure INSERT OR IGNORE
- Remove auto-create logic from storeObservation/storeSummary
- Delete 11 unused session management methods
- Derive prompt_number from user_prompts count
- Keep sdk_sessions table schema unchanged for compatibility

* refactor: simplify session management by removing unused methods and auto-creation logic

* Refactor session prompt number retrieval in SessionRoutes

- Updated the method of obtaining the prompt number from the session.
- Replaced `store.getPromptCounter(sessionDbId)` with `store.getPromptNumberFromUserPrompts(claudeSessionId)` for better clarity and accuracy.
- Adjusted the logic for incrementing the prompt number to derive it from the user prompts count instead of directly incrementing a counter.

* refactor: replace getPromptCounter with getPromptNumberFromUserPrompts in SessionManager

Phase 7 of session management simplification. Updates SessionManager to derive
prompt numbers from user_prompts table count instead of using the deprecated
prompt_counter column.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: simplify SessionCompletionHandler to use direct SQL query

Phase 8: Remove call to findActiveSDKSession() and replace with direct
database query in SessionCompletionHandler.completeByClaudeId().

This removes dependency on the deleted findActiveSDKSession() method
and simplifies the code by using a straightforward SELECT query.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: remove markSessionCompleted call from SDKAgent

- Delete call to markSessionCompleted() in SDKAgent.ts
- Session status is no longer tracked or updated
- Part of phase 9: simplifying session management

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: remove markSessionComplete method (Phase 10)

- Deleted markSessionComplete() method from DatabaseManager
- Removed markSessionComplete call from SessionCompletionHandler
- Session completion status no longer tracked in database
- Part of session management simplification effort

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* refactor: replace deleted updateSDKSessionId calls in import script (Phase 11)

- Replace updateSDKSessionId() calls with direct SQL UPDATE statements
- Method was deleted in Phase 3 as part of session management simplification
- Import script now uses direct database access consistently

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* test: add validation for SQL updates in sdk_sessions table

* refactor: enhance worker-cli to support manual and automated runs

* Remove cleanup hook and associated session completion logic

- Deleted the cleanup-hook implementation from the hooks directory.
- Removed the session completion endpoint that was used by the cleanup hook.
- Updated the SessionCompletionHandler to eliminate the completeByClaudeId method and its dependencies.
- Adjusted the SessionRoutes to reflect the removal of the session completion route.

* fix: update worker-cli command to use bun for consistency

* feat: Implement timestamp fix for observations and enhance processing logic

- Added `earliestPendingTimestamp` to `ActiveSession` to track the original timestamp of the earliest pending message.
- Updated `SDKAgent` to capture and utilize the earliest pending timestamp during response processing.
- Modified `SessionManager` to track the earliest timestamp when yielding messages.
- Created scripts for fixing corrupted timestamps, validating fixes, and investigating timestamp issues.
- Verified that all corrupted observations have been repaired and logic for future processing is sound.
- Ensured orphan processing can be safely re-enabled after validation.

* feat: Enhance SessionStore to support custom database paths and add timestamp fields for observations and summaries

* Refactor pending queue processing and add management endpoints

- Disabled automatic recovery of orphaned queues on startup; users must now use the new /api/pending-queue/process endpoint.
- Updated processOrphanedQueues method to processPendingQueues with improved session handling and return detailed results.
- Added new API endpoints for managing pending queues: GET /api/pending-queue and POST /api/pending-queue/process.
- Introduced a new script (check-pending-queue.ts) for checking and processing pending observation queues interactively or automatically.
- Enhanced logging and error handling for better monitoring of session processing.

* updated agent sdk

* feat: Add manual recovery guide and queue management endpoints to documentation

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-25 15:36:46 -05:00

144 lines
4.7 KiB
TypeScript
Executable File

#!/usr/bin/env bun
/**
* Investigate Timestamp Situation
*
* This script investigates the actual state of observations and pending messages
* to understand what happened with the timestamp corruption.
*/
import Database from 'bun:sqlite';
import { resolve } from 'path';
const DB_PATH = resolve(process.env.HOME!, '.claude-mem/claude-mem.db');
function formatTimestamp(epoch: number): string {
return new Date(epoch).toLocaleString('en-US', {
timeZone: 'America/Los_Angeles',
year: 'numeric',
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
second: '2-digit'
});
}
function main() {
console.log('🔍 Investigating timestamp situation...\n');
const db = new Database(DB_PATH);
try {
// Check 1: Recent observations on Dec 24
console.log('Check 1: All observations created on Dec 24, 2025...');
const dec24Start = 1735027200000; // Dec 24 00:00 PST
const dec24End = 1735113600000; // Dec 25 00:00 PST
const dec24Obs = db.query(`
SELECT id, sdk_session_id, created_at_epoch, title
FROM observations
WHERE created_at_epoch >= ${dec24Start}
AND created_at_epoch < ${dec24End}
ORDER BY created_at_epoch
LIMIT 100
`).all();
console.log(`Found ${dec24Obs.length} observations on Dec 24:\n`);
for (const obs of dec24Obs.slice(0, 20)) {
console.log(` #${obs.id}: ${formatTimestamp(obs.created_at_epoch)} - ${obs.title || '(no title)'}`);
}
if (dec24Obs.length > 20) {
console.log(` ... and ${dec24Obs.length - 20} more`);
}
console.log();
// Check 2: Observations from Dec 17-20
console.log('Check 2: Observations from Dec 17-20, 2025...');
const dec17Start = 1734422400000; // Dec 17 00:00 PST
const dec21Start = 1734768000000; // Dec 21 00:00 PST
const oldObs = db.query(`
SELECT id, sdk_session_id, created_at_epoch, title
FROM observations
WHERE created_at_epoch >= ${dec17Start}
AND created_at_epoch < ${dec21Start}
ORDER BY created_at_epoch
LIMIT 100
`).all();
console.log(`Found ${oldObs.length} observations from Dec 17-20:\n`);
for (const obs of oldObs.slice(0, 20)) {
console.log(` #${obs.id}: ${formatTimestamp(obs.created_at_epoch)} - ${obs.title || '(no title)'}`);
}
if (oldObs.length > 20) {
console.log(` ... and ${oldObs.length - 20} more`);
}
console.log();
// Check 3: Pending messages status
console.log('Check 3: Pending messages status...');
const statusCounts = db.query(`
SELECT status, COUNT(*) as count
FROM pending_messages
GROUP BY status
`).all();
console.log('Pending message counts by status:');
for (const row of statusCounts) {
console.log(` ${row.status}: ${row.count}`);
}
console.log();
// Check 4: Old pending messages from Dec 17-20
console.log('Check 4: Pending messages from Dec 17-20...');
const oldMessages = db.query(`
SELECT id, session_db_id, tool_name, status, created_at_epoch, completed_at_epoch
FROM pending_messages
WHERE created_at_epoch >= ${dec17Start}
AND created_at_epoch < ${dec21Start}
ORDER BY created_at_epoch
LIMIT 50
`).all();
console.log(`Found ${oldMessages.length} pending messages from Dec 17-20:\n`);
for (const msg of oldMessages.slice(0, 20)) {
const completedAt = msg.completed_at_epoch ? formatTimestamp(msg.completed_at_epoch) : 'N/A';
console.log(` #${msg.id}: ${msg.tool_name} - Status: ${msg.status}`);
console.log(` Created: ${formatTimestamp(msg.created_at_epoch)}`);
console.log(` Completed: ${completedAt}\n`);
}
if (oldMessages.length > 20) {
console.log(` ... and ${oldMessages.length - 20} more`);
}
// Check 5: Recently completed pending messages
console.log('Check 5: Recently completed pending messages...');
const recentCompleted = db.query(`
SELECT id, session_db_id, tool_name, status, created_at_epoch, completed_at_epoch
FROM pending_messages
WHERE completed_at_epoch IS NOT NULL
ORDER BY completed_at_epoch DESC
LIMIT 20
`).all();
console.log(`Most recent completed pending messages:\n`);
for (const msg of recentCompleted) {
const createdAt = formatTimestamp(msg.created_at_epoch);
const completedAt = formatTimestamp(msg.completed_at_epoch);
const lag = Math.round((msg.completed_at_epoch - msg.created_at_epoch) / 1000);
console.log(` #${msg.id}: ${msg.tool_name} (${msg.status})`);
console.log(` Created: ${createdAt}`);
console.log(` Completed: ${completedAt} (${lag}s later)\n`);
}
} catch (error) {
console.error('❌ Error:', error);
process.exit(1);
} finally {
db.close();
}
}
main();