MAESTRO: Add Docker E2E test against real OpenClaw gateway

Installs plugin on ghcr.io/openclaw/openclaw:main via `plugins install`,
starts mock worker + gateway, and verifies 16 checks (discovery, files,
SSE connectivity, gateway plugin load). Includes interactive mode for
human manual testing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Alex Newman
2026-02-07 21:28:16 -05:00
parent 1b9f601c41
commit 33ab7ba747
4 changed files with 601 additions and 104 deletions

69
openclaw/Dockerfile.e2e Normal file
View File

@@ -0,0 +1,69 @@
# Dockerfile.e2e — End-to-end test: install claude-mem plugin on a real OpenClaw instance
# Simulates the complete plugin installation flow a user would follow.
#
# Usage:
# docker build -f Dockerfile.e2e -t openclaw-e2e-test . && docker run --rm openclaw-e2e-test
#
# Interactive (for human testing):
# docker run --rm -it openclaw-e2e-test /bin/bash
FROM ghcr.io/openclaw/openclaw:main
USER root
# Install curl for health checks in e2e-verify.sh, and TypeScript for building
RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/*
RUN npm install -g typescript@5
# Create staging directory for the plugin source
WORKDIR /tmp/claude-mem-plugin
# Copy plugin source files
COPY package.json tsconfig.json openclaw.plugin.json ./
COPY src/ ./src/
# Build the plugin (TypeScript → JavaScript)
# NODE_ENV=production is set in the base image; override to install devDependencies
RUN NODE_ENV=development npm install && npx tsc
# Create the installable plugin package:
# OpenClaw `plugins install` expects package.json with openclaw.extensions field.
# The package name must match the plugin ID in openclaw.plugin.json (claude-mem).
# Only include the main plugin entry point, not test/mock files.
RUN mkdir -p /tmp/claude-mem-installable/dist && \
cp dist/index.js /tmp/claude-mem-installable/dist/ && \
cp dist/index.d.ts /tmp/claude-mem-installable/dist/ 2>/dev/null || true && \
cp openclaw.plugin.json /tmp/claude-mem-installable/ && \
node -e " \
const pkg = { \
name: 'claude-mem', \
version: '1.0.0', \
type: 'module', \
main: 'dist/index.js', \
openclaw: { extensions: ['./dist/index.js'] } \
}; \
require('fs').writeFileSync('/tmp/claude-mem-installable/package.json', JSON.stringify(pkg, null, 2)); \
"
# Switch back to app directory and node user for installation
WORKDIR /app
USER node
# Create the OpenClaw config directory
RUN mkdir -p /home/node/.openclaw
# Install the plugin using OpenClaw's official CLI
RUN node openclaw.mjs plugins install /tmp/claude-mem-installable
# Enable the plugin
RUN node openclaw.mjs plugins enable claude-mem
# Copy the e2e verification script and mock worker
COPY --chown=node:node e2e-verify.sh /app/e2e-verify.sh
USER root
RUN chmod +x /app/e2e-verify.sh && \
cp /tmp/claude-mem-plugin/dist/mock-worker.js /app/mock-worker.js
USER node
# Default: run the automated verification
CMD ["/bin/bash", "/app/e2e-verify.sh"]

View File

@@ -1,109 +1,241 @@
# OpenClaw Claude-Mem Plugin — Manual E2E Testing Checklist
# OpenClaw Claude-Mem Plugin — Testing Guide
This document covers end-to-end verification of the OpenClaw claude-mem plugin. It assumes you have a working OpenClaw gateway and a running claude-mem worker.
## Quick Start (Docker)
The fastest way to test the plugin is using the pre-built Docker E2E environment:
```bash
cd openclaw
# Automated test (builds, installs plugin on real OpenClaw, verifies everything)
./test-e2e.sh
# Interactive shell (for manual exploration)
./test-e2e.sh --interactive
# Just build the image
./test-e2e.sh --build-only
```
---
## Prerequisites
## Test Layers
### 1. Unit Tests (fastest)
```bash
cd openclaw
npm test # compiles TypeScript, runs 17 tests
```
Tests plugin registration, service lifecycle, command handling, SSE integration, and all 6 channel types.
### 2. Smoke Test
```bash
node test-sse-consumer.js
```
Quick check that the plugin loads and registers its service + command correctly.
### 3. Container Unit Tests (fresh install)
```bash
./test-container.sh # Unit tests in clean Docker
./test-container.sh --full # Integration tests with mock worker
```
### 4. E2E on Real OpenClaw (Docker)
```bash
./test-e2e.sh
```
This is the most comprehensive test. It:
1. Uses the official `ghcr.io/openclaw/openclaw:main` Docker image
2. Installs the plugin via `openclaw plugins install` (same as a real user)
3. Enables the plugin via `openclaw plugins enable`
4. Starts a mock claude-mem worker on port 37777
5. Starts the OpenClaw gateway with plugin config
6. Verifies the plugin loads, connects to SSE, and processes events
**All 16 checks must pass.**
---
## Human E2E Testing (Interactive Docker)
For manual walkthrough testing, use the interactive Docker mode:
```bash
./test-e2e.sh --interactive
```
This drops you into a fully-configured OpenClaw container with the plugin pre-installed.
### Step-by-step inside the container
#### 1. Verify plugin is installed
```bash
node openclaw.mjs plugins list
node openclaw.mjs plugins info claude-mem
node openclaw.mjs plugins doctor
```
**Expected:**
- `claude-mem` appears in the plugins list as "enabled" or "loaded"
- Info shows version 1.0.0, source at `/home/node/.openclaw/extensions/claude-mem/`
- Doctor reports no issues
#### 2. Inspect plugin files
```bash
ls -la /home/node/.openclaw/extensions/claude-mem/
cat /home/node/.openclaw/extensions/claude-mem/openclaw.plugin.json
cat /home/node/.openclaw/extensions/claude-mem/package.json
```
**Expected:**
- `dist/index.js` exists (compiled plugin)
- `openclaw.plugin.json` has `"id": "claude-mem"` and `"kind": "memory"`
- `package.json` has `openclaw.extensions` field pointing to `./dist/index.js`
#### 3. Start mock worker
```bash
node /app/mock-worker.js &
```
Verify it's running:
```bash
curl -s http://localhost:37777/health
# → {"status":"ok"}
curl -s --max-time 3 http://localhost:37777/stream
# → data: {"type":"connected","message":"Mock worker SSE stream"}
# → data: {"type":"new_observation","observation":{...}}
```
#### 4. Configure and start gateway
```bash
cat > /home/node/.openclaw/openclaw.json << 'EOF'
{
"gateway": {
"mode": "local",
"auth": {
"mode": "token",
"token": "e2e-test-token"
}
},
"plugins": {
"slots": {
"memory": "claude-mem"
},
"entries": {
"claude-mem": {
"enabled": true,
"config": {
"workerPort": 37777,
"observationFeed": {
"enabled": true,
"channel": "telegram",
"to": "test-chat-id-12345"
}
}
}
}
}
}
EOF
node openclaw.mjs gateway --allow-unconfigured --verbose --token e2e-test-token
```
**Expected in gateway logs:**
- `[claude-mem] OpenClaw plugin loaded — v1.0.0`
- `[claude-mem] Observation feed starting — channel: telegram, target: test-chat-id-12345`
- `[claude-mem] Connecting to SSE stream at http://localhost:37777/stream`
- `[claude-mem] Connected to SSE stream`
#### 5. Run automated verification (optional)
From a second shell in the container (or after stopping the gateway):
```bash
/bin/bash /app/e2e-verify.sh
```
---
## Manual E2E (Real OpenClaw + Real Worker)
For testing with a real claude-mem worker and real messaging channel:
### Prerequisites
- OpenClaw gateway installed and configured
- Claude-Mem worker running on port 37777 (default)
- Claude-Mem worker running on port 37777
- Plugin built: `cd openclaw && npm run build`
- Plugin registered in `~/.openclaw/openclaw.json`
---
## 1. Verify the Claude-Mem Worker
### 1. Install the plugin
```bash
# Health check — should return {"status":"ok"}
curl -s http://localhost:37777/health
# Build the plugin
cd openclaw && npm run build
# Verify SSE stream is active (will print events for ~3 seconds then exit)
curl -s -N http://localhost:37777/stream --max-time 3 2>/dev/null || true
# Install on OpenClaw (from the openclaw/ directory)
openclaw plugins install .
# Enable it
openclaw plugins enable claude-mem
```
**Expected:** Health returns `{"status":"ok"}`. SSE stream emits at least a `connected` event.
### 2. Configure
**If the worker is not running:**
```bash
cd /path/to/claude-mem
npm run build-and-sync
```
Then re-check health.
---
## 2. Verify Plugin Configuration
Check that `~/.openclaw/openclaw.json` has the plugin entry:
```bash
cat ~/.openclaw/openclaw.json
```
**Expected structure** (inside `plugins.entries`):
Edit `~/.openclaw/openclaw.json` to add plugin config:
```json
{
"claude-mem": {
"enabled": true,
"source": "/path/to/claude-mem/openclaw",
"config": {
"syncMemoryFile": true,
"workerPort": 37777,
"observationFeed": {
"plugins": {
"entries": {
"claude-mem": {
"enabled": true,
"channel": "telegram",
"to": "YOUR_CHAT_ID"
"config": {
"workerPort": 37777,
"observationFeed": {
"enabled": true,
"channel": "telegram",
"to": "YOUR_CHAT_ID"
}
}
}
}
}
}
```
**Key fields:**
- `observationFeed.enabled` must be `true`
- `observationFeed.channel` must match a supported channel: `telegram`, `discord`, `signal`, `slack`, `whatsapp`, `line`
- `observationFeed.to` must be the target chat/user/channel ID for the chosen channel
**Supported channels:** `telegram`, `discord`, `signal`, `slack`, `whatsapp`, `line`
---
## 3. Restart the OpenClaw Gateway
After any config change, restart the gateway so it picks up the new plugin config:
### 3. Restart gateway
```bash
openclaw restart
# or, depending on your setup:
openclaw gateway stop && openclaw gateway start
```
**Look for in gateway logs:**
**Look for in logs:**
- `[claude-mem] OpenClaw plugin loaded — v1.0.0`
- `[claude-mem] Observation feed starting — channel: telegram, target: ...`
- `[claude-mem] Connecting to SSE stream at http://localhost:37777/stream`
- `[claude-mem] Connected to SSE stream`
---
### 4. Trigger an observation
## 4. Trigger an Observation
Start a Claude Code session with claude-mem enabled and perform any action. The worker will emit a `new_observation` SSE event.
Start a Claude Code session with claude-mem enabled:
### 5. Verify delivery
```bash
claude
```
Perform any action that generates an observation (e.g., read a file, make a search, write code). The claude-mem worker will emit a `new_observation` SSE event.
---
## 5. Verify Message Delivery
Check the target messaging channel (e.g., Telegram) for a message formatted as:
Check the target messaging channel for:
```
🧠 Claude-Mem Observation
@@ -111,52 +243,37 @@ Check the target messaging channel (e.g., Telegram) for a message formatted as:
Optional subtitle
```
**Expected:** Within a few seconds of the observation being saved, a message appears in the configured channel.
---
## 6. Run Automated Tests
```bash
cd openclaw
# Full test suite (compiles TypeScript then runs tests)
npm test
# Smoke test (registration check only, requires prior build)
node test-sse-consumer.js
```
**Expected:** All 17 tests pass. Smoke test prints `PASS: Plugin registers service and command correctly`.
---
## Troubleshooting
### `api.log is not a function`
The plugin was built against the wrong API. Ensure `src/index.ts` uses `api.logger.info()` not `api.log()`. Rebuild with `npm run build`.
### Worker not running
- **Symptom:** Gateway logs show `SSE stream error: fetch failed. Reconnecting in 1s`
- **Fix:** Start the worker with `cd /path/to/claude-mem && npm run build-and-sync`
- **Symptom:** `SSE stream error: fetch failed. Reconnecting in 1s`
- **Fix:** Start the worker: `cd /path/to/claude-mem && npm run build-and-sync`
### Port mismatch
- **Symptom:** SSE connection fails even though worker health check passes
- **Fix:** Ensure `workerPort` in plugin config matches the worker's actual port (default: 37777). Check `~/.claude-mem/settings.json` for the worker port setting.
- **Fix:** Ensure `workerPort` in config matches the worker's actual port (default: 37777)
### Channel not configured
- **Symptom:** Gateway logs show `[claude-mem] Observation feed misconfigured — channel or target missing`
- **Fix:** Add both `channel` and `to` fields to `observationFeed` in plugin config. Restart the gateway.
- **Symptom:** `Observation feed misconfigured — channel or target missing`
- **Fix:** Add both `channel` and `to` to `observationFeed` in config
### Unknown channel type
- **Symptom:** Gateway logs show `[claude-mem] Unknown channel type: <name>`
- **Fix:** Use one of the supported channels: `telegram`, `discord`, `signal`, `slack`, `whatsapp`, `line`
- **Fix:** Use: `telegram`, `discord`, `signal`, `slack`, `whatsapp`, or `line`
### Feed disabled
- **Symptom:** Gateway logs show `[claude-mem] Observation feed disabled`
- **Fix:** Set `observationFeed.enabled` to `true` in plugin config. Restart the gateway.
- **Symptom:** `Observation feed disabled`
- **Fix:** Set `observationFeed.enabled: true`
### Messages not arriving
- **Symptom:** SSE connected, observations flowing, but no messages in chat
- **Fix:**
1. Verify the bot/integration is properly configured in the target channel
2. Check the target ID (`to`) is correct for the channel type
3. Look for `[claude-mem] Failed to send to <channel>: ...` in gateway logs
4. Test the channel directly through the OpenClaw gateway's channel testing tools
1. Verify the bot/integration is configured in the target channel
2. Check the target ID (`to`) is correct
3. Look for `Failed to send to <channel>` in logs
4. Test the channel via OpenClaw's built-in tools
### Memory slot conflict
- **Symptom:** `plugin disabled (memory slot set to "memory-core")`
- **Fix:** Add `"slots": { "memory": "claude-mem" }` to plugins config

265
openclaw/e2e-verify.sh Executable file
View File

@@ -0,0 +1,265 @@
#!/usr/bin/env bash
# e2e-verify.sh — Automated E2E verification for claude-mem plugin on OpenClaw
#
# This script verifies the complete plugin installation and operation flow:
# 1. Plugin is installed and visible in OpenClaw
# 2. Plugin loads correctly when gateway starts
# 3. Mock worker SSE stream is consumed by the plugin
# 4. Observations are received and formatted
#
# Exit 0 = all checks passed, Exit 1 = failure
set -euo pipefail
PASS=0
FAIL=0
TOTAL=0
pass() {
PASS=$((PASS + 1))
TOTAL=$((TOTAL + 1))
echo " PASS: $1"
}
fail() {
FAIL=$((FAIL + 1))
TOTAL=$((TOTAL + 1))
echo " FAIL: $1"
}
section() {
echo ""
echo "=== $1 ==="
}
# ─── Phase 1: Plugin Discovery ───
section "Phase 1: Plugin Discovery"
# Check plugin is listed
PLUGIN_LIST=$(node /app/openclaw.mjs plugins list 2>&1)
if echo "$PLUGIN_LIST" | grep -q "claude-mem"; then
pass "Plugin appears in 'plugins list'"
else
fail "Plugin NOT found in 'plugins list'"
echo "$PLUGIN_LIST"
fi
# Check plugin info
PLUGIN_INFO=$(node /app/openclaw.mjs plugins info claude-mem 2>&1 || true)
if echo "$PLUGIN_INFO" | grep -qi "claude-mem"; then
pass "Plugin info shows claude-mem details"
else
fail "Plugin info failed"
echo "$PLUGIN_INFO"
fi
# Check plugin is enabled
if echo "$PLUGIN_LIST" | grep -A1 "claude-mem" | grep -qi "enabled\|loaded"; then
pass "Plugin is enabled"
else
# Try to check via info
if echo "$PLUGIN_INFO" | grep -qi "enabled\|loaded"; then
pass "Plugin is enabled (via info)"
else
fail "Plugin does not appear enabled"
echo "$PLUGIN_INFO"
fi
fi
# Check plugin doctor reports no issues
DOCTOR_OUT=$(node /app/openclaw.mjs plugins doctor 2>&1 || true)
if echo "$DOCTOR_OUT" | grep -qi "no.*issue\|0 issue"; then
pass "Plugin doctor reports no issues"
else
fail "Plugin doctor reports issues"
echo "$DOCTOR_OUT"
fi
# ─── Phase 2: Plugin Files ───
section "Phase 2: Plugin Files"
# Check extension directory exists
EXTENSIONS_DIR="/home/node/.openclaw/extensions/openclaw-plugin"
if [ ! -d "$EXTENSIONS_DIR" ]; then
# Try alternative naming
EXTENSIONS_DIR="/home/node/.openclaw/extensions/claude-mem"
if [ ! -d "$EXTENSIONS_DIR" ]; then
# Search for it
FOUND_DIR=$(find /home/node/.openclaw/extensions/ -name "openclaw.plugin.json" -exec dirname {} \; 2>/dev/null | head -1 || true)
if [ -n "$FOUND_DIR" ]; then
EXTENSIONS_DIR="$FOUND_DIR"
fi
fi
fi
if [ -d "$EXTENSIONS_DIR" ]; then
pass "Plugin directory exists: $EXTENSIONS_DIR"
else
fail "Plugin directory not found under /home/node/.openclaw/extensions/"
ls -la /home/node/.openclaw/extensions/ 2>/dev/null || echo " (extensions dir not found)"
fi
# Check key files exist
for FILE in "openclaw.plugin.json" "dist/index.js" "package.json"; do
if [ -f "$EXTENSIONS_DIR/$FILE" ]; then
pass "File exists: $FILE"
else
fail "File missing: $FILE"
fi
done
# ─── Phase 3: Mock Worker + Plugin Integration ───
section "Phase 3: Mock Worker + Plugin Integration"
# Start mock worker in background
echo " Starting mock claude-mem worker..."
node /app/mock-worker.js &
MOCK_PID=$!
# Wait for mock worker to be ready
for i in $(seq 1 10); do
if curl -sf http://localhost:37777/health > /dev/null 2>&1; then
break
fi
sleep 0.5
done
if curl -sf http://localhost:37777/health > /dev/null 2>&1; then
pass "Mock worker health check passed"
else
fail "Mock worker health check failed"
kill $MOCK_PID 2>/dev/null || true
fi
# Test SSE stream connectivity (curl with max-time to capture initial SSE frame)
SSE_TEST=$(curl -s --max-time 2 http://localhost:37777/stream 2>/dev/null || true)
if echo "$SSE_TEST" | grep -q "connected"; then
pass "SSE stream returns connected event"
else
fail "SSE stream did not return connected event"
echo " Got: $(echo "$SSE_TEST" | head -5)"
fi
# ─── Phase 4: Gateway + Plugin Load ───
section "Phase 4: Gateway Startup with Plugin"
# Create a minimal config that enables the plugin with the mock worker.
# The memory slot must be set to "claude-mem" to match what `plugins install` configured.
# Gateway auth is disabled via token for headless testing.
mkdir -p /home/node/.openclaw
cat > /home/node/.openclaw/openclaw.json << 'EOFCONFIG'
{
"gateway": {
"mode": "local",
"auth": {
"mode": "token",
"token": "e2e-test-token"
}
},
"plugins": {
"slots": {
"memory": "claude-mem"
},
"entries": {
"claude-mem": {
"enabled": true,
"config": {
"workerPort": 37777,
"observationFeed": {
"enabled": true,
"channel": "telegram",
"to": "test-chat-id-12345"
}
}
}
}
}
}
EOFCONFIG
pass "OpenClaw config written with plugin enabled"
# Start gateway in background and capture output
GATEWAY_LOG="/tmp/gateway.log"
echo " Starting OpenClaw gateway (timeout 15s)..."
OPENCLAW_GATEWAY_TOKEN=e2e-test-token timeout 15 node /app/openclaw.mjs gateway --allow-unconfigured --verbose --token e2e-test-token > "$GATEWAY_LOG" 2>&1 &
GATEWAY_PID=$!
# Give the gateway time to start and load plugins
sleep 5
# Check if gateway started
if kill -0 $GATEWAY_PID 2>/dev/null; then
pass "Gateway process is running"
else
fail "Gateway process exited early"
echo " Gateway log:"
cat "$GATEWAY_LOG" 2>/dev/null | tail -30
fi
# Check gateway log for plugin load messages
if grep -qi "claude-mem" "$GATEWAY_LOG" 2>/dev/null; then
pass "Gateway log mentions claude-mem plugin"
else
fail "Gateway log does not mention claude-mem"
echo " Gateway log (last 20 lines):"
tail -20 "$GATEWAY_LOG" 2>/dev/null
fi
# Check for plugin loaded message
if grep -q "plugin loaded" "$GATEWAY_LOG" 2>/dev/null || grep -q "v1.0.0" "$GATEWAY_LOG" 2>/dev/null; then
pass "Plugin load message found in gateway log"
else
fail "Plugin load message not found"
fi
# Check for observation feed messages
if grep -qi "observation feed" "$GATEWAY_LOG" 2>/dev/null; then
pass "Observation feed activity in gateway log"
else
fail "No observation feed activity detected"
fi
# Check for SSE connection to mock worker
if grep -qi "connected.*SSE\|SSE.*stream\|connecting.*SSE" "$GATEWAY_LOG" 2>/dev/null; then
pass "SSE connection activity detected"
else
fail "No SSE connection activity in log"
fi
# ─── Cleanup ───
section "Cleanup"
kill $GATEWAY_PID 2>/dev/null || true
kill $MOCK_PID 2>/dev/null || true
wait $GATEWAY_PID 2>/dev/null || true
wait $MOCK_PID 2>/dev/null || true
echo " Processes stopped."
# ─── Summary ───
echo ""
echo "==============================="
echo " E2E Test Results"
echo "==============================="
echo " Total: $TOTAL"
echo " Passed: $PASS"
echo " Failed: $FAIL"
echo "==============================="
if [ "$FAIL" -gt 0 ]; then
echo ""
echo " SOME TESTS FAILED"
echo ""
echo " Full gateway log:"
cat "$GATEWAY_LOG" 2>/dev/null
exit 1
fi
echo ""
echo " ALL TESTS PASSED"
exit 0

46
openclaw/test-e2e.sh Executable file
View File

@@ -0,0 +1,46 @@
#!/usr/bin/env bash
# test-e2e.sh — Run E2E test of claude-mem plugin on real OpenClaw
#
# Usage:
# ./test-e2e.sh # Automated E2E test (build + run + verify)
# ./test-e2e.sh --interactive # Drop into shell for manual testing
# ./test-e2e.sh --build-only # Just build the image, don't run
set -euo pipefail
cd "$(dirname "$0")"
IMAGE_NAME="openclaw-claude-mem-e2e"
echo "=== Building E2E test image ==="
echo " Base: ghcr.io/openclaw/openclaw:main"
echo " Plugin: @claude-mem/openclaw-plugin (PR #1012)"
echo ""
docker build -f Dockerfile.e2e -t "$IMAGE_NAME" .
if [ "${1:-}" = "--build-only" ]; then
echo ""
echo "Image built: $IMAGE_NAME"
echo "Run manually with: docker run --rm $IMAGE_NAME"
exit 0
fi
echo ""
echo "=== Running E2E verification ==="
echo ""
if [ "${1:-}" = "--interactive" ]; then
echo "Dropping into interactive shell."
echo ""
echo "Useful commands inside the container:"
echo " node openclaw.mjs plugins list # Verify plugin is installed"
echo " node openclaw.mjs plugins info claude-mem # Plugin details"
echo " node openclaw.mjs plugins doctor # Check for issues"
echo " node /app/mock-worker.js & # Start mock worker"
echo " node openclaw.mjs gateway --allow-unconfigured --verbose # Start gateway"
echo " /bin/bash /app/e2e-verify.sh # Run automated verification"
echo ""
docker run --rm -it "$IMAGE_NAME" /bin/bash
else
docker run --rm "$IMAGE_NAME"
fi