mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
f4183f99c7
* feat: self-hosted Docker stack with nginx, Redis REST proxy, and seeders
Multi-stage Docker build: esbuild TS handler compilation, vite frontend
build, nginx + Node.js API under supervisord. Upstash-compatible Redis
REST proxy with command allowlist for security. AIS relay WebSocket
sidecar. Seeder wrapper script with auto-sourced env vars from
docker-compose.override.yml. Self-hosting guide with architecture
diagram, API key setup, and troubleshooting.
Security: Redis proxy command allowlist (blocks FLUSHALL/CONFIG/EVAL),
nginx security headers (X-Content-Type-Options, X-Frame-Options,
Referrer-Policy), non-root container user.
* feat(docker): add Docker secrets support for API keys
Entrypoint reads /run/secrets/* files and exports as env vars at
startup. Secrets take priority over environment block values and
stay out of docker inspect / process metadata.
Both methods (env vars and secrets) work simultaneously.
* fix(docker): point supervisord at templated nginx config
The entrypoint runs envsubst on nginx.conf.template and writes
the result to /tmp/nginx.conf (with LOCAL_API_PORT substituted
and listening on port 8080 for non-root). But supervisord was
still launching nginx with /etc/nginx/nginx.conf — the default
Alpine config that listens on port 80, which fails with
"Permission denied" under the non-root appuser.
* fix(docker): remove KEYS from Redis allowlist, fix nginx header inheritance, add LLM vars to seeders
- Remove KEYS from redis-rest-proxy allowlist (O(N) blocking, Redis DoS risk)
- Move security headers into each nginx location block to prevent add_header
inheritance suppression
- Add LLM_API_URL / LLM_API_KEY / LLM_MODEL to run-seeders.sh grep filter
so LLM API keys set in docker-compose.override.yml are forwarded to seed scripts
* fix(docker): add path-based POST to Redis proxy, expand allowlist, add missing seeder secrets
- Add POST /{command}/{args...} handler to redis-rest-proxy so Upstash-style
path POSTs work (setCachedJson uses POST /set/<key>/<value>/EX/<ttl>)
- Expand allowlist: HLEN, LTRIM (seed-military-bases, seed-forecasts),
ZREVRANGE (premium-stock-store), ZRANDMEMBER (seed-military-bases)
- Add ACLED_EMAIL, ACLED_PASSWORD, OPENROUTER_API_KEY, OLLAMA_API_URL,
OLLAMA_MODEL to run-seeders.sh so override keys reach host-run seeders
---------
Co-authored-by: Elie Habib <elie.habib@gmail.com>
53 lines
1.7 KiB
Bash
Executable File
53 lines
1.7 KiB
Bash
Executable File
#!/bin/sh
|
|
# Run all seed scripts against the local Redis REST proxy.
|
|
# Usage: ./scripts/run-seeders.sh
|
|
#
|
|
# Requires the worldmonitor stack to be running (uvx podman-compose up -d).
|
|
# The Redis REST proxy listens on localhost:8079 by default.
|
|
|
|
UPSTASH_REDIS_REST_URL="${UPSTASH_REDIS_REST_URL:-http://localhost:8079}"
|
|
UPSTASH_REDIS_REST_TOKEN="${UPSTASH_REDIS_REST_TOKEN:-wm-local-token}"
|
|
export UPSTASH_REDIS_REST_URL UPSTASH_REDIS_REST_TOKEN
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
|
|
|
# Source API keys from docker-compose.override.yml if present.
|
|
# These keys are configured for the container but seeders run on the host.
|
|
OVERRIDE="$PROJECT_DIR/docker-compose.override.yml"
|
|
if [ -f "$OVERRIDE" ]; then
|
|
_env_tmp=$(mktemp)
|
|
grep -E '^\s+[A-Z_]+:' "$OVERRIDE" \
|
|
| grep -v '#' \
|
|
| sed 's/^\s*//' \
|
|
| sed 's/: */=/' \
|
|
| sed "s/[\"']//g" \
|
|
| grep -E '^(NASA_FIRMS|GROQ|AISSTREAM|FRED|FINNHUB|EIA|ACLED_ACCESS_TOKEN|ACLED_EMAIL|ACLED_PASSWORD|CLOUDFLARE|AVIATIONSTACK|OPENROUTER_API_KEY|LLM_API_URL|LLM_API_KEY|LLM_MODEL|OLLAMA_API_URL|OLLAMA_MODEL)' \
|
|
| sed 's/^/export /' > "$_env_tmp"
|
|
. "$_env_tmp"
|
|
rm -f "$_env_tmp"
|
|
fi
|
|
ok=0 fail=0 skip=0
|
|
|
|
for f in "$SCRIPT_DIR"/seed-*.mjs; do
|
|
name="$(basename "$f")"
|
|
printf "→ %s ... " "$name"
|
|
output=$(node "$f" 2>&1)
|
|
rc=$?
|
|
last=$(echo "$output" | tail -1)
|
|
|
|
if echo "$last" | grep -qi "skip\|not set\|missing.*key\|not found"; then
|
|
printf "SKIP (%s)\n" "$last"
|
|
skip=$((skip + 1))
|
|
elif [ $rc -eq 0 ]; then
|
|
printf "OK\n"
|
|
ok=$((ok + 1))
|
|
else
|
|
printf "FAIL (%s)\n" "$last"
|
|
fail=$((fail + 1))
|
|
fi
|
|
done
|
|
|
|
echo ""
|
|
echo "Done: $ok ok, $skip skipped, $fail failed"
|