eliott/worldmonitor
1
0
Fork 0
mirror of https://github.com/koala73/worldmonitor.git synced 2026-04-25 17:14:57 +02:00
Code Issues Packages Projects Releases Wiki Activity
3,613 Commits 138 Branches 48 Tags
main
Commit Graph

2 Commits

Author SHA1 Message Date
Elie Habib
21d33c4bb5 fix(hyperliquid-flow): fetch both default dex and xyz builder dex (#3077) 
Root cause: Hyperliquid's commodity and FX perps (xyz:CL, xyz:BRENTOIL,
xyz:GOLD, xyz:SILVER, xyz:PLATINUM, xyz:PALLADIUM, xyz:COPPER, xyz:NATGAS,
xyz:EUR, xyz:JPY) live on a separate 'xyz' builder dex, NOT the default
perp dex. The MIT reference repo listed these with xyz: prefixes but
didn't document that they require {type:metaAndAssetCtxs, dex:xyz} as a
separate POST.

Production symptom (Railway bundle logs 2026-04-14 04:10):
  [Hyperliquid-Flow] SKIPPED: validation failed (empty data)

The seeder polled the default dex only, matched 4 of 14 whitelisted assets
(BTC/ETH/SOL/PAXG), and validateFn rejected snapshots with <12 assets.
Seed-meta was refreshed on the skipped path so health stayed OK but
market:hyperliquid:flow:v1 was never written.

Fix:
- New fetchAllMetaAndCtxs(): parallel-fetches both dexes and merges
  {universe, assetCtxs} by concatenation. xyz entries already carry the
  xyz: prefix in their universe names.
- New validateDexPayload(raw, dexLabel, minUniverse): per-dex floor so the
  thinner xyz dex (~63 entries) does not false-trip the default floor of
  50. Errors include the dex label for debuggability.
- validateUpstream(): back-compat wrapper — accepts either the legacy
  single-dex [meta, assetCtxs] tuple (buildSnapshot tests) or the merged
  {universe, assetCtxs} shape from fetchAllMetaAndCtxs.

Tests: 37/37 green. New tests cover dual-dex fetch merge, cross-dex error
propagation, xyz floor accept/reject, and merged-shape pass-through.
 2026-04-14 08:28:57 +04:00
Elie Habib
e32d9b631c feat(market): Hyperliquid perp positioning flow as leading indicator (#3074) 
* feat(market): Hyperliquid perp positioning flow as leading indicator

Adds a 4-component composite (funding × volume × OI × basis) "positioning
stress" score for ~14 perps spanning crypto (BTC/ETH/SOL), tokenized gold
(PAXG), commodity perps (WTI, Brent, Gold, Silver, Pt, Pd, Cu, NatGas), and
FX perps (EUR, JPY). Polls Hyperliquid /info every 5min via Railway cron;
publishes a single self-contained snapshot with embedded sparkline arrays
(60 samples = 5h history). Surfaces as a new "Perp Flow" tab in
CommoditiesPanel with separate Commodities / FX sections.

Why: existing CFTC COT is weekly + US-centric; market quotes are price-only.
Hyperliquid xyz: perps give 24/7 global positioning data that has been shown
to lead spot moves on commodities and FX by minutes-to-hours.

Implementation:
- scripts/seed-hyperliquid-flow.mjs — pure scoring math, symbol whitelist,
  content-type + schema validation, prior-state read via readSeedSnapshot(),
  warmup contract (first run / post-outage zeroes vol/OI deltas),
  missing-symbol carry-forward, $500k/24h min-notional guard to suppress
  thin xyz: noise. TTL 2700s (9× cadence).
- proto/worldmonitor/market/v1/get_hyperliquid_flow.proto + service.proto
  registration; make generate regenerated client/server bindings.
- server/worldmonitor/market/v1/get-hyperliquid-flow.ts — getCachedJson
  reader matching get-cot-positioning.ts seeded-handler pattern.
- server/gateway.ts cache-tier entry (medium).
- api/health.js: hyperliquidFlow registered with maxStaleMin:15 (3× cadence)
  + transitional ON_DEMAND_KEYS gate for the first ~7 days of bake-in.
- api/seed-health.js mirror with intervalMin:5.
- scripts/seed-bundle-market-backup.mjs entry (NIXPACKS auto-redeploy on
  scripts/** watch).
- src/components/MarketPanel.ts: CommoditiesPanel grows a Perp Flow tab
  + fetchHyperliquidFlow() RPC method; OI Δ1h derived from sparkOi tail.
- src/App.ts: prime via primeVisiblePanelData() + recurring refresh via
  refreshScheduler.scheduleRefresh() at 5min cadence (panel does NOT own
  setInterval; matches the App.ts:1251 lifecycle convention).
- 28 unit tests covering scoring parity, warmup flag, min-notional guard,
  schema rejection, missing-symbol carry-forward, post-outage cold start,
  sparkline cap, alert threshold.

Tests: test:data 5169/5169, hyperliquid-flow-seed 28/28, route-cache-tier
5/5, typecheck + typecheck:api green. One pre-existing test:sidecar failure
(cloud-fallback origin headers) is unrelated and reproduces on origin/main.

* fix(hyperliquid-flow): address review feedback — volume baseline window, warmup lifecycle, error logging

Two real correctness bugs and four review nits from PR #3074 review pass.

Correctness fixes:

1. Volume baseline was anchored to the OLDEST 12 samples, not the newest.
   sparkVol is newest-at-tail (shiftAndAppend), so slice(0, 12) pinned the
   rolling mean to the first hour of data forever once len >= 12. Volume
   scoring would drift further from current conditions each poll. Switched
   to slice(-VOLUME_BASELINE_MIN_SAMPLES) so the baseline tracks the most
   recent window. Regression test added.

2. Warmup flag flipped to false on the second successful poll while volume
   scoring still needed 12+ samples to activate. UI told users warmup
   lasted ~1h but the badge disappeared after 5 min. Tied per-asset warmup
   to real baseline readiness (coldStart OR vol samples < 12 OR prior OI
   missing). Snapshot-level warmup = any asset still warming. Three new
   tests cover the persist-through-baseline-build, clear-once-ready, and
   missing-OI paths.

Review nits:

- Handler: bare catch swallowed Redis/parse errors; now logs err.message.
- Panel: bare catch in fetchHyperliquidFlow hid RPC 500s; now logs.
- MarketPanel.ts: deleted hand-rolled RawHyperliquidAsset; mapHyperliquidFlowResponse
  now takes GetHyperliquidFlowResponse from the generated client so proto
  drift fails compilation instead of silently.
- Seeder: added @ts-check + JSDoc on computeAsset per type-safety rule.
- validateUpstream: MAX_UPSTREAM_UNIVERSE=2000 cap bounds memory.
- buildSnapshot: logs unknown xyz: perps upstream (once per run) so ops
  sees when Hyperliquid adds markets we could whitelist.

Tests: 37/37 green; typecheck + typecheck:api clean.

* fix(hyperliquid-flow): wire bootstrap hydration per AGENTS.md mandate

Greptile review caught that AGENTS.md:187 mandates new data sources be wired
into bootstrap hydration. Plan had deferred this on "lazy deep-dive signal"
grounds, but the project convention is binding.

- server/_shared/cache-keys.ts: add hyperliquidFlow to BOOTSTRAP_CACHE_KEYS
  + BOOTSTRAP_TIERS ('slow' — non-blocking, page-load-parallel).
- api/bootstrap.js: add to inlined BOOTSTRAP_CACHE_KEYS + SLOW_KEYS so
  bootstrap.test.mjs canonical-mirror assertions pass.
- src/components/MarketPanel.ts:
  - Import getHydratedData from @/services/bootstrap.
  - New mapHyperliquidFlowSeed() normalizes the raw seed-JSON shape
    (numeric fields) into HyperliquidFlowView. The RPC mapper handles the
    proto shape (string-encoded numbers); bootstrap emits the raw blob.
  - fetchHyperliquidFlow now reads hydrated data first, renders
    immediately, then refreshes from RPC — mirrors FearGreedPanel pattern.

Tests: 72/72 green (bootstrap + cache-tier + hyperliquid-flow-seed).
 2026-04-14 08:05:40 +04:00