eliott/worldmonitor
1
0
Fork 0
mirror of https://github.com/koala73/worldmonitor.git synced 2026-04-25 17:14:57 +02:00
Code Issues Packages Projects Releases Wiki Activity

perf(map): lazy supercluster init, memoize filterByTime, lazy static layers (#1985)

Browse Source 
Avoid wasted startup work for non-tech variants and reduce per-render
cost in buildLayers():

- Remove eager rebuildTechHQSupercluster/rebuildDatacenterSupercluster
  from map load and basemap-switch handlers; build lazily in
  updateClusterData() only when the exact conditions are met
- Add WeakMap-backed filterByTimeCached() with 1-minute time bucket;
  replaces 10 inline filterByTime() calls in buildLayers()
- Rewrite GlobeMap.initStaticLayers() to delegate to
  ensureStaticDataForLayer(), skipping datasets for disabled layers;
  hook into setLayers() and enableLayer() for lazy init on first enable
This commit is contained in:
Elie Habib
2026-03-21 15:32:51 +04:00
committed by GitHub
parent 99a7793e99
commit 61a2b5f286
3 changed files with 347 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files

149
plans/map-performance-improvements.md Normal file
View File

@@ -0,0 +1,149 @@
# Plan: Map Performance Improvements (Revised v2)
## Context
worldmonitor.app renders a real-time geopolitical map using three engines:
- **DeckGLMap** (WebGL, deck.gl over MapLibre) — primary desktop map
- **GlobeMap** (globe.gl / Three.js) — 3D globe mode
- **Map.ts** (Leaflet + D3 SVG) — mobile fallback
Default web layout enables 12 map layers. Performance profiling (built-in console.warn at >16ms) revealed `buildLayers()` is a hot path.
## Diagnosis
Three real bottlenecks identified:
1. **Supercluster indexes built unconditionally at startup** — `rebuildTechHQSupercluster()` and `rebuildDatacenterSupercluster()` run on map `'load'` and basemap switch for ALL users, even when `SITE_VARIANT !== 'tech'` and datacenters are off.
2. **`filterByTime()` re-runs 10× on every `buildLayers()` call** — called on every data update, zoom change, and layer toggle. `filterByTime()` uses `Date.now()` as its cutoff, so results expire over time — but rebuilding them on every render is wasteful when data hasn't changed within the current minute.
3. **`initStaticLayers()` in GlobeMap processes all 9 static datasets unconditionally** — MILITARY_BASES, NUCLEAR_FACILITIES, GAMMA_IRRADIATORS, SPACEPORTS, ECONOMIC markers, AI_DATA_CENTERS, WATERWAYS, MINERALS, UNDERSEA_CABLES/PIPELINES all processed at startup for every variant.
### Ruled out (no issue):
- `rafSchedule` is NOT a continuous loop — fires only when called, already correct
- `flushMarkersImmediate()` already gates by `this.layers.xxx` — no redundant pushes
- GlobeMap debounce (100ms + 300ms max) already coalesces rapid updates
---
## Change 1: Lazy Supercluster Initialization
**File**: `src/components/DeckGLMap.ts`
**Remove** the unconditional eager builds at `'load'` and basemap-switch:
```typescript
// REMOVE these two lines from 'load' handler (and basemap-switch handler):
this.rebuildTechHQSupercluster();
this.rebuildDatacenterSupercluster();
```
**Add** lazy-init inside `updateClusterData()`, before the existing cluster usage. `updateClusterData()` already computes the exact conditions needed:
```typescript
// In updateClusterData(), after computing useTechHQ / useDatacenterClusters:
const useTechHQ = SITE_VARIANT === 'tech' && layers.techHQs;
const useDatacenterClusters = layers.datacenters && zoom < 5;
if (useTechHQ && !this.techHQSC) this.rebuildTechHQSupercluster();
if (useDatacenterClusters && !this.datacenterSC) this.rebuildDatacenterSupercluster();
```
First time these layers are active at the right zoom, the cluster is built once and cached.
**Risk**: Very low. `updateClusterData()` already has early-return guards and is called at every render. The lazy-init path is a one-time cost, identical to the current eager cost — just deferred.
---
## Change 2: Memoized `filterByTime` Helper
**File**: `src/components/DeckGLMap.ts`
Add a single memoized wrapper using `WeakMap` (avoids strong-ref memory leak on old array replacements):
```typescript
private _timeFilterCache = new WeakMap<object, { min: number; range: TimeRange; result: unknown[] }>();
private filterByTimeCached<T>(items: T[], key: (t: T) => Date | string | number): T[] {
const min = Math.floor(Date.now() / 60000); // 1-minute bucket
const range = this.state.timeRange;
const cached = this._timeFilterCache.get(items as object);
if (cached && cached.min === min && cached.range === range) return cached.result as T[];
const result = this.filterByTime(items, key);
this._timeFilterCache.set(items as object, { min, range, result });
return result;
}
```
Cache invalidation:
- **New data**: `set*()` methods assign a new array reference → WeakMap miss, old entry GC'd
- **Time range change**: `range` differs → cache miss
- **Clock advance**: `min` bucket (per-minute) expires naturally → recompute picks up newly-expired events
In `buildLayers()`, replace all 10 inline calls (change `filterByTime` → `filterByTimeCached`). No setter or state changes needed.
**Risk**: Low. WeakMap eliminates memory leak. 1-minute bucket means at most 60s of stale filtering — acceptable given AIS data refreshes every 20s and triggers a new array ref.
---
## Change 3: Guard `initStaticLayers()` in GlobeMap
**File**: `src/components/GlobeMap.ts`
Wrap all 9 datasets in `initStaticLayers()` with their layer-state guards. Add `ensureStaticDataForLayer(layer)` called from both `setLayers()` (newly-enabled keys) and `enableLayer()` (programmatic enables — URL restore, search, panel actions):
```typescript
private ensureStaticDataForLayer(layer: keyof MapLayers): void {
switch (layer) {
case 'bases': if (!this.milBaseMarkers.length) this.milBaseMarkers = MILITARY_BASES.map(...); break;
case 'nuclear': if (!this.nuclearSiteMarkers.length) this.nuclearSiteMarkers = NUCLEAR_FACILITIES.filter(...).map(...); break;
case 'irradiators': if (!this.irradiatorSiteMarkers.length) this.irradiatorSiteMarkers = GAMMA_IRRADIATORS.map(...); break;
case 'spaceports': if (!this.spaceportSiteMarkers.length) this.spaceportSiteMarkers = SPACEPORTS.filter(...).map(...); break;
case 'economic': if (!this.economicMarkers.length) this.economicMarkers = ECONOMIC_CENTERS.map(...); break;
case 'datacenters': if (!this.datacenterMarkers.length) this.datacenterMarkers = AI_DATA_CENTERS.filter(...).map(...); break;
case 'waterways': if (!this.waterwayMarkers.length) this.waterwayMarkers = STRATEGIC_WATERWAYS.map(...); break;
case 'minerals': if (!this.mineralMarkers.length) this.mineralMarkers = CRITICAL_MINERALS.filter(...).map(...); break;
case 'tradeRoutes': if (!this.tradeRouteSegments.length) this.tradeRouteSegments = resolveTradeRouteSegments(); break;
case 'cables':
case 'pipelines': if (!this.globePaths.length) this.globePaths = [...CABLES.map(...), ...PIPELINES.map(...)]; break;
}
}
```
Hook into `setLayers()`:
```typescript
for (const k of Object.keys(layers) as (keyof MapLayers)[]) {
if (!prev[k] && layers[k]) this.ensureStaticDataForLayer(k); // newly enabled
// existing channel flush logic continues...
}
```
Hook into `enableLayer()`:
```typescript
public enableLayer(layer: keyof MapLayers): void {
if (layer === 'dayNight') return;
if (this.layers[layer]) return;
(this.layers as any)[layer] = true;
this.ensureStaticDataForLayer(layer); // lazy init for programmatic enables
this.flushLayerChannels(layer);
this.enforceLayerLimit();
}
```
**Risk**: Low. Static datasets never change at runtime. For variants where layers are on by default, `initStaticLayers()` runs at startup as before.
---
## Files Changed
| File | Change |
|---|---|
| `src/components/DeckGLMap.ts` | Changes 1 + 2 (~30 lines net) |
| `src/components/GlobeMap.ts` | Change 3 (~80 lines net) |
## Expected Improvements
- **Startup**: Superclusters not built until first needed; GlobeMap static data skipped for non-default layers (~40-200ms per variant)
- **Per-render**: `buildLayers()` hits WeakMap cache instead of re-filtering; at most 1 recompute per minute per active layer

62
src/components/DeckGLMap.ts
View File

@@ -490,8 +490,6 @@ export class DeckGLMap {
this.maplibreMap?.on('load', () => {
localizeMapLabels(this.maplibreMap);
this.rebuildTechHQSupercluster();
this.rebuildDatacenterSupercluster();
this.initDeck();
this.loadCountryBoundaries();
this.fetchServerBases();
@@ -687,8 +685,6 @@ export class DeckGLMap {
});
this.maplibreMap.on('load', () => {
localizeMapLabels(this.maplibreMap);
this.rebuildTechHQSupercluster();
this.rebuildDatacenterSupercluster();
this.initDeck();
this.loadCountryBoundaries();
this.fetchServerBases();
@@ -882,6 +878,41 @@ export class DeckGLMap {
});
}
private _timeFilterCache = new WeakMap<object, { min: number; range: TimeRange; result: unknown[] }>();
private filterByTimeCached<T>(
items: T[],
getTime: (item: T) => Date | string | number | undefined | null
): T[] {
const min = Math.floor(Date.now() / 60000);
const range = this.state.timeRange;
const cached = this._timeFilterCache.get(items as object);
if (cached && cached.min === min && cached.range === range) return cached.result as T[];
const result = this.filterByTime(items, getTime);
this._timeFilterCache.set(items as object, { min, range, result });
return result;
}
private filterMilitaryFlightClustersByTimeCached(clusters: MilitaryFlightCluster[]): MilitaryFlightCluster[] {
const min = Math.floor(Date.now() / 60000);
const range = this.state.timeRange;
const cached = this._timeFilterCache.get(clusters as object);
if (cached && cached.min === min && cached.range === range) return cached.result as MilitaryFlightCluster[];
const result = this.filterMilitaryFlightClustersByTime(clusters);
this._timeFilterCache.set(clusters as object, { min, range, result });
return result;
}
private filterMilitaryVesselClustersByTimeCached(clusters: MilitaryVesselCluster[]): MilitaryVesselCluster[] {
const min = Math.floor(Date.now() / 60000);
const range = this.state.timeRange;
const cached = this._timeFilterCache.get(clusters as object);
if (cached && cached.min === min && cached.range === range) return cached.result as MilitaryVesselCluster[];
const result = this.filterMilitaryVesselClustersByTime(clusters);
this._timeFilterCache.set(clusters as object, { min, range, result });
return result;
}
private getFilteredProtests(): SocialUnrestEvent[] {
return this.filterByTime(this.protests, (event) => event.time);
}
@@ -1087,6 +1118,9 @@ export class DeckGLMap {
this.lastSCBoundsKey = boundsKey;
this.lastSCMask = layerMask;
if (useTechHQ && !this.techHQSC) this.rebuildTechHQSupercluster();
if (useDatacenterClusters && !this.datacenterSC) this.rebuildDatacenterSupercluster();
if (useProtests && this.protestSC) {
this.protestClusters = this.protestSC.getClusters(bbox, zoom).map(f => {
const coords = f.geometry.coordinates as [number, number];
@@ -1274,16 +1308,16 @@ export class DeckGLMap {
COLORS = getOverlayColors();
const layers: (Layer | null | false)[] = [];
const { layers: mapLayers } = this.state;
const filteredEarthquakes = mapLayers.natural ? this.filterByTime(this.earthquakes, (eq) => eq.occurredAt) : [];
const filteredNaturalEvents = mapLayers.natural ? this.filterByTime(this.naturalEvents, (event) => event.date) : [];
const filteredWeatherAlerts = mapLayers.weather ? this.filterByTime(this.weatherAlerts, (alert) => alert.onset) : [];
const filteredOutages = mapLayers.outages ? this.filterByTime(this.outages, (outage) => outage.pubDate) : [];
const filteredCableAdvisories = mapLayers.cables ? this.filterByTime(this.cableAdvisories, (advisory) => advisory.reported) : [];
const filteredFlightDelays = mapLayers.flights ? this.filterByTime(this.flightDelays, (delay) => delay.updatedAt) : [];
const filteredMilitaryFlights = mapLayers.military ? this.filterByTime(this.militaryFlights, (flight) => flight.lastSeen) : [];
const filteredMilitaryVessels = mapLayers.military ? this.filterByTime(this.militaryVessels, (vessel) => vessel.lastAisUpdate) : [];
const filteredMilitaryFlightClusters = mapLayers.military ? this.filterMilitaryFlightClustersByTime(this.militaryFlightClusters) : [];
const filteredMilitaryVesselClusters = mapLayers.military ? this.filterMilitaryVesselClustersByTime(this.militaryVesselClusters) : [];
const filteredEarthquakes = mapLayers.natural ? this.filterByTimeCached(this.earthquakes, (eq) => eq.occurredAt) : [];
const filteredNaturalEvents = mapLayers.natural ? this.filterByTimeCached(this.naturalEvents, (event) => event.date) : [];
const filteredWeatherAlerts = mapLayers.weather ? this.filterByTimeCached(this.weatherAlerts, (alert) => alert.onset) : [];
const filteredOutages = mapLayers.outages ? this.filterByTimeCached(this.outages, (outage) => outage.pubDate) : [];
const filteredCableAdvisories = mapLayers.cables ? this.filterByTimeCached(this.cableAdvisories, (advisory) => advisory.reported) : [];
const filteredFlightDelays = mapLayers.flights ? this.filterByTimeCached(this.flightDelays, (delay) => delay.updatedAt) : [];
const filteredMilitaryFlights = mapLayers.military ? this.filterByTimeCached(this.militaryFlights, (flight) => flight.lastSeen) : [];
const filteredMilitaryVessels = mapLayers.military ? this.filterByTimeCached(this.militaryVessels, (vessel) => vessel.lastAisUpdate) : [];
const filteredMilitaryFlightClusters = mapLayers.military ? this.filterMilitaryFlightClustersByTimeCached(this.militaryFlightClusters) : [];
const filteredMilitaryVesselClusters = mapLayers.military ? this.filterMilitaryVesselClustersByTimeCached(this.militaryVesselClusters) : [];
// UCDP is a historical dataset (events aged months); time-range filter always zeroes it out
const filteredUcdpEvents = mapLayers.ucdpEvents ? this.ucdpEvents : [];

249
src/components/GlobeMap.ts
View File

@@ -2093,105 +2093,154 @@ export class GlobeMap {
}
private initStaticLayers(): void {
this.milBaseMarkers = (MILITARY_BASES as MilitaryBase[]).map(b => ({
_kind: 'milbase' as const,
_lat: b.lat,
_lng: b.lon,
id: b.id,
name: b.name,
type: b.type,
country: b.country ?? '',
}));
this.nuclearSiteMarkers = NUCLEAR_FACILITIES
.filter(f => f.status !== 'decommissioned')
.map(f => ({
_kind: 'nuclearSite' as const,
_lat: f.lat,
_lng: f.lon,
id: f.id,
name: f.name,
type: f.type,
status: f.status,
}));
this.irradiatorSiteMarkers = (GAMMA_IRRADIATORS as GammaIrradiator[]).map(g => ({
_kind: 'irradiator' as const,
_lat: g.lat,
_lng: g.lon,
id: g.id,
city: g.city,
country: g.country,
}));
this.spaceportSiteMarkers = (SPACEPORTS as Spaceport[])
.filter(s => s.status === 'active')
.map(s => ({
_kind: 'spaceport' as const,
_lat: s.lat,
_lng: s.lon,
id: s.id,
name: s.name,
country: s.country,
operator: s.operator,
launches: s.launches,
}));
this.economicMarkers = (ECONOMIC_CENTERS as EconomicCenter[]).map(c => ({
_kind: 'economic' as const,
_lat: c.lat,
_lng: c.lon,
id: c.id,
name: c.name,
type: c.type,
country: c.country,
description: c.description ?? '',
}));
this.datacenterMarkers = (AI_DATA_CENTERS as AIDataCenter[])
.filter(d => d.status !== 'decommissioned')
.map(d => ({
_kind: 'datacenter' as const,
_lat: d.lat,
_lng: d.lon,
id: d.id,
name: d.name,
owner: d.owner,
country: d.country,
chipType: d.chipType,
}));
this.waterwayMarkers = (STRATEGIC_WATERWAYS as StrategicWaterway[]).map(w => ({
_kind: 'waterway' as const,
_lat: w.lat,
_lng: w.lon,
id: w.id,
name: w.name,
description: w.description ?? '',
}));
this.mineralMarkers = (CRITICAL_MINERALS as CriticalMineralProject[])
.filter(m => m.status === 'producing' || m.status === 'development')
.map(m => ({
_kind: 'mineral' as const,
_lat: m.lat,
_lng: m.lon,
id: m.id,
name: m.name,
mineral: m.mineral,
country: m.country,
status: m.status,
}));
this.tradeRouteSegments = resolveTradeRouteSegments();
this.globePaths = [
...(UNDERSEA_CABLES as UnderseaCable[]).map(c => ({
id: c.id,
name: c.name,
points: c.points,
pathType: 'cable' as const,
status: 'ok',
})),
...(PIPELINES as Pipeline[]).map(p => ({
id: p.id,
name: p.name,
points: p.points,
pathType: p.type,
status: p.status,
})),
];
for (const k of Object.keys(this.layers) as (keyof MapLayers)[]) {
if (this.layers[k]) this.ensureStaticDataForLayer(k);
}
}
private ensureStaticDataForLayer(layer: keyof MapLayers): void {
switch (layer) {
case 'bases':
if (!this.milBaseMarkers.length) {
this.milBaseMarkers = (MILITARY_BASES as MilitaryBase[]).map(b => ({
_kind: 'milbase' as const,
_lat: b.lat,
_lng: b.lon,
id: b.id,
name: b.name,
type: b.type,
country: b.country ?? '',
}));
}
break;
case 'nuclear':
if (!this.nuclearSiteMarkers.length) {
this.nuclearSiteMarkers = NUCLEAR_FACILITIES
.filter(f => f.status !== 'decommissioned')
.map(f => ({
_kind: 'nuclearSite' as const,
_lat: f.lat,
_lng: f.lon,
id: f.id,
name: f.name,
type: f.type,
status: f.status,
}));
}
break;
case 'irradiators':
if (!this.irradiatorSiteMarkers.length) {
this.irradiatorSiteMarkers = (GAMMA_IRRADIATORS as GammaIrradiator[]).map(g => ({
_kind: 'irradiator' as const,
_lat: g.lat,
_lng: g.lon,
id: g.id,
city: g.city,
country: g.country,
}));
}
break;
case 'spaceports':
if (!this.spaceportSiteMarkers.length) {
this.spaceportSiteMarkers = (SPACEPORTS as Spaceport[])
.filter(s => s.status === 'active')
.map(s => ({
_kind: 'spaceport' as const,
_lat: s.lat,
_lng: s.lon,
id: s.id,
name: s.name,
country: s.country,
operator: s.operator,
launches: s.launches,
}));
}
break;
case 'economic':
if (!this.economicMarkers.length) {
this.economicMarkers = (ECONOMIC_CENTERS as EconomicCenter[]).map(c => ({
_kind: 'economic' as const,
_lat: c.lat,
_lng: c.lon,
id: c.id,
name: c.name,
type: c.type,
country: c.country,
description: c.description ?? '',
}));
}
break;
case 'datacenters':
if (!this.datacenterMarkers.length) {
this.datacenterMarkers = (AI_DATA_CENTERS as AIDataCenter[])
.filter(d => d.status !== 'decommissioned')
.map(d => ({
_kind: 'datacenter' as const,
_lat: d.lat,
_lng: d.lon,
id: d.id,
name: d.name,
owner: d.owner,
country: d.country,
chipType: d.chipType,
}));
}
break;
case 'waterways':
if (!this.waterwayMarkers.length) {
this.waterwayMarkers = (STRATEGIC_WATERWAYS as StrategicWaterway[]).map(w => ({
_kind: 'waterway' as const,
_lat: w.lat,
_lng: w.lon,
id: w.id,
name: w.name,
description: w.description ?? '',
}));
}
break;
case 'minerals':
if (!this.mineralMarkers.length) {
this.mineralMarkers = (CRITICAL_MINERALS as CriticalMineralProject[])
.filter(m => m.status === 'producing' || m.status === 'development')
.map(m => ({
_kind: 'mineral' as const,
_lat: m.lat,
_lng: m.lon,
id: m.id,
name: m.name,
mineral: m.mineral,
country: m.country,
status: m.status,
}));
}
break;
case 'tradeRoutes':
if (!this.tradeRouteSegments.length) {
this.tradeRouteSegments = resolveTradeRouteSegments();
}
break;
case 'cables':
case 'pipelines':
if (!this.globePaths.length) {
this.globePaths = [
...(UNDERSEA_CABLES as UnderseaCable[]).map(c => ({
id: c.id,
name: c.name,
points: c.points,
pathType: 'cable' as const,
status: 'ok',
})),
...(PIPELINES as Pipeline[]).map(p => ({
id: p.id,
name: p.name,
points: p.points,
pathType: p.type,
status: p.status,
})),
];
}
break;
}
}
public setMilitaryFlights(flights: MilitaryFlight[]): void {
@@ -2402,6 +2451,7 @@ export class GlobeMap {
this.layers = { ...layers, dayNight: false };
let needMarkers = false, needArcs = false, needPaths = false, needPolygons = false;
for (const k of Object.keys(layers) as (keyof MapLayers)[]) {
if (!prev[k] && layers[k]) this.ensureStaticDataForLayer(k);
if (prev[k] === layers[k]) continue;
const ch = GlobeMap.LAYER_CHANNELS.get(k);
if (!ch) { needMarkers = true; continue; }
@@ -2432,6 +2482,7 @@ export class GlobeMap {
if (layer === 'dayNight') return;
if (this.layers[layer]) return;
(this.layers as any)[layer] = true;
this.ensureStaticDataForLayer(layer);
const toggle = this.layerTogglesEl?.querySelector(`.layer-toggle[data-layer="${layer}"] input`) as HTMLInputElement | null;
if (toggle) toggle.checked = true;
this.flushLayerChannels(layer);