Synchronizing MapLibre Tile Events with GPU Buffer Uploads

The overlay is aligned and its data lags. A pan brings new tiles into view, the host renders them, and the overlay’s geometry for those tiles appears a frame or two later — or, worse, appears in a burst that drops a frame. Both come from the same mistake: doing GPU work in the event handler that fired, rather than marking state dirty and draining it inside the frame the host controls. This page is that pattern — which events matter, what the invalidation set holds, and how many uploads a frame should take — plus the cache interaction that decides how much of this work happens at all. It is one stage of MapLibre and Mapbox WebGPU interop.

Marking dirty against uploading in the handler Two lanes compare two arrangements. In the handler arrangement a tile finishes loading, the event handler immediately calls writeBuffer, and whether that upload is visible to the frame about to be recorded depends on network timing — so the tile appears late on a slow connection and on time on a fast one. In the drain arrangement the handler only adds a key to a dirty set, the custom layer render callback drains a bounded number of keys inside the host frame, and every upload is visible to exactly the frame that drained it. UPLOAD IN THE HANDLER vs DRAIN IN THE FRAME In the handler Drained Tile loads sourcedata fires writeBuffer now between frames Visible… sometimes depends on timing Tile loads mark the key dirty render() drains bounded per frame Visible this frame deterministic The dirty set also gives the natural place to bound how much work one frame takes.
The lower lane is deterministic because it puts every GPU operation at a point the host defines. The upper one is correct on a fast connection and intermittently wrong on a slow one, which is the worst kind of correct.

Runnable reference implementation

Events mark tiles dirty. The render callback drains the set. Nothing touches the GPU outside render.

typescript
class OverlayData {
  private readonly dirty = new Set<string>();
  private readonly resident = new Map<string, GPUBuffer>();
  private readonly UPLOADS_PER_FRAME = 4;

  attach(map: maplibregl.Map): void {
    // A tile finished loading in the host. Mark it; do not touch the GPU.
    map.on("sourcedata", (e) => {
      if (e.sourceId !== "basemap" || !e.tile) return;
      if (e.isSourceLoaded) this.dirty.add(keyOf(e.tile.tileID));
    });
    // The camera settled: recompute which tiles we want at all.
    map.on("moveend", () => this.reconcile(map));
  }

  /** Called from the custom layer's render(), inside the host frame. */
  drain(device: GPUDevice): void {
    let budget = this.UPLOADS_PER_FRAME;
    for (const key of this.dirty) {
      if (budget-- <= 0) break;              // the rest wait for the next frame
      this.dirty.delete(key);
      const bytes = this.decoded.get(key);
      if (!bytes) continue;                  // still fetching; it will be re-marked
      const buffer = this.resident.get(key) ?? this.allocate(device, key, bytes.byteLength);
      device.queue.writeBuffer(buffer, 0, bytes);
    }
  }
}

The four-uploads-per-frame budget is the load-bearing detail. A pan that crosses several tile boundaries can mark a dozen tiles dirty in one frame, and uploading all of them costs more than the frame has — while uploading four spreads the cost over three frames during which the missing tiles simply render at the coarser zoom below them, which nobody notices.

Parameter reference

Value Setting here Guidance
Uploads per frame 4 At ~1 MiB per tile that is a few milliseconds. Raise it if tiles are small, lower it if the host frame is already tight.
Events listened to sourcedata, moveend sourcedata fires often; filter on isSourceLoaded or the handler runs dozens of times per tile.
Drain location inside render The only place where the host’s encoder and the frame’s timing are both known.
Dirty set keys, not payloads Holding decoded bytes in the set couples invalidation to memory; keep them in a separate cache.
Reconcile trigger moveend, not move move fires continuously during a drag; reconciling per event does the same work dozens of times.
Frame cost against uploads drained per frame A bar chart in milliseconds of the added frame cost for different per-frame upload budgets, at roughly one mebibyte per tile. Draining two tiles adds about 1.4 milliseconds. Four adds about 2.7. Eight adds about 5.4. Draining everything, which during a fast pan can be a dozen tiles, adds about 8.1 milliseconds on top of whatever the host frame already costs, which is enough to overrun the budget on a busy basemap. ADDED FRAME COST BY DRAIN BUDGET · ms 2 per frame 1.4 ms 4 per frame 2.7 ms — default 8 per frame 5.4 ms Unbounded 8.1 ms in a burst 0 3 6 9 ms comfortable watch the host budget hitches on a pan Tiles not drained this frame render at the coarser zoom below them, which is invisible.
The overlay shares the host frame, so this cost lands on top of whatever the basemap already spends. Four is a default rather than a rule — a busy vector basemap may only afford two.

Why the event handler is the wrong place

Doing the upload where the event fires looks simpler and produces three problems, each of which is intermittent enough to be hard to attribute.

The first is timing. Queue operations issued between the host’s submit and its next frame are ordered against submissions but are not part of one, so whether a given upload is visible to the frame that is about to be recorded depends on when the network happened to deliver the tile. That produces a tile that appears one frame late on a slow connection and on time on a fast one — a bug that never reproduces on the developer’s machine.

The second is burstiness. Several tiles resolving in the same tick means several uploads in the same tick, with no opportunity to spread them. A drain point gives one, for free.

The third is ordering against the host. The overlay’s data for a tile should become visible in the same frame the host’s own version of that tile does, and only the host knows when that is. Draining inside render gets that alignment automatically; an event handler is guessing.

Reconciling, and what “wanted” means

The dirty set says what has changed; the reconcile says what the overlay wants at all, and the two are different questions with different triggers.

The wanted set is a function of the camera: the tiles covering the viewport at the current zoom, plus a ring of margin so a small pan does not expose an empty area. Recomputing it is cheap — a few dozen key computations — but doing it on every move event during a drag does it dozens of times a second for an answer that only changes when the viewport crosses a tile boundary. moveend is the right trigger for the full reconcile, with an optional cheap check on move that only recomputes when the tile bounds actually change.

Reconciling has two outputs. Tiles in the wanted set that are not resident become fetch requests, and tiles resident but no longer wanted become eviction candidates — passed to the LRU cache rather than freed immediately, because a pan that reverses will want them again within a second.

The subtlety is that “wanted” should be computed against the host’s tile set rather than independently. The host has already decided which zoom level to show, including the hysteresis it applies during a zoom gesture so the level does not flap, and an overlay that computes its own level from map.getZoom() will disagree during exactly that gesture — showing overlay geometry at one zoom over a basemap at another for as long as the transition lasts.

Failure modes

  • Overlay geometry appears a frame after the basemap tile. The upload happened in the event handler, after the frame that would have shown it was already recorded.
  • A hitch every time the user pans quickly. No per-frame upload budget, so a dozen tiles uploaded in one frame.
  • sourcedata fires hundreds of times. It is emitted for many kinds of source state change, not only tile completion. Filter on isSourceLoaded and on the source id.
  • The overlay keeps buffers for tiles the map has evicted. Only sourcedata was handled and nothing listens for removal. Reconcile against the host’s current tile set on moveend rather than relying on a removal event.
  • Overlay and basemap show different zoom levels during a pinch. The overlay computed its level from map.getZoom() rather than following the host’s tile set, so it missed the hysteresis the host applies during the gesture.
  • The map stops updating the overlay while idle. Correct behaviour: the host only redraws when it thinks something changed. Call triggerRepaint when the drain leaves work outstanding.
Which host events are worth listening to A table of four MapLibre events with when each fires and what the overlay should do. The sourcedata event fires on many kinds of source state change including tile completion, and the overlay should filter it and mark keys dirty. The moveend event fires when the camera settles, and the overlay should reconcile its wanted tile set against the host. The move event fires continuously during a drag and the overlay should ignore it. The idle event fires when the host has finished all pending work, and the overlay can use it to do optional prefetching. EVENT · WHEN · WHAT TO DO Fires Overlay response sourcedata often, many causes filter, mark dirty moveend camera settles reconcile the tile set move continuously ignore it idle all work done optional prefetch None of these should touch the GPU — every one of them only updates plain state.
The third row is where most of the wasted work goes. Reconciling on every `move` event during a drag does the same set computation dozens of times a second for an answer that only matters once.

Backend / Python interop note

The overlay usually fetches its own data rather than reading the host’s parsed tiles, which means the same tile key is requested twice — once by the host for the basemap, once by the overlay for its geometry. Two properties on the server side make that cheap.

The first is aligning the tile schemes. If the overlay’s endpoint uses the same (z, x, y) scheme as the basemap, the overlay’s invalidation set can be exactly the host’s tile set, with no remapping and no partial coverage. An endpoint on a different grid forces the overlay to compute which of its tiles a host tile overlaps, which is both extra work and a source of gaps at boundaries.

The second is cache friendliness, for the same reasons as everywhere else: immutable URLs with a version in the path, a long max-age, and an ETag. A pan that returns to a previously visited area then costs no network at all, and the drain becomes a decode and an upload rather than a round trip.

There is a third consideration that decides how much of this machinery is needed at all: whether the overlay’s data is tiled in the first place. A layer of a few thousand features — a set of monitoring stations, a delivery network, a study area — is not tiled and does not need any of this. It is fetched once, uploaded once, and drawn every frame with a viewport cull. The invalidation machinery on this page exists for datasets too large to hold at once, and applying it to a small layer adds complexity that buys nothing.

Where the overlay’s data is derived rather than raw — an aggregate, a filtered subset — the derivation belongs on the server for the usual reason: it is a vectorised column operation there and a per-record loop in the browser. Shipping it through the same Arrow transport as everything else keeps the decode out of the frame entirely.