Evicting Tile Textures Without Stalling the Queue

The sub-problem is narrow and sharp: a tile cache decides that layer 17 of a texture_2d_array is no longer worth keeping, and the GPU may still be sampling it for a frame that was submitted a moment ago. Calling texture.destroy() there is a use-after-free that the validator rejects; waiting for the GPU to finish before evicting stalls the pipeline. The answer is to stop destroying textures at all and to recycle layers under a submission counter, which makes eviction a bookkeeping operation with no synchronisation in it. This page is that mechanism in full. It is one stage of texture and tile atlas management in WebGPU, and it composes with the byte accounting in building an LRU VRAM cache for tile buffers.

Runnable reference implementation

The whole design rests on one monotonically increasing counter. Every submission gets a number; every layer records the number of the last submission that referenced it; a layer is reusable once the queue has retired that submission.

class LayerPool {
  private readonly free: number[] = [];
  private readonly lastUsed = new Map<number, number>();   // layer -> submission
  private submission = 0;
  private retired = 0;

  constructor(private readonly layerCount: number) {
    for (let i = layerCount - 1; i >= 0; i--) this.free.push(i);
  }

  /** Called once per frame, after the encoder is submitted. */
  markSubmitted(device: GPUDevice): void {
    const id = ++this.submission;
    // Resolves when this submission has retired; nothing awaits it inline.
    void device.queue.onSubmittedWorkDone().then(() => {
      if (id > this.retired) this.retired = id;
    });
  }

  /** Called while recording, for every layer the frame reads. */
  touch(layer: number): void {
    this.lastUsed.set(layer, this.submission + 1);   // the frame being recorded
  }

  /** Eviction: the layer goes on the free list, nothing is destroyed. */
  release(layer: number): void {
    this.free.push(layer);
  }

  /** Allocation: only hand out a layer whose last reader has retired. */
  acquire(): number | null {
    for (let i = this.free.length - 1; i >= 0; i--) {
      const layer = this.free[i];
      if ((this.lastUsed.get(layer) ?? 0) <= this.retired) {
        this.free.splice(i, 1);
        return layer;
      }
    }
    return null;      // everything free is still in flight — try next frame
  }
}

The acquire returning null is the important behaviour, and it is deliberately not an error. A frame that cannot get a layer simply does not upload that tile this frame; the tile renders from the coarser zoom level below it, which is what the pyramid is for, and the allocation succeeds a frame or two later once the in-flight submissions retire. No code path ever waits.

The counter is also why nothing is destroyed. texture.destroy() is called exactly once per array, when a whole zoom level is retired — a deliberate, rare event that can afford to await onSubmittedWorkDone() because it is not on the frame path.

Parameter reference

The reserve is worth the four layers it costs. Without it, a fast zoom can find every layer in flight, upload nothing for several frames, and show a visibly blank map during exactly the interaction where the user is paying attention.

Why recycling beats destroying

There is a simpler-looking design where each tile owns its own single-layer texture and eviction destroys it. It is worth understanding why that design loses, because it is the one most people write first.

The first cost is the destroy hazard itself. A texture destroyed while a submitted command buffer still references it is invalid, so the simple design needs a deferred-destroy queue, which is the same counter this page describes plus an extra object lifetime to track. The complexity does not go away; it moves.

The second is binding. One texture per tile means one bind group per tile and a setBindGroup call per tile in the render pass — sixty state changes to draw sixty tiles. The array design binds once and selects layers with an integer that travels in the instance data, so a whole viewport of tiles is one bind group and one instanced draw.

The third is allocation churn. Creating and destroying a megabyte texture per tile, several times a second during a pan, is exactly the pattern that fragments a driver’s memory allocator. Layers in a pre-allocated array cannot fragment, because nothing is ever allocated after start-up.

The one thing recycling gives up is variable tile sizes, since every layer of an array shares dimensions. For a basemap of fixed-size tiles that is no loss at all.

Sizing the pool

Three numbers decide how large the layer array should be, and getting any of them wrong shows up as either wasted memory or a map that will not fill in.

The first is the visible set: how many tiles cover the viewport at the current zoom. For a 1440-pixel-wide window at 512-pixel tiles that is about four across and three down, so twelve — and a little more once the viewport is not aligned to the tile grid, which it never is. Sixteen is the honest number for that window.

The second is the pan margin. A tile that comes into view during a pan and has to be fetched, decoded and uploaded before it can be drawn will appear a beat late; keeping one ring of tiles outside the viewport resident removes that entirely. One ring around a four-by-three visible set is another eighteen tiles.

The third is the in-flight band — the layers that have been released but whose last reader has not retired. With two submissions in flight and a pan releasing a few tiles per frame, that is typically four to eight layers unavailable at any moment.

Sixteen plus eighteen plus eight is forty-two, and rounding up to sixty-four leaves headroom for a window larger than the one the numbers were derived from. That is the reasoning behind the default at the top of this page, and it is worth redoing rather than inheriting when the tile size or the target window changes.

Failure modes

• Destroyed texture used in a submit. A texture destroyed while an in-flight submission referenced it. Fix: do not destroy on the frame path; recycle layers instead.
• A recycled layer shows the previous tile when zoomed out. The upload overwrote mip level zero and left the older levels intact. The mip chain has to be rebuilt as part of the layer hand-off, not as part of the fetch.
• The map goes blank during a fast zoom. Every free layer is still in flight and acquire keeps returning null. Fix: hold back a small reserve, and confirm the in-flight submission count is bounded at two or three.
• Layers leak until nothing is free. release is called on some eviction paths and not others — typically the path that fires when a fetch fails. Fix: release in a finally, and assert that free plus allocated equals the layer count once per second in development.
• A tile flickers between two images. Two cache entries were handed the same layer, because acquire was called twice without the first result being recorded. Fix: make the pool the only allocator and never track layer numbers elsewhere.

Backend / Python interop note

Eviction is a client-side concern, but the tile server decides how expensive a mistake is, and one server-side property matters more than any other: cache headers that let the browser re-serve an evicted tile without a network round trip.

A tile evicted from VRAM will very often be wanted again within seconds — a pan that goes out and comes back, a zoom that overshoots. If the response carried a long Cache-Control: max-age and an ETag, the refetch is served from the HTTP cache in a millisecond and the only real cost is the decode and upload. Without them, every eviction is a potential network request, and an aggressive VRAM budget becomes a bandwidth problem instead of a memory one.

The practical guidance for a Python tile service is to treat tile URLs as immutable — put a version or a data timestamp in the path rather than in a query parameter — and to serve them with a year-long max-age. That combination lets the browser cache hold what VRAM cannot, and it turns the eviction policy into a pure memory decision rather than a memory-versus-latency trade.

Related

• Texture and tile atlas management in WebGPU — the topic this page belongs to.
• Uploading raster tiles into a texture_2d_array — what happens to a layer once it is acquired.
• Generating mipmaps for WebGPU map tiles — the chain that must be rebuilt on recycle.
• Building an LRU VRAM cache for tile buffers — the byte accounting that decides when to evict.
• VRAM budget management across tile zoom levels — how large the pool should be.