Texture-set overflow thrashes both backends: replace the slot ladder with texture arrays

[obiot]

Symptom

A scene stays smooth up to the multi-texture limit and then falls off a cliff the moment one more distinct texture enters the frame — reported at 16 → 17. The drop is far larger than one extra texture should cost, and it is not a hardware wall: nothing about the 17th texture is more expensive to sample than the 16th.

Mechanism

TextureCache.allocateTextureUnit() (packages/melonjs/src/video/texture/cache.js:68) scans for a free unit and, when there is none:

// No units available — flush the current batch and reset assignments
if (this.renderer.currentBatcher) {
    this.renderer.currentBatcher.flush();
}
this.units.clear();
this.usedUnits.clear();
...
emit(GPU_TEXTURE_CACHE_RESET);

The eviction policy is evict everything. One texture too many discards all N live assignments, and the emitted GPU_TEXTURE_CACHE_RESET makes every batcher drop its cached bindings too (webgl/batchers/material_batcher.js:88). QuadBatcher.addQuad holds a second copy of the same wipe when the assigned unit lands beyond the shader's sampler array (webgl/batchers/quad_batcher.js:281-292).

That is why the cliff is a cliff rather than a slope. Below the limit every texture keeps its unit for the whole frame and the scene batches. One over, and because submission order is world/z order — not texture order — an interleaved draw list makes the overflow recur: each wipe invalidates all N bindings, the next N draws each re-allocate and re-bind, and the next foreign texture wipes them again. Worst case degenerates toward a draw call per sprite plus a full re-bind cycle between them.

WebGPU has the same failure mode with a smaller blast radius. Its quad batcher resolves each quad to a slot in a pending segment of MAX_QUAD_TEXTURES = 8 (webgpu/pipeline/cache.js:15, webgpu/batchers/quad_batcher.js:207-233), and resetSegment() (:289) clears all eight slots on flush. It is submission-ordered for the same reason, so a round-robin over MAX_QUAD_TEXTURES + 1 textures flushes once per cycle there too, at a lower threshold. The difference is scope: WebGPU's reset stays inside one batcher, while WebGL's also clears the shared unit map and notifies every other batcher.

Root cause, stated plainly

Both backends emulate "many textures in one draw" with a slot ladder: N sampler bindings plus a switch on a per-quad id. They do that because neither shading language lets you index a sampler dynamically — GLSL ES 3.00 requires sampler-array indices to be constant expressions, and WGSL requires uniform control flow, which is why quad.wgsl:18-19 samples with textureSampleLevel(…, 0.0) instead of textureSample.

The ladder is the thing that has a capacity. Everything above — the wipes, the resets, the cliff — is capacity management for a structure that only exists to work around dynamic indexing.

Target: texture arrays

sampler2DArray is core in WebGL 2; texture_2d_array is core in WebGPU. One binding, N layers, and the layer index is fully dynamic, because it is a coordinate rather than control flow:

texture(uTextures, vec3(uv, layer))

This removes the ladder instead of managing it:

• No capacity to exhaust. No slot allocation, no eviction, no GPU_TEXTURE_CACHE_RESET, no cliff. MAX_ARRAY_TEXTURE_LAYERS is ≥256 on GLES 3.0.
• No reordering required. Draws never need grouping by texture set, so the painter's-algorithm constraint — which makes any sort-by-texture scheme unsound for overlapping alpha-blended sprites — never arises.
• The vertex stream does not move. The per-quad aTextureId both backends already ship becomes the layer index unchanged.
• Mips come back on WebGPU. The textureSampleLevel(…, 0.0) workaround exists only because the slot index is non-uniform across a draw. A layer coordinate is not a branch, so implicit-derivative sampling works.
• Both shaders get simpler. buildMultiTextureFragment(n) stops generating a per-count variant and collapses to one static shader; quad.wgsl loses its eight hand-written cases.
• The backends converge rather than continuing to drift, which is the standing 20.0 direction.

Since 20.0 dropped WebGL 1, sampler2DArray is unconditionally available on master — no capability check, no fallback path for the API itself.

The constraint

Every layer of an array must share dimensions and format. Depth is not the problem; uniformity is.

Projects that already pack into atlases bucket well — atlas pages tend to be a handful of identical power-of-two sheets, which is precisely the shape an array wants. Loose images of arbitrary size do not. So the design work is:

1. Size-class bucketing — one array per (dimensions, format) class. Arrays are still banks, but far coarser ones, and switching between them is rare rather than per-overflow.
2. Layer admission — allocating a layer for a texture first seen mid-session, via texSubImage3D / copyExternalImageToTexture with a z offset, plus a growth policy when a class fills (reallocate-and-copy, or a second array in the same class).
3. What stays out — render targets, video textures with changing dimensions, and anything whose size is not known at admission.

Fallback

Textures that cannot be bucketed keep the existing slot ladder, so the ladder does not disappear — it stops being the common path. Two ladder improvements remain worthwhile on their own merits, and are worth landing first because they are small, independent, and useful even if the array work stalls:

• Bounded eviction. Replace the wipe in allocateTextureUnit with an eviction that frees only what it needs, and narrow GPU_TEXTURE_CACHE_RESET so overflow in one batcher stops invalidating every other one. Converts an N-binding loss into a 1-binding replacement.
• Reclaim the WebGPU sampler budget. Group 1 spends two bindings per slot (pipeline/cache.js:266-277) — 8 textures + 8 samplers, half of each base per-stage limit (16/16). But TextureStore.getSampler() already deduplicates samplers by (filter, addressModeU, addressModeV, mipmaps), so a segment resolves to one or two distinct sampler objects bound redundantly across 8 bindings. Selecting the sampler by a small id beside the texture switch reallocates that budget toward texture slots — on the order of 12 + 4 for the same 16 bindings. It is a trade: per-stage sampled-texture headroom drops from 8 to 4 while sampler headroom rises from 8 to 12, so it needs checking against what the lit family (color + normal, plus the map_d opacity pair) and ShaderEffect's extra samplers bind in the same fragment stage.

Not pursuing: bank-coherent submission. Sorting draws by texture set was the other way to bound the ladder's cost, and it is the one that collides with alpha-blended draw order — sound only for non-overlapping runs, depth-tested content, or an explicit opt-in. Texture arrays make it unnecessary, so it should not be built.

Explicitly not the fix

• LRU eviction. The obvious policy, defeated by exactly the access pattern that triggers this: a round-robin over N+1 textures evicts, every time, the one needed next. Helps the skewed case, worthless in the pathological one.
• Raising the ladder width. Many desktop GPUs report 32 MAX_TEXTURE_IMAGE_UNITS, so maxBatchTextures could rise where the device allows. That moves the cliff, it does not remove it, and it costs shader compile time and hurts on mobile. A knob.
• Telling users to pack their atlases better. Already the recommended practice and already done in the projects that hit this. The engine should degrade predictably when it is not enough.

Later

binding_array (bindless) would remove the ladder for the non-bucketable tail as well, with no uniformity constraint at all — but it is a WGSL feature with no WebGL 2 equivalent, so it can only ever help one backend, and not yet. Texture arrays are the portable answer and do not block it.

Before any of it

There is no instrumentation for this today. Land flushes-per-frame and cache-resets-per-frame counters first, so the "before" is measured rather than inferred. The claim to validate is that a scene above the ladder width stops producing resets at all once its textures live in an array.

Naming

MAX_QUAD_TEXTURES = 8 on WebGPU and maxBatchTextures = min(maxTextures, 16) on WebGL are two spellings of one concept. Whatever lands should give them one name in one place.