Refractometer

A prism's index is read at its angle of minimum deviation. This page sweeps WebGPU kernel configurations on this device, finds the fastest one, and shows where performance falls off a cliff. Every configuration is checked against a JavaScript reference before it is timed, so a fast kernel with wrong output cannot win. The best configurations are saved as a JSON profile keyed by the GPU adapter.

Kernels follow eg2-wgsl-engine (the 4-layer EmbeddingGemma 2 query encoder) and embeddinggemma2-webgpu-tail (the 2-bit corpus scan). Source: oaustegard/experiments/refractometer.

Adapter

Requesting a WebGPU adapter…

A full run takes a few minutes on a laptop GPU. Keep the tab in front and the device plugged in; a phone that heats up throttles, so compare runs made under the same conditions.

What each sweep measures

a) bandwidth
vec4 copy and vec4 read-reduce over buffers from 16 KiB up; the knees are where data stops fitting in a cache.
b) reduction
sum of 4M floats: workgroup size × items per thread, tree in workgroup memory.
b) q4 GEMV
the encoder's 4-bit matmuls (MatMulNBits layout) at its real shapes: columns per workgroup C, k-splits S, rows R. The engine uses C=32, S=8, R=8.
b) scan
the tail's 2-bit scan, 65,536 vectors × 8 queries: workgroup size, lookup table in workgroup memory or storage, vectors per thread. The tail uses 256 / workgroup / 1.
c) tiled matmul
f32 128×2048×512 (gate projection at 128 tokens): tile, k-step, workgroup shape and workgroup memory. The occupancy cliff is where more workgroup memory per workgroup stops paying.
d) subgroups
the reduction again, with subgroupAdd in place of the workgroup-memory tree (needs the subgroups feature).
e) f16
the best matmul tiles with f16 data, accumulating in f32 or f16 (needs shader-f16).

Timing: each sample is one compute pass of back-to-back dispatches lasting about 2 ms, timed with timestamp-query where the adapter has it, else from submit to onSubmittedWorkDone. Median and p10–p90 over 25 samples after warmup. A cliff is marked where the best configuration at some x falls below 75% (bandwidth: 80%) of the best at smaller x. Configurations above WebGPU's default limits (16 KiB workgroup memory, 256 invocations) are marked, and a second winner is kept without them, since a device created by onnxruntime-web cannot raise its limits.