EmbeddingGemma 2 with a fused WebGPU tail

The model runs in onnxruntime-web on WebGPU. Its output buffer stays on the GPU and is encoded by RemexTail into 2-bit codes (remex, seed 42) and searched with an asymmetric scan. No embedding is read back to the CPU on the encode path. The only readback before encoding is a 768-float sum that sets the centring mean.

Startup

loading

Startup gate not run yet.

Search

Corpus: 150 short sentences on 10 topics. Each hit shows the GPU score next to the exact fp32 cosine. Recall@10 compares the GPU ranking with the fp32 top 10.

Benchmark

Batch of 32 documents: ORT embed alone; embed plus getData readback plus encodeRef on the CPU; embed plus RemexTail encode on the GPU. Then a scan of 20 queries over 100,000 rows on the GPU against searchRef on the CPU (first 10,000 rows, scaled). Medians of repeated runs, timed with performance.now().