How this works, and how well — sources, licence, measured numbers
What this is, and what it is not
The taxonomy is the 860 distinct product_class values in Wayfair's
WANDS dataset. Every label is embedded once, in
the page; your text is embedded the same way and the nearest label by cosine wins. That is the
entire classifier. It never emits a category that is not in the list, because it only ever
picks from the list.
It is a shortlister, not an oracle. It puts the gold category first for roughly
43% of held-out queries and in the top three for 59%. Quantising the encoder to int8
to make it a 33 MB download costs about two points of acc@1 against fp32 and
almost nothing at acc@3 — the table below carries both. Queries whose wording has nothing in
common with any label — gurney slade 56, a product name — have no
reason to land anywhere near the right answer, and do not.
Measured
| arm | acc@1 | acc@3 | tokens sent |
| this page — snap the query directly, int8 | 0.434 | 0.588 | 0 |
| the same encoder at fp32, off the page | 0.455 | 0.594 | 0 |
| a cheap model writes the label first, then snap | 0.571 | — | 6 |
| send all 860 labels, ask for a constrained choice | 0.701 | 0.744 | 5,265 |
468 queries, one gold label each. The second row is what a server round-trip buys, and the page
can score it: the written labels are baked in, produced offline by
gemini-3.5-flash-lite. The third row is the honest ceiling — when the
vocabulary fits in a prompt, sending it wins by a wide margin. This approach is for the case
where it does not: a provider enum cap, a five-thousand-label vocabulary, or per-call cost at
volume.
Running a small model locally to write the label instead does not work. Pleias
Monad (57M) and
Baguettotron (321M) score 0.425 and
0.400 — below the encoder's own baseline — because they echo the query or drift into
invented wording. As likelihood rerankers over this page's top ten they score 0.325 and 0.350,
while the right answer sits in that top ten 82.5% of the time. What a remote model contributes
here is a prior over how taxonomies name things, and that is the first thing a shrinking model
loses.
One trap worth knowing about
This page pins the encoder to the WASM backend. Running the same int8 weights through
transformers.js on device: "webgpu" returns a collapsed embedding
space — every pair of labels about 0.995 apart, so the ranking is noise — and
it does so silently, with no error and a plausible-looking result. The first version of
this page shipped that way and answered Pillow with
Wedding, Drains, Fabric, Flags, Candles. The weights are not at fault: the same
model_quantized.onnx matches fp32 PyTorch to three decimals under
onnxruntime. Hence the smoke test on load — the page holds its own gold labels, so
it checks that the index it just built can rank before it will show you anything.
Credits and licence
Queries, labels and relevance judgements from
WANDS, © Wayfair, MIT. Encoder
Xenova/gte-small (int8 ONNX,
33 MB, MIT); runtime
transformers.js. Method, arms and
artifacts: oaustegard/experiments.
Pattern after Doug Turnbull,
“Don't classify. Hallucinate!”