Latent Calculator
A calculator wired into a frozen 135M model between two layers, with no tokens in either direction
A frozen language model, a calculator, and no tokens between them. SmolLM2-135M never sees the tool: a small trained head reads the operands out of its layer-16 activations, a Python calculator does the arithmetic, and a small trained encoder writes the result back into the residual stream after layer 16, once per answer step. Every model weight is frozen. Nothing is appended to the prompt. The model then answers as if it knew.
Below: the same twelve prompts through the frozen model alone, through the usual text route (the result pasted into the prompt as tokens), and through the latent port. Then the numbers over 2,000 prompts.
Random rows from the in-distribution test split (operand lengths 1-4 and 6 digits). "Query" is what the trained head read out of the activations; the calculator ran on exactly that.
| Prompt | Answer | Frozen model | Tool as text | Latent port | Query read from layer 16 |
|---|---|---|---|---|---|
| 52 - 604109 = | -604057 | 100000000000000 wrong | The last two numbers are the same, so… wrong | -604057 right | sub 52 604109 → -604057 |
| Which is larger, 3436 or 9549? | less | The answer is 9549. wrong | ## How to convert 100000000 wrong | less right | cmp 3436 9549 → less |
| Is 40 greater or less than 240434? | less | The answer is 40 less than 24043 wrong | ## How to calculate 40 percent of 2404 wrong | less right | cmp 40 240434 → less |
| Is 428583 greater or less than 53? | greater | The answer is 428583 is 42 wrong | ## How to calculate 428583 percent of wrong | greater right | cmp 428583 53 → greater |
| Compare 8891 and 65: | greater | The 8891 is a 16-bit number wrong | 8891 and 65: [less] 88 wrong | greater right | cmp 8891 65 → greater |
| What is 539451 plus 927818? | 1467269 | 539451 + 927818 is wrong | Is 539451 a prime number? wrong | 1467269 right | add 539451 927818 → 1467269 |
| 915061 * 966343 = | 884262791923 | 100000000000000 wrong | The first 10000000000 wrong | 884267296266 wrong | mul 915061 966343 → 884262791923 |
| Compute 689851 times 471707. | 325407545657 | ### Division wrong | ### Division wrong | 325407545045 wrong | mul 689851 471707 → 325407545657 |
| Compare 1212 and 975558: | less | The 1212 is a composite number. It ca… wrong | ## What is 1212 in words? wrong | less right | cmp 1212 975558 → less |
| What is 105 multiplied by 36? | 3780 | The 105 multiplied by 36 is equal to wrong | 105 x 36 = 3780 wrong | 3780 right | mul 105 36 → 3780 |
| Add 614793 and 284983: | 899776 | ``` wrong | The 284983 is a safe number. wrong | 899776 right | add 614793 284983 → 899776 |
| Add 52 and 122: | 174 | The 52 and 122 are the same as the wrong | 122 + 52 = 176. wrong | 174 right | add 52 122 → 174 |
Exact match of the generated answer. In distribution means unseen operand values at training lengths; held-out length means at least one 5-digit operand, a length never trained on.
| Route | In distribution | Held-out length | Tool tokens | ms / answer |
|---|---|---|---|---|
| Frozen model | 0% | 0% | 0.0 | 489 |
| Tool result as text (answer anywhere in output) | 19% | 18% | 6.1 | 554 |
| Single vector, oracle operands (phase 1) | 60% | 34% | 0.0 | 235 |
| Latent port, oracle operands | 91% | 85% | 0.0 | 357 |
| Latent port, learned query | 90% | 41% | 0.0 | 449 |
| Learned query, by operator | In distribution | Held-out length |
|---|---|---|
add | 99% | 34% |
sub | 98% | 26% |
mul | 66% | 18% |
cmp | 100% | 86% |
| By longest operand | 1 digits | 2 digits | 3 digits | 4 digits | 6 digits |
|---|---|---|---|---|---|
| in distribution | 100% | 100% | 100% | 95% | 79% |
- Asking. A cross-attention head with 13 learned queries reads the operator and twelve digit slots out of the layer-16 activations of every prompt token (232k parameters). It recovers the full query on 99% of in-distribution prompts. Reading one vector at the last token, the phase-1 design, managed 10%: the digits live at their own tokens, not at the question mark.
- Computing. Plain Python on the decoded query. Exact.
- Reading. An encoder turns the result (sign, digits, or a comparison word) plus the answer-step index into one vector of the model's hidden size and adds it to the residual stream after layer 16, at the query position and again at every answer step. The digits are stored most-significant first, so answer step j reads slot j. The frozen upper layers turn that into the next token. Trained by the model's own next-token loss; 91% exact with true operands.
- Cost. Zero tokens in either direction. The port's own compute is a few matrix products per step; the wall-clock difference in the tables is mostly that the frozen model rambles for 16 tokens and the ported one stops after the answer.
- Twelve-digit products. Multiplying two 6-digit numbers lands at 66%; the wrong answers have the leading digits right and the last few off. Addition and subtraction, whose results stay under eight digits, are at 99% and 98%.
- Lengths it never saw. On prompts with a 5-digit operand the reading side holds up (85% with true operands) but the query head is right only 45% of the time, so the whole pipeline lands at 41%. The head counts digit positions from the end of the prompt, and an unseen operand length shifts the count.
- The text route scores zero on exact match here because a 135M base model wraps the pasted result in prose or repeats it; the "contains" column is its fair number.
SmolLM2-135M, frozen, fp32, 4 CPU cores. Trained parts: query head 232k, result encoder about 1M. Port at layer 16 of 30, chosen by a linear-probe sweep. 20,000 synthetic training prompts over twelve templates. Code, predictions and results: oaustegard/experiments/latent-calculator.