A small model reads your prompt. A larger one answers from its memory.
Turn the bridge on and SmolLM2-135M reads your question while SmolLM2-360M, almost three times larger, answers from the small one's translated memory, reading exactly one token of your prompt itself. Both models and the bridge run in this tab. Every claim below has a button that checks it on your machine.
Who read what counted by the models, this reply
Your first tokens this session
Is the answer really coming from the transported memory?
Five runs of the same question, each a different memory. If the bridge works, restoring a model's own memory changes nothing, and swapping in another question's memory changes the answer.
Time to first token: the bridge against the 360M reading the prompt itself
The protocol
8 prompts × 3 paired repetitions per length, the order rotating between arms, one warm-up per arm discarded. The speed-up is native ÷ bridge, the geometric mean over prompts, with its 95% interval (docs/BRIDGE.md §3.4). The ceiling is the answering model alone ÷ the reading model alone: the most the bridge could gain. The bridge arm's own reading of the prompt is inside its time; nothing is excluded. Every run's counters are recorded, so you can check that native runs read the prompt and bridged runs did not.
The numbers
| length | first token: speed-up (95% interval) · median | first visible text | ceiling |
|---|
What we claim, and what backs it
| claim | evidence | check it |
|---|---|---|
| The bridged 360M reaches its first token sooner than the 360M reading the prompt | 1.87× (95% CI 1.83–1.91), paired, in WebAssembly on a CPU, from this page's own Measure protocol. | |
| The answering model reads one token of your prompt | The models' own counters, shown with every reply: the 135M reads all but the last token, the 360M reads 0 prompt history, and 1 bridge token crosses. | |
| The answer comes from the transported memory | Restoring a model's own memory reproduces its reply exactly; another question's memory changes the answer. | |
| The bridged answer stays close to the 360M's own | Held-out NCR 0.930 (the paper's corrected bridge on its own pair: 0.918), and 6 / 7 held-out questions right, as many as the 360M reading everything; 3 / 7 without the learned correction. | Model card → |
| LatentPort's own speed-up reproduces | Its 4B → 27B pair on two RTX 3090s with upstream's timing code and statistic: 1.36× at 128 prompt tokens, 2.94× at ~2K, 3.39× at ~16K. | The result → |
Carried to other models measured on GPUs, from each bridge's card
One reply, bridged
The small model reads
SmolLM2-135M reads every prompt token but the last, building its key/value memory, layer by layer.
The bridge translates
For each layer of the 360M, a fitted ridge map turns the 135M's residual stream into the keys and values the 360M would have built, then the paper's rank-4 learned correction refines them.
The larger model reads one token
The 360M takes that memory as its own and reads exactly one token of the prompt itself: the bridge token.
It answers
The 360M generates as usual. It skipped reading the prompt, which is where the time goes on a long one, so its first token arrives sooner.
The words on this page
- Time to first token
- From pressing Ask to the first token the answering model samples. The bridged time includes the small model's reading.
- Bridge token
- The one token of your prompt the larger model reads itself, on top of the translated memory.
- Ceiling
- The larger model alone ÷ the small model alone: the most any handoff could gain, since the small model still has to read.
- NCR
- The paper's native-closure ratio: of the gap between the larger model reading nothing and reading everything, the share the bridge closes, scored on the larger model's own replies.
- Wrong memory (donor)
- The control: the same bridge fed another question's memory. A real bridge's answer changes with it.
- 95% interval
- The range the paired speed-up's geometric mean falls in across prompts, LatentPort's own statistic.
Weights kept in this browser
The models and the bridge are downloaded from Hugging Face once, at pinned commits, and kept in this browser's storage, so a reload does not download them again. They never leave your machine.