Skip to main content

plan

Function plan 

Source
fn plan(n_prompt: usize, k: usize) -> Plan
Expand description

The card’s ceiling is 1.3x the input plus 32, taken over the whole prompt rather than the transcript, so it sits looser than the figure it comes from.

n_ctx covers the highest position ever written: the prompt, then at most budget emitted tokens, then a whole k-token draft written speculatively past the last accepted one. That highest index is n_prompt + budget + k - 1, so the count needed is one more than that and this leaves exactly one slot spare.

A batch holds either the whole prompt on the first decode or one confirmed token plus a full draft on every later one, so it takes the larger.