Skip to main content

generate

Function generate 

Source
pub fn generate(
    model: &LlamaModel,
    session: &mut Session<'_>,
    source: &[LlamaToken],
    ngram: usize,
    k: usize,
) -> Result<String, NormalizeError>
Expand description

Greedy generation, speculating from source.

ctx must already hold the decoded prompt, and start is the position the next token occupies.