SamplerChain

Members

Functions

accept

void accept(llama_token token)

Feed the accepted token back (needed for repetition penalties and similar).

dist

SamplerChain dist(uint seed)

Adds stochastic (dist) sampling with an optional seed.

dry

SamplerChain dry(const(llama_vocab)* vocab, int nCtxTrain, float multiplier, float base, int allowedLength, int penaltyLastN, string[] seqBreakers)

Adds DRY (Don't Repeat Yourself) sampling. seqBreakers lists token strings that reset the repetition check (e.g. ["\n"]). Pass seqBreakers = [] to use no breakers.

grammar

SamplerChain grammar(const(llama_vocab)* vocab, string grammarStr, string grammarRoot)

Adds grammar-constrained sampling. grammarStr is a GBNF grammar; grammarRoot is the root rule name (usually "root").

greedy

SamplerChain greedy()

Adds greedy (argmax) sampling.

logitBias

SamplerChain logitBias(int nVocab, const(llama_logit_bias)[] biases)

Adds per-token logit bias adjustments. Each entry in biases is a {token, bias} pair; bias > 0 increases probability, bias < 0 decreases it.

minP

SamplerChain minP(float p, size_t minKeep)

Adds min-P sampling.

mirostatV2

SamplerChain mirostatV2(float tau, float eta, uint seed)

Adds Mirostat v2 sampling (adaptive entropy targeting).

penalties

SamplerChain penalties(int penaltyLastN, float penaltyRepeat, float penaltyFreq, float penaltyPresent)

Adds repetition / frequency / presence penalties. penaltyLastN = -1 uses the full context; penaltyLastN = 0 disables the penalty.

printPerf

void printPerf()

Undocumented in source. Be warned that the author may not have intended to support it.

sample

llama_token sample(llama_context* ctx, int batchIdx)

Sample the next token. batchIdx = -1 uses the last output position.

sample

llama_token sample(LlamaContext ctx, int batchIdx)

Sample the next token from a LlamaContext.

temp

SamplerChain temp(float t)

Adds temperature scaling.

tempExt

SamplerChain tempExt(float t, float delta, float exponent)

Adds temperature sampling with dynamic range extension (delta and exponent).

topK

SamplerChain topK(int k)

Adds top-K filtering.

topNSigma

SamplerChain topNSigma(float n)

Adds top-N-sigma sampling (keeps tokens within n sigma of the top logit).

topP

SamplerChain topP(float p, size_t minKeep)

Adds top-P (nucleus) sampling.

typical

SamplerChain typical(float p, size_t minKeep)

Adds typical-P sampling.

xtc

SamplerChain xtc(float p, float t, size_t minKeep, uint seed)

Adds XTC (exclude top choices) sampling.

Mixins

__anonymous

mixin Owned!(llama_sampler, llama_sampler_free)

Undocumented in source.

Static functions

create

SamplerChain create(bool noPerf)

Create a new sampler chain.

Mixed In Members

From mixin Owned!(llama_sampler, llama_sampler_free)

this

this()

Undocumented in source.

this(this)

this(this)

Undocumented in source.

~this

~this()

Undocumented in source.

opCast

bool opCast()

True when the handle holds a non-null pointer.

ptr

T* ptr [@property getter]

Raw C pointer (mutable).

ptr

const(T)* ptr [@property getter]

Raw C pointer (const view).

• SamplerChain