llama.batch

Undocumented in source.

Members

allocBatch OwnedBatch allocBatch(int nTokensMax, int embd): Allocates a batch for up to nTokensMax tokens. Pass embd > 0 for embedding batches.
batchAdd void batchAdd(llama_batch batch, llama_token id, llama_pos pos, llama_seq_id seqId, bool logits): Append one token to a pre-allocated batch (created via allocBatch).
batchClear void batchClear(llama_batch batch): Reset a batch's token count to zero (keeps allocated memory).
batchGetOne llama_batch batchGetOne(const(llama_token)[] tokens): Wraps a token slice into a batch. The slice must outlive the returned batch.
batchGetOne llama_batch batchGetOne(llama_token* tokens, int nTokens): Wraps a raw token pointer into a batch; for C interop.

OwnedBatch struct OwnedBatch: A llama_batch that frees itself when it goes out of scope.