InputChunks

A list of tokenized input chunks produced by MtmdContext.tokenize. Supports foreach iteration over const(mtmd_input_chunk)* elements.

extern (D) @nogc nothrow

struct InputChunks {

mixin Owned!(mtmd_input_chunks, mtmd_input_chunks_free) ;

static InputChunks create();

size_t length [@property getter];

bool empty [@property getter];

const(mtmd_input_chunk)* opIndex(size_t idx);

int opApply(int delegate(const(mtmd_input_chunk)*) dg);

int opApply(int delegate(size_t, const(mtmd_input_chunk)*) dg);

size_t nTokens [@property getter];

llama_pos nPos [@property getter];

}

Members

Functions

opApply int opApply(int delegate(const(mtmd_input_chunk)*) dg): foreach (chunk; chunks) — iterates each const(mtmd_input_chunk)*.
opApply int opApply(int delegate(size_t, const(mtmd_input_chunk)*) dg): foreach (i, chunk; chunks) — indexed iteration.
opIndex const(mtmd_input_chunk)* opIndex(size_t idx): Index into the chunk list.

Mixins

__anonymous mixin Owned!(mtmd_input_chunks, mtmd_input_chunks_free): Undocumented in source.

Properties

empty bool empty [@property getter]: True when no chunks are present.
length size_t length [@property getter]: Number of chunks.
nPos llama_pos nPos [@property getter]: Total position count (may differ from nTokens for M-RoPE models).
nTokens size_t nTokens [@property getter]: Total token count across all chunks.

Static functions

create InputChunks create(): Create an empty chunk list (to be filled by MtmdContext.tokenize).

Mixed In Members

From `mixin Owned!(mtmd_input_chunks, mtmd_input_chunks_free)`

this this(): Undocumented in source.
this(this) this(this): Undocumented in source.
~this ~this(): Undocumented in source.
opCast bool opCast(): True when the handle holds a non-null pointer.
ptr T* ptr [@property getter]: Raw C pointer (mutable).
ptr const(T)* ptr [@property getter]: Raw C pointer (const view).

Meta

Source

See Implementation

llama mtmd aliases

enums

mtmd_input_chunk_type

functions

structs