llama-cpp-d ~master (2026-03-20T18:06:26.5280947)
Dub
Repo
fimPreToken
llama
vocab
Undocumented in source. Be warned that the author may not have intended to support it.
@
trusted
@
nogc
nothrow
llama_token
fimPreToken
(
const
(
llama_vocab
)*
vocab
)
Meta
Source
See Implementation
llama
vocab
functions
bosToken
detokenize
eosToken
eotToken
fimMidToken
fimPadToken
fimPreToken
fimRepToken
fimSepToken
fimSufToken
isControl
isEog
nlToken
padToken
sepToken
tokenAttr
tokenScore
tokenText
tokenToString
tokenize
vocabType