Demonstrate context-state save and load for reproducible generation.
Encodes a prompt into context 1, saves the state to a file, then restores it in context 2 and generates the same token sequence — verifying that both runs match.
Usage: `save-load-state -m model.gguf [-n n_predict] [-ngl n_gpu_layers] [-s state_file] prompt`
See Source File
Demonstrate context-state save and load for reproducible generation.
Encodes a prompt into context 1, saves the state to a file, then restores it in context 2 and generates the same token sequence — verifying that both runs match.
Usage: `save-load-state -m model.gguf [-n n_predict] [-ngl n_gpu_layers] [-s state_file] prompt`