Graphorin API reference v0.12.1
Graphorin API reference / @graphorin/provider-llamacpp-node / / LlamaCppNodeAdapterOptions
Interface: LlamaCppNodeAdapterOptions
Defined in: packages/provider-llamacpp-node/src/adapter.ts:48
Options accepted by llamaCppNodeAdapter.
Stable
Properties
| Property | Modifier | Type | Description | Defined in |
|---|---|---|---|---|
acceptsSensitivity? | readonly | readonly Sensitivity[] | Sensitivity override (defaults to the loopback envelope). | packages/provider-llamacpp-node/src/adapter.ts:60 |
capabilities? | readonly | Partial<ProviderCapabilities> | Capability declaration. Merged on top of the defaults table. | packages/provider-llamacpp-node/src/adapter.ts:58 |
contextSize? | readonly | number | Optional context-window override. | packages/provider-llamacpp-node/src/adapter.ts:54 |
gpuLayers? | readonly | number | "auto" | Number of layers to offload to the GPU. Default 'auto'. | packages/provider-llamacpp-node/src/adapter.ts:52 |
modelOverride? | readonly | LlamaModelInstance | Optional model override that short-circuits the loadLlamaModule(...).loadModel(...) flow. Tests pass a fixture shaped instance. | packages/provider-llamacpp-node/src/adapter.ts:71 |
modelPath | readonly | string | Filesystem path to the .gguf model file. | packages/provider-llamacpp-node/src/adapter.ts:50 |
name? | readonly | string | Provider name attached to spans / log lines. | packages/provider-llamacpp-node/src/adapter.ts:56 |
persistentSession? | readonly | boolean | W-096: reuse ONE session (context + KV cache) across requests instead of creating and disposing a fresh one per call - an agent loop then avoids re-prefilling the growing transcript on every step. Requests serialise through a promise mutex (a llama context sequence is single-threaded), and the chat history re-syncs via setChatHistory before each prompt. Strictly opt-in: the default per-request lifecycle stays memory-safe and concurrency-safe; the cached session also skips per-request disposal (it lives until the process / instance is released). Sessions WITHOUT setChatHistory cannot re-sync and silently degrade to per-request behaviour. | packages/provider-llamacpp-node/src/adapter.ts:96 |
runtimeOverrides? | readonly | LlamaCppNodeRuntimeOverrides | Test-only runtime override. When unset the adapter loads node-llama-cpp lazily on first call. | packages/provider-llamacpp-node/src/adapter.ts:65 |
sessionFactory? | readonly | (model, system?) => Promise<LlamaSessionInstance> | Optional session factory override. When unset, the adapter builds a real session from the peer (PS-3): model.createContext() → new LlamaChatSession({ contextSequence }), streaming through prompt(text, { onTextChunk }). Overrides (runtimeOverrides.createSession or this option) keep the test seam. | packages/provider-llamacpp-node/src/adapter.ts:80 |