NodeLlamaCppBackend
Defined in: native-node-llamacpp/src/index.ts:79
Backend adapter for node-llama-cpp.
Uses native bindings to llama.cpp for efficient local inference.
Implements
Section titled “Implements”Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new NodeLlamaCppBackend(
config):NodeLlamaCppBackend
Defined in: native-node-llamacpp/src/index.ts:99
Parameters
Section titled “Parameters”config
Section titled “config”Returns
Section titled “Returns”NodeLlamaCppBackend
Properties
Section titled “Properties”metadata
Section titled “metadata”
readonlymetadata:AdapterMetadata
Defined in: native-node-llamacpp/src/index.ts:80
Adapter metadata for identification and capabilities.
Implementation of
Section titled “Implementation of”Methods
Section titled “Methods”dispose()
Section titled “dispose()”dispose():
Promise<void>
Defined in: native-node-llamacpp/src/index.ts:350
Clean up resources.
Returns
Section titled “Returns”Promise<void>
execute()
Section titled “execute()”execute(
request):Promise<IRChatResponse>
Defined in: native-node-llamacpp/src/index.ts:176
Execute a non-streaming chat request.
Parameters
Section titled “Parameters”request
Section titled “request”Returns
Section titled “Returns”Promise<IRChatResponse>
Implementation of
Section titled “Implementation of”executeStream()
Section titled “executeStream()”executeStream(
request):IRChatStream
Defined in: native-node-llamacpp/src/index.ts:248
Execute a streaming chat request.
Parameters
Section titled “Parameters”request
Section titled “request”Returns
Section titled “Returns”Implementation of
Section titled “Implementation of”fromIR()
Section titled “fromIR()”fromIR(
request):IRChatRequest
Defined in: native-node-llamacpp/src/index.ts:158
Convert IR request to provider format (passthrough - uses IR internally).
Parameters
Section titled “Parameters”request
Section titled “request”Returns
Section titled “Returns”Implementation of
Section titled “Implementation of”initialize()
Section titled “initialize()”initialize():
Promise<void>
Defined in: native-node-llamacpp/src/index.ts:115
Initialize the model and context. Must be called before using execute() or executeStream().
Returns
Section titled “Returns”Promise<void>
toIR()
Section titled “toIR()”toIR(
response,_originalRequest,_latencyMs):IRChatResponse
Defined in: native-node-llamacpp/src/index.ts:165
Convert provider response to IR format (passthrough - uses IR internally).
Parameters
Section titled “Parameters”response
Section titled “response”_originalRequest
Section titled “_originalRequest”_latencyMs
Section titled “_latencyMs”number