NVIDIABackendAdapter
Defined in: backend/src/providers/nvidia.ts:67
Backend adapter for NVIDIA NIM API.
NVIDIA NIM (NVIDIA Inference Microservices) provides optimized inference with OpenAI-compatible API. Supports various models including Llama, Mistral, and other popular open-source models.
Examples
Section titled “Examples”import { NVIDIABackendAdapter } from 'ai.matey';
const adapter = new NVIDIABackendAdapter({ apiKey: process.env.NVIDIA_API_KEY,});const adapter = new NVIDIABackendAdapter({ apiKey: process.env.NVIDIA_API_KEY,});
const response = await adapter.execute({ messages: [{ role: 'user', content: 'Hello!' }], parameters: { model: 'meta/llama-3.1-405b-instruct', },});const adapter = new NVIDIABackendAdapter({ apiKey: process.env.NVIDIA_API_KEY, baseURL: 'https://your-nim-endpoint.nvidia.com/v1',});const stream = adapter.executeStream({ messages: [{ role: 'user', content: 'Tell me a story' }], parameters: { model: 'nvidia/llama-3.1-nemotron-70b-instruct', },});
for await (const chunk of stream) { if (chunk.type === 'content') { console.log(chunk.delta); }}Extends
Section titled “Extends”Implements
Section titled “Implements”Constructors
Section titled “Constructors”Constructor
Section titled “Constructor”new NVIDIABackendAdapter(
config):NVIDIABackendAdapter
Defined in: backend/src/providers/nvidia.ts:71
Parameters
Section titled “Parameters”config
Section titled “config”Returns
Section titled “Returns”NVIDIABackendAdapter
Overrides
Section titled “Overrides”OpenAIBackendAdapter.constructor
Properties
Section titled “Properties”baseURL
Section titled “baseURL”
protectedreadonlybaseURL:string
Defined in: backend/src/providers/openai.ts:162
Inherited from
Section titled “Inherited from”config
Section titled “config”
protectedreadonlyconfig:BackendAdapterConfig
Defined in: backend/src/providers/openai.ts:161
Inherited from
Section titled “Inherited from”metadata
Section titled “metadata”
readonlymetadata:AdapterMetadata
Defined in: backend/src/providers/openai.ts:160
Adapter metadata for identification and capabilities.
Implementation of
Section titled “Implementation of”Inherited from
Section titled “Inherited from”Methods
Section titled “Methods”estimateCost()
Section titled “estimateCost()”estimateCost(
request):Promise<number|null>
Defined in: backend/src/providers/nvidia.ts:127
Estimate cost for NVIDIA NIM.
Parameters
Section titled “Parameters”request
Section titled “request”Returns
Section titled “Returns”Promise<number | null>
Implementation of
Section titled “Implementation of”Overrides
Section titled “Overrides”OpenAIBackendAdapter.estimateCost
execute()
Section titled “execute()”execute(
request,signal?):Promise<IRChatResponse>
Defined in: backend/src/providers/openai.ts:221
Execute non-streaming chat completion request.
Parameters
Section titled “Parameters”request
Section titled “request”signal?
Section titled “signal?”AbortSignal
Returns
Section titled “Returns”Promise<IRChatResponse>
Implementation of
Section titled “Implementation of”Inherited from
Section titled “Inherited from”executeStream()
Section titled “executeStream()”executeStream(
request,signal?):IRChatStream
Defined in: backend/src/providers/openai.ts:260
Execute streaming chat completion request.
Parameters
Section titled “Parameters”request
Section titled “request”signal?
Section titled “signal?”AbortSignal
Returns
Section titled “Returns”Implementation of
Section titled “Implementation of”Inherited from
Section titled “Inherited from”OpenAIBackendAdapter.executeStream
fromIR()
Section titled “fromIR()”fromIR(
request):OpenAIRequest
Defined in: backend/src/providers/openai.ts:605
Convert IR request to OpenAI format.
Public method for testing and debugging - see what will be sent to OpenAI.
Parameters
Section titled “Parameters”request
Section titled “request”Returns
Section titled “Returns”Implementation of
Section titled “Implementation of”Inherited from
Section titled “Inherited from”healthCheck()
Section titled “healthCheck()”healthCheck():
Promise<boolean>
Defined in: backend/src/providers/nvidia.ts:107
Health check for NVIDIA NIM API.
Returns
Section titled “Returns”Promise<boolean>
Implementation of
Section titled “Implementation of”Overrides
Section titled “Overrides”OpenAIBackendAdapter.healthCheck
invalidateModelCache()
Section titled “invalidateModelCache()”invalidateModelCache():
OpenAIBackendAdapter
Defined in: backend/src/providers/openai.ts:591
Invalidate the cached model list.
Forces the next listModels() call to fetch fresh data from the API (unless static models are configured).
Returns
Section titled “Returns”This adapter for method chaining
Inherited from
Section titled “Inherited from”OpenAIBackendAdapter.invalidateModelCache
listModels()
Section titled “listModels()”listModels(
options?):Promise<ListModelsResult>
Defined in: backend/src/providers/openai.ts:501
List available models from OpenAI.
This method supports three sources:
- Static config (config.models) - highest priority
- Cache (if enabled and not expired)
- Remote API (/v1/models endpoint)
Parameters
Section titled “Parameters”options?
Section titled “options?”Returns
Section titled “Returns”Promise<ListModelsResult>
Implementation of
Section titled “Implementation of”Inherited from
Section titled “Inherited from”OpenAIBackendAdapter.listModels
toIR()
Section titled “toIR()”toIR(
response,originalRequest,latencyMs):IRChatResponse
Defined in: backend/src/providers/openai.ts:652
Convert OpenAI response to IR format.
Public method for testing and debugging - parse OpenAI responses manually.
Parameters
Section titled “Parameters”response
Section titled “response”originalRequest
Section titled “originalRequest”latencyMs
Section titled “latencyMs”number