NodeLlamaCppConfig

Defined in: native-node-llamacpp/src/index.ts:55

Extends

• Partial<BackendAdapterConfig>

Properties

apiKey?

readonly optional apiKey: string

Defined in: ai.matey.types/dist/types/adapters.d.ts:103

API key for authentication. Should be injected from environment or secure config.

Inherited from

Partial.apiKey

---

baseURL?

readonly optional baseURL: string

Defined in: ai.matey.types/dist/types/adapters.d.ts:108

Base URL for API endpoint. Useful for proxies or alternative endpoints.

Inherited from

Partial.baseURL

---

batchSize?

optional batchSize: number

Defined in: native-node-llamacpp/src/index.ts:69

Batch size for prompt processing. Default: 512

---

browserMode?

readonly optional browserMode: boolean

Defined in: ai.matey.types/dist/types/adapters.d.ts:155

Enable browser-compatible mode.

⚠️ SECURITY WARNING: Enabling browser mode may expose API keys in client-side code. This option should ONLY be used for development and testing. Production applications should always use proxy servers to protect API keys.

Each provider implements browser compatibility differently:

• Anthropic: Adds anthropic-dangerous-direct-browser-access: true header
• Gemini: Already browser-compatible (API key in URL), this flag has no effect
• OpenAI: Already browser-compatible, this flag has no effect
• Other providers: May have provider-specific implementations

Default

false

Example

// Development only - DO NOT use in production!
const backend = new AnthropicBackendAdapter({
  apiKey: process.env.ANTHROPIC_API_KEY,
  browserMode: true  // ⚠️ Exposes API key in browser
});

Inherited from

Partial.browserMode

---

cacheModels?

readonly optional cacheModels: boolean

Defined in: ai.matey.types/dist/types/adapters.d.ts:181

Enable model list caching.

Default

true

Inherited from

Partial.cacheModels

---

contextSize?

optional contextSize: number

Defined in: native-node-llamacpp/src/index.ts:59

Context window size. Default: 2048

---

custom?

readonly optional custom: Record<string, unknown>

Defined in: ai.matey.types/dist/types/adapters.d.ts:131

Provider-specific configuration options.

Inherited from

Partial.custom

---

debug?

readonly optional debug: boolean

Defined in: ai.matey.types/dist/types/adapters.d.ts:123

Enable debug logging.

Default

false

Inherited from

Partial.debug

---

defaultModel?

readonly optional defaultModel: string

Defined in: ai.matey.types/dist/types/adapters.d.ts:162

Default model to use when no model is specified in the request. This provides a fallback model for requests that don’t specify one.

Example

'gpt-4o' for OpenAI, 'claude-3-5-sonnet-20241022' for Anthropic

Inherited from

Partial.defaultModel

---

gpuLayers?

optional gpuLayers: number

Defined in: native-node-llamacpp/src/index.ts:61

Number of layers to offload to GPU. 0 = CPU only. Default: 0

---

headers?

readonly optional headers: Record<string, string>

Defined in: ai.matey.types/dist/types/adapters.d.ts:127

Custom HTTP headers to include in requests.

Inherited from

Partial.headers

---

maxRetries?

readonly optional maxRetries: number

Defined in: ai.matey.types/dist/types/adapters.d.ts:118

Maximum number of retries for transient failures.

Default

0

Inherited from

Partial.maxRetries

---

modelPath

modelPath: string

Defined in: native-node-llamacpp/src/index.ts:57

Path to the GGUF model file. Can be relative (resolved from cwd) or absolute.

---

models?

readonly optional models: readonly (string | AIModel)[]

Defined in: ai.matey.types/dist/types/adapters.d.ts:171

Static model list (used when provider doesn’t have listing endpoint or to override remote list).

Can be either:

• Array of model IDs (strings) - will be normalized to AIModel objects
• Array of full AIModel objects with capabilities

Inherited from

Partial.models

---

modelsCacheScope?

readonly optional modelsCacheScope: "global" | "instance"

Defined in: ai.matey.types/dist/types/adapters.d.ts:193

Cache scope strategy.

• ‘global’: Share cache across all adapter instances (default)
• ‘instance’: Each adapter instance has its own cache

Default

'global'

Inherited from

Partial.modelsCacheScope

---

modelsCacheTTL?

readonly optional modelsCacheTTL: number

Defined in: ai.matey.types/dist/types/adapters.d.ts:186

Cache TTL in milliseconds.

Default

3600000 (1 hour)

Inherited from

Partial.modelsCacheTTL

---

modelsEndpoint?

readonly optional modelsEndpoint: string

Defined in: ai.matey.types/dist/types/adapters.d.ts:176

URL endpoint for fetching models (overrides default). Used for custom model endpoints or proxies.

Inherited from

Partial.modelsEndpoint

---

streaming?

readonly optional streaming: StreamingConfig

Defined in: ai.matey.types/dist/types/adapters.d.ts:204

Streaming configuration for this backend.

Controls how streaming responses are delivered:

• mode: ‘delta’ (incremental only) or ‘accumulated’ (full text each chunk)
• includeBoth: Whether to provide both delta and accumulated in chunks
• bufferStrategy: How to buffer for accumulated mode

Default

{ mode: 'delta', includeBoth: false, bufferStrategy: 'memory' }

Inherited from

Partial.streaming

---

temperature?

optional temperature: number

Defined in: native-node-llamacpp/src/index.ts:63

Sampling temperature. Default: 0.7

---

threads?

optional threads: number

Defined in: native-node-llamacpp/src/index.ts:71

Number of CPU threads to use. Defaults to optimal value.

---

timeout?

readonly optional timeout: number

Defined in: ai.matey.types/dist/types/adapters.d.ts:113

Request timeout in milliseconds.

Default

30000

Inherited from

Partial.timeout

---

topK?

optional topK: number

Defined in: native-node-llamacpp/src/index.ts:67

Top-k sampling. Default: 40

---

topP?

optional topP: number

Defined in: native-node-llamacpp/src/index.ts:65

Top-p sampling. Default: 0.9