AI settings

Configure LLM providers, failover order and reasoning effort, set the knowledge-base embedding model, and check provider health and 24-hour call volume.

AI settings is in the Settings group and is visible to administrators only.

AI settings

Model activity

Overview

Card	Shows
Model availability	How many enabled providers are healthy, and how deep the failover chain is
Calls (24h)	Calls, failures and p95 latency counted from the audit index. Empty when auditing is off
Failures since start	Failures since the gateway started, whether a provider is failing repeatedly, and the last failure. Counters reset on restart

Failover chain and call activity

Failover chain

The failover chain lists providers in Model routing order, with the one in use, its last success and any consecutive failures. When a provider keeps failing, the gateway moves to the next one, and every answer records which model produced it. The chart on the right shows calls over the last 24 hours.

Model failover is an Enterprise feature. If the activated license does not include it, the gateway uses only the first enabled provider.

Model routing

Routing

Each provider has a card. List order is failover order, and a new provider goes to the end. Expand a card to set these fields.

Field	Description
Kind	`openai` for any OpenAI-compatible endpoint, including Volcengine Ark, DeepSeek, Qwen, Ollama and vLLM. `azure` also needs API version and Deployment name (not the model name)
Model	`ark-code-latest` on the Ark coding plan, or the endpoint id of a custom Ark endpoint
Base URL	The model endpoint
API Key	Leave empty to keep the stored key; its last 4 characters are shown
Tags	For example `primary,cloud`. Labels only
Timeout (seconds)	Per-call limit. A provider added in the UI starts at 30; if unset in the configuration, the default is 180
Reasoning	See below

Replace a provider

Saving an empty list makes every LLM call fail, so add the new provider before you remove the old one.

Steps

Select Add a provider, fill it in and enable it.
On the old provider's card, select Remove.
Select Save and reload.

Reasoning effort

Thinking models (Doubao, GPT-5, Claude, Qwen3, Gemini 2.5 and others) reason before answering, which adds 30 to 90 seconds per call. Reasoning lets the gateway decide per task.

Level	Behavior
Auto (recommended)	No reasoning for titles, angles and index selection. Low for natural language to DSL. High for investigation, triage and detection rules
Off	No reasoning for any task
Low, High	The same level for every task

The gateway identifies the vendor from the Base URL and model name and translates the level into that vendor's parameter: Ark thinking, OpenAI reasoning_effort, Qwen enable_thinking, Claude thinking, and the Gemini and OpenRouter equivalents. Unrecognized models get no parameter. If a request with the parameter returns 400, the gateway retries without it.

On OpenAI o-series and GPT-5, High sends reasoning_effort=high, which is billed above the default.

Knowledge-base embedding model

Embedding

Runbooks need an embedding model.

Steps

In Runbook embedding model, enter the Model ID, for example doubao-embedding, text-embedding-3-small, or bge-m3 on a local Ollama.
(Optional) Enter a Base URL and API Key. Empty fields reuse the chat model's endpoint and key.
Select Test connection. On success, Vector dimensions is filled in.
Select Save.

Saving enables runbooks. If the existing KB index has a different dimension from the new model, the server refuses the save and you must rebuild the index first.

Air-gapped sites use a local Ollama with bge-m3 at http://<ollama-host>:11434/v1. bge-m3 has 1,024 dimensions and takes about 50 ms per query on CPU.

Storage

The configuration is stored in /app/state/llm_providers.yml in the state volume, with API keys encrypted. When the file does not exist, the gateway uses the LLM_* variables in .env; once you save in the UI, the file applies.

If the file has an embedding: section but no providers: section (only the embedding model was saved in the UI), the gateway also falls back to the LLM_* variables for the chat model. An explicit providers: [] means no chat model.