AI models
Which LLMs, in what failover order, how much reasoning; the knowledge-base embedding model; provider health and 24-hour call volume.
Overview
| Card | Meaning |
|---|---|
| Model availability | How many enabled providers are healthy; how many failover levels |
| Calls (24 h) | Calls, failures and p95 latency from the audit index (absent when audit is off) |
| Failures total | Since gateway start, whether a provider is failing consecutively, the last failure |
Failover chain and activity
In routing order: the active provider, last success, consecutive failures. A provider that keeps failing drops to the next one, and every answer records which model produced it. On the right, the 24-hour call curve.
Model routing
One card per provider; expand for settings:
| Field | Notes |
|---|---|
| Kind | openai (every OpenAI-compatible endpoint: Volcengine Ark, DeepSeek, Qwen, Ollama, vLLM …) or azure (needs API version and deployment name, not a model name) |
| Model | ark-code-latest on the Ark coding plan, an endpoint id for a custom Ark endpoint |
| Base URL | The endpoint |
| API key | Paste a new one; empty keeps the stored key (last 4 shown) |
| Tags | primary,cloud; labels only |
| Timeout (s) | Per-call cap, 180 default |
| Reasoning | See below |
The order is the failover order; drag to change. To replace a provider: "Add provider", fill and enable the new one, delete the old one, then "Save and reload". Saving an empty list fails every LLM call.
Reasoning level
Thinking models (Doubao / GPT-5 / Claude / Qwen3 / Gemini 2.5 …) reason before answering, costing 30–90 s per call. The gateway grades by task:
| Level | Behaviour |
|---|---|
| Auto (recommended) | No reasoning for titles, angles, index picking; low for NL→DSL; high for investigation, triage, detection rules |
| Off | No reasoning anywhere |
| Low / High | Override every task |
The gateway identifies the vendor from the base URL and model name and translates the level into each vendor's parameter (Ark thinking, OpenAI reasoning_effort, Qwen enable_thinking, Claude thinking, Gemini, OpenRouter); unknown models get no parameter, and a 400 on the parameter triggers a retry without it. On OpenAI o-series / GPT-5, "High" sends reasoning_effort=high, which bills more than the default.
Knowledge-base embedding model
Runbooks need an embedding model. Enter the model id (doubao-embedding, text-embedding-3-small, a local Ollama bge-m3), endpoint (empty reuses the chat endpoint), key (empty reuses it). "Test connection" fills in the vector dimension on success; save to enable the knowledge base.
Saving is refused when the existing KB index's dimension does not match the new model; rebuild the index first.
Air-gapped sites: local Ollama + bge-m3 (1,024 dims, ~50 ms per query on CPU), endpoint http://<ollama-host>:11434/v1.
Where it is stored
/app/state/llm_providers.yml in the state volume, API keys encrypted. The LLM_* variables in .env only seed the first start.
Users and roles
Three roles: administrators change configuration, analysts investigate, read-only only looks. Create, disable, reset passwords, delete; role changes apply at once. Needs the separate user table (Postgres).
Activation
Which capabilities need a licence, how to get a trial, online and offline activation, licence states, upgrade notes.