AI provider setup¶

Three providers to choose from. Exactly one is active as the chat provider per instance; the embedding model can be configured independently.

Ollama (local, recommended for self-hosting)¶

Pros¶

data stays in the stack
no API cost
no internet access needed

Prerequisites¶

Several GB of RAM for a usable model — small 3B models run with a few GB, larger models need considerably more. Don't plan Ollama on a tightly sized instance.
GPU recommended, but optional (NVIDIA with nvidia-container-toolkit)
disk space for the downloaded models

Activate¶

Ollama is an optional compose profile and off by default. To activate:

# .env
COMPOSE_PROFILES=ai
AI_ENABLED=true

docker compose --profile ai up -d ollama

GPU acceleration¶

In the compose file, GPU passthrough for the ollama service is commented out by default:

ollama:
  image: ollama/ollama:latest
  profiles: [ai]
  # Uncomment for GPU acceleration (requires nvidia-container-toolkit):
  # deploy:
  #   resources:
  #     reservations:
  #       devices:
  #         - capabilities: [gpu]

Without a GPU, Ollama runs on the CPU — workable for smaller models, noticeably slower for large ones.

Install a model¶

Admin → AI:

Provider: Ollama
Enter a model name (default on a fresh install: qwen2.5:3b — a compact model that's practical even without a GPU)
Install — progress streams live
Enter and install an embedding model separately (e.g. nomic-embed-text)

Larger models generally give better answers, but need more RAM/VRAM accordingly. If you want more quality than the default model offers, enter any model available through Ollama — the rule of thumb stays: bigger model = more RAM needed.

Manage models¶

Admin → AI → Models:

list installed models
delete models
install additional models

Anthropic (cloud)¶

Prerequisites¶

an Anthropic account + API key
outbound access to the Anthropic API

Configuration¶

Admin → AI:

Provider: Anthropic
set the API key
enter a model name (Vesana falls back internally to a current Haiku model by default if none is set — any other Anthropic model can be entered freely)
configure the embedding model separately (always runs via Ollama, see above — Anthropic has no embedding API)

Cost¶

Pay-as-you-go per token, depending on the chosen model and usage. Keep an eye on costs in the Anthropic console, especially with many tenants/users.

External OpenAI-compatible provider¶

For vLLM, LM Studio, text-generation-inference, or any other endpoint with an OpenAI-compatible chat-completion API:

Admin → AI:

Provider: External
enter the API URL (e.g. http://gpu-server:8080/v1)
API key (if the endpoint requires one)
model name

Test connection¶

After configuring: Test connection sends a small test request to the chosen provider and shows whether it's reachable (for Ollama, also the list of installed models).

Configurable parameters¶

Parameter	Meaning
`provider`	`ollama` / `anthropic` / `external`
`model`	chat model
`embed_model`	embedding model (always runs via Ollama with Anthropic)
`temperature`	default 0.3 — low = factual, high = creative
`api_url` / `api_key`	for Anthropic and External (key stored encrypted)

Other sampling parameters (output length, nucleus sampling) aren't currently separately configurable in the admin UI.

Behavior on provider outage¶

If the configured provider is unreachable (Anthropic outage, Ollama container stopped, external endpoint down), the request fails with an error message — there's deliberately no automatic switch to another provider, to avoid surprise cloud costs or unexpected data flowing to a different vendor. Cached service analyses remain available regardless.

Permission¶

admin.system covers the entire provider configuration (provider, model, API key, model install/delete, connection test). Other users only see whether AI is active, without access to provider details.

Next¶

AI chat & analysis — what the provider is needed for
Wiki — embeddings are only useful once wiki articles exist