Skip to content

AI provider setup

Three providers to choose from. Exactly one is active as the chat provider per instance; the embedding model can be configured independently.

Pros

  • requests to the model stay in the stack (the AI's web search still goes outside, see Privacy)
  • no API cost
  • no cloud account needed (the Ollama image and the models are downloaded once from the internet)

Prerequisites

  • Several GB of RAM for a usable model — small 3B models run with a few GB, larger models need considerably more. Don't plan Ollama on a tightly sized instance.
  • GPU recommended, but optional (NVIDIA with nvidia-container-toolkit)
  • disk space for the downloaded models

Activate

Admin → System → AI:

  1. Turn on AI enabled — the main switch for chat, service analysis and wiki embeddings.
  2. Card Local AI (Ollama container) → Enable Ollama. Vesana pulls the Ollama image and starts the container; the first time this takes a few minutes. The setting survives updates.

Disable stops and removes the container. Downloaded models are kept by default; with Also delete downloaded models their disk space is freed as well.

Without the UI, the same works via .env — the UI writes the same value:

# .env
COMPOSE_PROFILES=ai

docker compose --profile ai up -d ollama

GPU acceleration

In the compose file, GPU passthrough for the ollama service is commented out by default:

ollama:
  image: ollama/ollama:latest
  profiles: [ai]
  # Uncomment for GPU acceleration (requires nvidia-container-toolkit):
  # deploy:
  #   resources:
  #     reservations:
  #       devices:
  #         - capabilities: [gpu]

Without a GPU, Ollama runs on the CPU — workable for smaller models, noticeably slower for large ones.

Choose a model

Admin → System → AI, provider Ollama, section Chat Model:

  • Four suggested models, one per size class (very small to large), with download size, parameter count and the rating fits / tight / too large — measured against the server's free memory (only when Ollama runs on the same server). Recommended is the largest class that fits comfortably, otherwise the smallest one that fits tightly. The suggestions come from the Vesana licence portal, which checks the current models of the Ollama library every night; if it cannot be reached, Vesana shows a built-in list. Download fetches a model, progress streams live; clicking an installed model makes it the active chat model.
  • Use a different model: enter any name from the Ollama library and press Enter to download.
  • Other installed models lists everything else; installed models can also be deleted there.

Larger models generally give better answers, but need more RAM/VRAM accordingly.

The Ollama server address can be pointed at your own Ollama instance under Advanced: Change server address.

Only visible when the preview feature "Wiki" is turned on. Section Wiki Search: download nomic-embed-text and activate it with a click. If no dedicated embedding model is set, Vesana computes embeddings with the chat model — that works, but is slower and less accurate.

Anthropic (cloud)

Prerequisites

  • an Anthropic account + API key
  • outbound access to the Anthropic API

Configuration

Admin → System → AI:

  1. Provider: Anthropic
  2. paste the API key from the Anthropic console — Vesana sets the API URL itself
  3. enter a model name, e.g. claude-haiku-4-5-20251001 (fast, inexpensive), claude-sonnet-5 or claude-opus-5. The name is required; Vesana does not replace an empty field with a default model.
  4. Only with the wiki turned on: Anthropic has no embedding API. Semantic wiki search additionally needs the Ollama container with nomic-embed-text (the Local AI card and the Wiki Search section appear for the Anthropic provider for this reason). Without an embedding model, the AI finds wiki articles via full-text search only.

Vesana remembers per model which parameters it rejects and leaves them out of further requests — newer models that, for example, don't accept a temperature work without extra settings. For such models the temperature setting has no effect.

Cost

Pay-as-you-go per token, depending on the chosen model and usage. Keep an eye on costs in the Anthropic console, especially with many tenants/users.

External OpenAI-compatible provider

For vLLM, LM Studio, text-generation-inference, or any other endpoint with an OpenAI-compatible chat-completion API:

Admin → System → AI:

  1. Provider: External Provider
  2. enter the API URL as the base URL without /v1 (e.g. http://gpu-server:8080 or https://api.openai.com) — Vesana appends /v1/chat/completions, /v1/embeddings and /v1/models itself
  3. API key (if the endpoint requires one)
  4. the provider's exact model name

With this provider, embeddings for the wiki search go to the same provider's /v1/embeddings endpoint.

Test connection

Test queries the chosen provider's model list (uses no tokens) and shows whether it is reachable and the key is valid.

API key and address belong together

A stored API key is only sent to the address it was stored for. If you change the provider or the API URL, re-enter the key — otherwise Vesana rejects both saving and the test.

Configurable parameters

Parameter Meaning
provider ollama / anthropic / external
model chat model
embed_model embedding model; empty = chat model (always via Ollama with Anthropic)
temperature default 0.3, adjustable 0–1 under Advanced Settings — low = factual, high = creative
api_url / api_key for Anthropic and External (key stored encrypted)

Other sampling parameters (output length, nucleus sampling) aren't currently separately configurable in the admin UI.

Behavior on provider outage

If the configured provider is unreachable (Anthropic outage, Ollama container stopped, external endpoint down), the request fails with an error message — there's deliberately no automatic switch to another provider, to avoid surprise cloud costs or unexpected data flowing to a different vendor. Cached service analyses remain available regardless.

Permission

admin.system covers the entire provider configuration (provider, model, API key, model install/delete, connection test). Saving and Test additionally require a fresh identity confirmation. Only a super admin can turn the Ollama container on or off. Other users only see whether AI is active, without access to provider details.

Next

  • AI chat & analysis — what the provider is needed for
  • Wiki — embeddings are only useful once wiki articles exist