Skip to content

Logs

Vesana can collect and search logs — file tailing, journald, Windows event log. Server-side they land in TimescaleDB hypertables, in the frontend on the LogsPage.

Architecture

flowchart LR
    F[File logs] --> AT[Agent: file tailing]
    J[journald] --> AT
    W[Windows event log] --> AT
    AT --> Q[Disk queue (backpressure)]
    Q --> ZSTD[zstd compression]
    ZSTD -->|HTTPS POST| R[Receiver /logs/ingest]
    R --> DB[(logs hypertable)]
    DB --> S[Search / FTS]
    DB --> SSE[Live tail SSE]

Prerequisites

  • Agent installed (logs are collected only by the agent — no collector path)
  • At least one log source configured per host

Add a log source

Host detail → Logs → Add source:

Field Example
Name nginx-access
Type file / journald / eventlog
Path / Unit / Channel /var/log/nginx/access.log / nginx.service / Application
Severity filter info and above
Tags optional, comma-separated

The agent picks up the new source on next config refresh.

File tailing

type: file
path: /var/log/nginx/access.log
encoding: utf-8
parse:
  format: regex
  pattern: '^(?P<ip>\S+) - - \[(?P<ts>[^\]]+)\] "(?P<method>\S+) (?P<url>\S+).*" (?P<status>\d+)'

Glob patterns allowed: /var/log/myapp/*.log. Rotated files are detected (inode-based).

journald

type: journald
unit: nginx.service
priority: 4    # 0 emerg ... 7 debug

Windows event log

type: eventlog
channel: Application
sources:
  - "VesanaAgent"
  - "Windows Update"
levels:
  - error
  - warning

/logs is the search UI. Filters:

  • Full-text search (German FTS index, ILIKE fallback for partial matches)
  • Host
  • Source
  • Severity
  • Time (relative last 1h / last 24h or absolute)

Hits in collapsible entries (full event with metadata).

Live tail

Set filter → Start live tail:

Server-Sent Events stream new entries to the UI. Auto-scroll, pause button, max 1 000 visible entries (older discarded).

Behind the scenes: GET /api/v1/logs/stream?host=...&severity=error with Content-Type: text/event-stream.

Log alert rules

Patterns that trigger automatic alerts. Three modes:

Any-match

Any log entry matching → alert.

mode: any_match
pattern: 'CRITICAL.*disk.*failed'
hosts: [host-uuid-1, host-uuid-2]

Useful for „does the word panic show up anywhere?".

Threshold

More than N matches per time window.

mode: threshold
pattern: 'auth.*failed'
window_minutes: 5
threshold: 10

Useful for brute force.

Implementation: Redis sorted set holding the last N matches with timestamps.

Absence

No match in window → alert.

mode: absence
pattern: 'backup.*completed'
window_minutes: 1440    # 24 h

Useful for „backup cron should log OK exactly once daily".

Implementation: Redis TTL key reset on each match. When the key expires without renewal → alert.

Compression

Agent compresses log packets before POST with zstd (level 3). Typically 8–12× smaller than raw. Receiver decompresses in /api/v1/logs/ingest and bulk-inserts into the hypertable.

Retention

Table Default Configurable in
logs 30 days Admin → Settings → Retention

TimescaleDB compression runs on older chunks — events stay searchable but disk-efficient (factor ~6).

Disk queue (backpressure)

Agent buffers log events locally in a disk queue (default 50 MB) before sending. If server is temporarily unreachable:

  1. Agent keeps buffering
  2. On reaching the limit, oldest events drop (logs aren't valuable enough to halt the agent)
  3. Once server is back, queue is drained

Disk queue path: /var/lib/vesana-agent/queue/ (Linux) or C:\ProgramData\Vesana\Agent\queue\.

When it isn't enough

  • Very high volumes (> 10 000 events/s per host) — run dedicated solution like Loki/Grafana, only the important parts go to Vesana
  • Structured telemetry (OpenTelemetry logs) — not in Vesana yet; workaround: log to journald, parse pattern
  • Compliance storage (WORM) — Vesana isn't WORM, separate solution needed

Next