Anomalies & forecasts¶
Vesana learns, for every suitable check, what is normal at which hour, and forecasts for fill levels when the limit will be reached. Both run on their own once there is enough data.
No second layer of problems
The error overview, the hosts list and the host detail show statuses only. You see anomalies and forecasts in the check detail and on the Anomalies & forecasts page. They only become a problem when you switch that on for the check (or as the default) — then they set an ordinary status, see Status from anomaly and forecast.
Normal range¶
Per check, Vesana learns from the hourly values of the last 8 weeks which range is normal for that hour:
| Data available | Profile |
|---|---|
| less than 7 days | still learning ("Learning · day 3 of 7") — no normal range |
| from 7 days | daily profile: per hour, weekdays and weekends separately |
| from 21 days | weekly profile: per weekday and hour |
It uses the median and the robust spread (MAD) of the hour and its neighbouring hours — a single outlier in the learning period moves nothing. Not learned:
- periods in a downtime
- periods in which the check itself had a problem (WARNING/CRITICAL/…)
- periods with an anomaly — only their average; their peaks do count (see below), otherwise a regular peak could never prove to be normal. Anomalies marked as normal are learned fully.
- everything before "Relearn normal range"
Usual peaks are part of it. Besides the hourly average Vesana learns the peaks of every hour: what is reached at this hour on most days (backup at 3 am) and what occurs in at least 5 % of the hours over the whole learning window (builds at varying times) lies within the normal range. A one-off peak stays an anomaly.
The range has a minimum width: ±5 points for percentages, ±5 ms or half the normal value for response times. "2 % → 6 % CPU" is therefore never an anomaly, and a value that is always 0 still gets a range.
Sensitivity¶
| Level | Width | When |
|---|---|---|
| Low | 6 robust standard deviations | only clear outliers |
| Medium (default) | 4.5 | the usual case |
| High | 3.5 | quiet values where small jumps already matter |
| Off | — | this check is not evaluated |
The default applies instance-wide (see Settings) and can be overridden — on the check (check detail → Configuration, right-click on the Anomalies & forecasts page) or, as super admin, on the profile for all hosts with that profile. Order: check › profile › adjusted automatically › default.
Automatically adjusted sensitivity¶
If a check had at least 4 anomalies within 21 days that all passed on their own (no CRITICAL during the anomaly), Vesana sets it one level less sensitive overnight — never below "Low". This only happens when nobody decided themselves: no level on the check or profile, no manual normal range. The adjustment is shown with its reason in the Settings tab (filter "Adjusted automatically") and in the check detail and can be discarded there.
Manual normal range¶
If the learned range is off, set it yourself (check detail → Configuration → Normal range, or right-click → Set normal range):
| Effect | |
|---|---|
| Learned | Vesana derives the range (default) |
| At least | learned, but values in this range are never an anomaly, e.g. "up to 70 %" |
| Fixed | exactly this range, nothing learned — applies immediately, even while learning |
Super admins can also set the range for all hosts with a profile; a range on the check wins over the profile's.
Where does the normal range come from?¶
On the normal range in the check detail, in the configuration and on every anomaly, "Where from?" shows the numbers behind the current range: hours learned from, daily or weekly profile, typical value and spread of this hour, width from the sensitivity, usual peaks, a manual range and what was not learned (downtime, problem, anomaly, before "relearn").
Direction¶
Values where "too high" is the problem (CPU, load, response time, error counters) are only evaluated upwards. If the check's threshold direction is lower (free space, fill level, uptime), only downwards.
Anomaly¶
Every 5 minutes Vesana compares the average of the last 15 minutes with the normal range of the hour.
- If the value is outside for at least 15 minutes, an anomaly starts. A short spike stays quiet.
- Once it is back inside for 15 minutes, the anomaly ends.
- No anomaly while the check itself has a problem (the regular alert says so), in a downtime, when the host is unreachable, in the first 30 minutes after returning from "No Data" (a restart is always unusual) and in the first hour after a change to the check (whoever rebuilds something changes it on purpose).
- In a downtime a running anomaly rests: it does not end and is evaluated normally afterwards.
An anomaly says in one sentence what is going on: "92 % above the normal range 20–35 %".
After a rebuild (new hardware, migration, new service) use Configuration → Relearn normal range on the check: everything before no longer counts, the check learns again.
Not an anomaly? Right-click → Is normal …: the span is learned, no longer sets a status and is listed under "Marked normal"; optionally "never treat values up to X as an anomaly" right away. Is an anomaly after all undoes it. If an anomaly runs for more than a day, the row asks "new normal?" — Accept as the new normal relearns from its start.
Suitable checks¶
Every check with a numeric value. Excluded are fill levels and remaining lifetimes — disks (agent_disk, ssh_disk, SNMP storage), certificates, internet line: there the forecast answers the right question.
Forecast¶
For every fill level Vesana forecasts daily when the check's critical limit will be reached — for disks per partition, for scripts with percentage series per series (e.g. datastores). Fill level means: a percentage or a disk check. Counters and ages (backup age, number of errors) and load/CPU values get no forecast — they have a threshold, but no capacity.
| Limit | the check's CRIT threshold, otherwise WARN, otherwise 100 % for percentages |
| Direction | follows the check: for "lower" (toner, free space) against the lower limit |
| Data | daily maximum (for "lower": daily minimum) of the last 30 days |
| Clean-up | a clear drop starts over — only data since the last clean-up counts |
| Slope | robust (Theil–Sen), single outliers do not tip it |
| Result | a range: "limit 90 % in 9–14 days" |
No number if it cannot be backed up: no growth (flat, falling, within noise or more than a year away) or not enough data (fewer than 7 days since the last clean-up). RAM, CPU and load get no forecast — they do not grow, they fluctuate.
Urgent forecast¶
| Level | when the likely remaining time … |
|---|---|
| WARNING | ≤ warning days (default 14) |
| CRITICAL | ≤ critical days (default 3) |
In addition the value must have covered at least half the way to the limit — an almost empty disk that grows fast is not urgent yet. The days can be overridden per check (Configuration → Normal range & forecast).
Where you see it¶
- Check detail: in the graph the normal range as a calm band, anomalies as marked spans and, for fill levels, a dashed forecast line up to the limit; in the status tab "Normal range 20–35 %" or "Learning · day 3 of 7", a running anomaly and the forecast per partition/series.
- Anomalies & forecasts page (navigation, right-click on any row = this page's menu: is normal, normal range, sensitivity, status options, open check/device): at the top the scope — search plus tenants and devices as multi-select. Tabs Anomalies (running and ended, with a time range, sorted by deviation, a small graph per row; expanded: start, end, duration, peak, normal range and the history around the anomaly), Forecasts (all fill levels by time to the limit, with a fill-level bar and growth per day; states "Approaching", "Reached", "Stable", "Insufficient data"; expanded: the graph the forecast is calculated from: highest value per day, trend since the last clean-up, range from earliest to latest — for every partition) and Settings (per check learning progress, sensitivity with its source, normal range, status options; one check by right-click, several by ticking, all in the filter via "Select all in filter"). Refreshing the lists keeps expanded rows, the page and menus in place. From the host detail, Anomalies & forecasts leads to the page, filtered to that host.
- Dashboards: widgets Anomalies and capacity forecasts.
The error overview, the hosts list and a host's check list only show anomalies and forecasts once they have set a status — then as an ordinary WARNING/CRITICAL.
Status from anomaly and forecast¶
Whether an anomaly or a forecast may set a status is decided per check (check detail → Configuration, "Default" / "On" / "Off") or for all checks as the default (see Settings).
| Default | Status | |
|---|---|---|
| Status from forecast | on | WARNING from the warning level, CRITICAL from the critical level (see Urgent forecast) |
| Status from anomaly | off | at most WARNING, only after 30 minutes of a running anomaly and from learning day 14 |
The status is only raised, never lowered, and carries its reason in the check's message ("Anomalie: außerhalb des Normalbereichs 80–150 ms", "Prognose: C: erreicht 90 % in ~9 Tagen"). From then on it is a status like any other: error overview, alert rules, ACK and downtime apply unchanged. There are no separate notifications for anomalies and forecasts — if you want to be notified, switch the status on.
Settings¶
Admin → System → Anomalies & forecasts (super admin):
| Default | |
|---|---|
| Anomalies & forecasts | on |
| Sensitivity | Medium |
| Forecast: warning from | 14 days |
| Forecast: critical from | 3 days |
| Status from forecast | on |
| Status from anomaly | off |
| Anomaly sets status after | 30 minutes |
Off means: nothing is computed, shown or set as a status. Learned ranges stay stored and apply immediately after switching back on.
What it cannot do¶
- Seasonality beyond a week — month end, holidays: not learned as a pattern of their own.
- Correlations — "CPU high AND disk IO high" is not a joint anomaly; every check is evaluated on its own.
- Forecasts for jumpy growth — a one-off large increase (backup copy) shifts the remaining time until enough days afterwards are available.
Next¶
- Alert rules — notify once an anomaly or forecast has set a status
- Dashboards → Widget reference — widgets Anomalies and capacity forecasts