Overview
An alert watches a metric or a resource and tells your channels when something is wrong, and again when it is back to normal.
When to use it
- The disk is getting full.
- A backup failed tonight.
- An app keeps restarting.
- A machine went offline.
How it works
- Evaluator
- Runs every 30 seconds after the sampler.
- Conditions
- A metric above or below a line for a while; a crash loop; a failure; a failed run; a machine offline.
A firing alert opens an incident and notifies its channels; it can remind you while it lasts. When the condition clears, the incident closes with a back-to-normal message.
Architecture
- Samples to Evaluator
- Evaluator to Incident: firing
- Evaluator to Channels: notify
Flows
CPU too high
- CPU of app/api stays above 0.9 cores for 5 minutes.
- The alert fires; ops gets a message.
- When it drops, ops hears it’s back to normal.
Set it up
In the console
- Create a channel first.
- Open Alerts and choose New alert.
- Build the rule as a sentence and choose channels; Test it.
With the CLI
enclave alert create api-cpu --metric cpu --resource app/api --above 0.9 --channel ops --waitWith OpenTofu
OpenTofu configuration
resource "enclave_alert" "api_cpu" {
name = "api-cpu"
channels = ["ops"]
condition = {
type = "metric"
metric = "cpu"
resources = ["app/api"]
operator = "above"
threshold = 0.9
}
}Settings reference
Every setting and output of the OpenTofu resource; the console and the CLI use the same names in their own words. Generated from platform 4d30582.
| Name | Type | Use | What it does |
|---|---|---|---|
channels | list of text | required | Channels to notify. |
condition | list of objects | required | What the alert watches. |
name | text | required | Name of the alert: 3–40 lowercase letters, digits or hyphens, starting with a letter. Changing it replaces the alert. |
enabled | true or false | optional | false turns the alert off (default on). |
project | text | optional | The project to create it in. Defaults to the provider's project (ENCLAVE_PROJECT, else default). Changing it replaces the resource. |
repeat_minutes | number | optional | Remind this often while firing; 0 (default) never. |
id | text | output | Resource ID. |
state | text | output | ok, firing, no_data or disabled (as last evaluated). |
Limits and security
Good to know
- An alert fires once; set a reminder interval to be told again.
Security
- Webhook notifications are signed when the channel has a secret.