⭐ PRO+

Alerting Engine Module

Configurable alerting with multiple notification channels to keep you informed about critical infrastructure events in real-time.

Overview

The Alerting Engine provides proactive notification when your infrastructure needs attention. Define custom alert rules based on host conditions, and receive notifications through your preferred channels - email, webhooks, Slack, or Microsoft Teams.

🔔

Define Rules

Set conditions that trigger alerts

→

📩

Configure Channels

Email, Slack, Teams, webhooks

→

⚡

Get Notified

Respond to issues promptly

Alert Conditions

Create alert rules based on various infrastructure conditions:

🚫

Host Down

Alert when a host stops reporting for a configurable time threshold (default: 10 minutes).

🔄

Reboot Required

Alert when a host requires a reboot to apply system updates or configuration changes.

📦

Updates Available

Alert when a host has pending package updates, with configurable minimum count.

💾

Disk Usage

Alert when disk usage exceeds a threshold percentage on specified mount points.

🔒

CVE Severity

Alert when hosts have vulnerabilities at or above a specified severity level.

📈

Custom Metrics

Alert based on custom metric thresholds with configurable operators and values.

Notification Channels

Configure multiple notification channels to receive alerts:

📧

Email

Multiple recipients per channel
HTML-formatted alert details
Severity-based subject lines

🔗

Webhook

JSON payload with full alert details
Custom HTTP headers
Integration with any webhook-capable system

💬

Slack

Block Kit formatted messages
Severity-based color coding
Alert type-specific emojis

💻

Microsoft Teams

Adaptive Card formatting
Severity-based theme colors
Structured alert presentation

Alert Severity Levels

Alerts are classified by severity to help prioritize response:

Critical Immediate action required - critical system failures or security risks

High Urgent attention needed - significant issues affecting operations

Medium Should be addressed soon - potential problems identified

Low Monitor - minor issues that can be addressed during maintenance

Info Informational - status changes and non-critical notifications

Key Features

Alert Cooldown

Configure cooldown periods to prevent alert fatigue. Once an alert fires for a host+rule combination, subsequent alerts are suppressed for the cooldown duration (default: 60 minutes).

Host Filters

Apply alert rules to specific hosts using tag-based filters. For example, create different alerting policies for production vs. development environments.

Multi-Channel Routing

Link multiple notification channels to a single rule. Critical alerts can go to both email and Slack, while informational alerts might only go to a webhook.

Alert Acknowledgment

Acknowledge alerts to indicate they're being addressed. Acknowledged alerts are tracked with username and timestamp for audit purposes.

Alert Resolution

Mark alerts as resolved when the underlying issue is fixed. Resolution status helps track mean time to resolution (MTTR) metrics.

Creating Alert Rules

To create an alert rule:

Navigate to Alerts > Alert Rules in the main navigation
Click "Create Rule" to open the rule editor
Enter a descriptive name for the rule
Select the condition type and configure its parameters
Choose the alert severity level
Set the cooldown period (in minutes)
Select one or more notification channels
Save the rule - it will be evaluated on the next cycle

# Example: Host Down Rule Configuration
{
  "name": "Production Host Down",
  "condition_type": "host_down",
  "condition_params": {"minutes_threshold": 10},
  "severity": "critical",
  "cooldown_minutes": 60,
  "host_filter": {"tags": ["production"]},
  "channel_ids": ["email-ops", "slack-alerts"]
}

Configuring Notification Channels

Configure notification channels in Settings > Alerting:

Email Channel

{
  "name": "Operations Team",
  "channel_type": "email",
  "config": {
    "recipients": ["ops@example.com", "oncall@example.com"]
  }
}

Slack Channel

{
  "name": "Slack Alerts",
  "channel_type": "slack",
  "config": {
    "webhook_url": "https://hooks.slack.com/services/..."
  }
}

Microsoft Teams Channel

{
  "name": "Teams Alerts",
  "channel_type": "teams",
  "config": {
    "webhook_url": "https://outlook.office.com/webhook/..."
  }
}

Webhook Channel

{
  "name": "Custom Integration",
  "channel_type": "webhook",
  "config": {
    "url": "https://api.example.com/alerts",
    "method": "POST",
    "headers": {"Authorization": "Bearer token123"}
  }
}

API Access

The Alerting Engine is fully accessible via the REST API:

# Alert Rules
GET    /api/v1/alerting/rules           # List all rules
POST   /api/v1/alerting/rules           # Create a rule
GET    /api/v1/alerting/rules/{id}      # Get rule details
PUT    /api/v1/alerting/rules/{id}      # Update a rule
DELETE /api/v1/alerting/rules/{id}      # Delete a rule

# Notification Channels
GET    /api/v1/alerting/channels        # List channels
POST   /api/v1/alerting/channels        # Create channel
PUT    /api/v1/alerting/channels/{id}   # Update channel
DELETE /api/v1/alerting/channels/{id}   # Delete channel
POST   /api/v1/alerting/channels/{id}/test  # Test channel

# Alerts
GET    /api/v1/alerting/alerts          # List alerts
POST   /api/v1/alerting/alerts/{id}/acknowledge  # Acknowledge
POST   /api/v1/alerting/alerts/{id}/resolve      # Resolve

# Summary
GET    /api/v1/alerting/summary         # Alert summary stats

Alert Evaluation

The alerting engine runs in the background and evaluates all enabled rules:

Default evaluation interval: 60 seconds
Asynchronous evaluation prevents blocking the main application
Cooldown periods prevent duplicate alerts for the same issue
Notifications are dispatched immediately when alerts fire

Requirements

Professional or Enterprise license with alerting_engine module
Network connectivity to license server for module download
SMTP configuration for email notifications (optional)
Outbound network access for webhook/Slack/Teams notifications