How to set up a Prometheus CPU high usage alert step by step

Why monitor CPU usage with Prometheus?

Keeping servers running without unexpected downtime requires constant vigilance over hardware resources. While monitoring server power consumption via IPMI reveals electrical costs, the logical behavior of a machine depends on software performance metrics. Prometheus has become the industry standard for monitoring because it collects metrics continuously as time-series data. In practice, this means it stores numerical values accompanied by timestamps, allowing operators to look back and understand application behavior over days.

When a server's CPU hits maximum capacity, processes slow down, HTTP requests take longer to answer, and users experience widespread latency. The major challenge in modern engineering is not just discovering that a system crashed, but getting warned before the issue impacts the end user. To achieve this goal, we build automated rules that watch performance indicators and dispatch warnings when the processor remains overloaded for too long.

How PromQL calculates CPU consumption

To build an efficient alert, we must understand how Prometheus views processor activity. The monitoring system does not read ready-made percentages directly from the operating system, but rather cumulative counters of time spent by the CPU executing various tasks. Each core spends cycles in user mode, system mode, waiting for disk input/output, or simply idling.

The Prometheus query language, called PromQL, transforms these raw counters into percentage rates that are easy to analyze. We use functions like rate() combined with the node_cpu_seconds_total selector to compute how much the processor was utilized over recent minutes. Just as we diagnose bottlenecks in physical servers, tracking CPU usage prevents cascading failures, much like freeing up disk space on Linux when Docker fills the cache prevents sudden crashes from storage exhaustion.

Step-by-step guide to configuring the alert rule YAML

Alerting rules in Prometheus reside inside YAML configuration files read by the server upon startup. We need to define a specific file, usually named alert.rules.yml, where we specify the mathematical expression, severity level, and wait time before dispatching the warning.

To configure the rule in a practical and safe manner, follow the procedure below in your monitoring environment:

1. Open your server terminal and navigate to the directory where Prometheus configuration files are stored, typically located at /etc/prometheus/.
2. Create a new text file named cpu_alerts.yml using an editor such as nano or vim.
3. Insert the alert rule structure using PromQL to detect when CPU usage exceeds the safe threshold of eighty percent for more than five minutes.

groups:
- name: cpu_alerts
  rules:
  - alert: HighCpuUsage
    expr: 100 - (avg by (instance) (rate(node_cpu_seconds_total{mode='idle'}[5m])) * 100) > 80
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: 'High CPU usage detected on instance {{ $labels.instance }}'
      description: 'Processor consumption is above 80% for the last 5 minutes.'

The parameter for: 5m in the code above acts as a patience filter for the system. In practice, it ensures that Prometheus waits for the processor to stay overloaded for five continuous minutes before triggering the alarm, preventing false positives caused by brief spikes lasting only a few seconds.

Integrating Prometheus with Alertmanager

Creating the rule inside Prometheus is only the first step of intelligent monitoring. The Prometheus server detects that the condition has been met, but Alertmanager is the complementary component designed specifically to manage notifications and deliver messages to the human team.

Alertmanager receives raw generated alerts, eliminates duplicates when multiple servers fail at the same time, groups correlated events, and routes them to external channels. We can configure it to send instant messages to Slack, dispatch emails to the on-call team, or even trigger phone call bots for critical emergency incidents in the middle of the night.

Final thoughts on proactive monitoring

Configuring well-calibrated CPU alerts transforms the operation of a technological infrastructure, shifting the team from a reactive posture to preventive action. The secret to good monitoring is not generating hundreds of noisy warnings that end up being ignored, but building assertive rules based on real usage data. Tuning tolerance thresholds and integrating tools like Prometheus and Alertmanager ensures stability and peace of mind to support continuously growing applications.