Alerta event handler

Alerta is a monitoring tool used to consolidate and deduplicate alerts from multiple sources for quick ‘at-a-glance’ visualization. Kapacitor can be configured to send alert messages to Alerta.

Configuration

Configuration as well as default option values for the Alerta event handler are set in your kapacitor.conf. Below is an example configuration:

[alerta]
  enabled = true
  url = "http://127.0.0.1"
  token = "mysupersecretauthtoken"
  environment = "production"
  origin = "kapacitor"

enabled

Set to true to enable the Alerta event handler.

url

The Alerta URL.

token

Default Alerta authentication token.

token-prefix

Default token prefix. If you receive invalid token errors, you may need to change this to “Key”.

environment

Default Alerta environment.

origin

Default origin of alert.

correlate

When an alert with the same resource is received with an event in the correlate list of related events, the alert is correlated. For more information, see Alerta documentation.

Options

The following Alerta event handler options can be set in a handler file or when using .alerta() in a TICKscript.

* Required

Note: The resource and event properties are required. Alerta cannot be configured globally because of these required properties.

Example: handler file

topic: topic-name
id: handler-id
kind: alerta
options:
  token: 'mysupersecretauthtoken'
  token-prefix: 'Bearer'
  resource: '{{ .Name }}'
  event: '{{ .ID }}'
  environment: 'Production'
  group: '{{ .Group }}'
  value: 'some-value'
  origin: 'kapacitor'
  correlate: 'event1', 'event2'
  service: ['service1', 'service2']
  correlate: ['service1', 'service2']
  timeout: 24h
  attributes:
    key1: value1
    key2: 8
    booleanAttribute: TRUE

Example: TICKscript

|alert()
  // ...
  .stateChangesOnly()
  .alerta()
    .token('mysupersecretauthtoken')
    .tokenPrefix('Bearer')
    .resource('{{ .Name }}')
    .event('{{ .ID }}')
    .environment('Production')
    .group('{{ .Group }}')
    .value('some-value')
    .origin('kapacitor')
    .correlate('event1', 'event2')
    .service('service1', 'service2')
    .correlated('service1', 'service2')
    .timeout(24h)
    .attribute('booleanAttribute', TRUE)
    .attribute('key1', 'value1')

Using the Alerta event handler

With the Alerta event handler enabled and configured in your kapacitor.conf, use the .alerta() attribute in your TICKscripts to send alerts to Alerta or define an Alerta handler that subscribes to a topic and sends published alerts to Alerta.

To avoid posting a message every alert interval, use AlertNode.StateChangesOnly so only events where the alert changed state are sent to Alerta.

The examples below use the following Alerta configuration defined in the kapacitor.conf:

Alerta settings in kapacitor.conf

[alerta]
  enabled = true
  url = "http://127.0.0.1"
  token = "mysupersecretauthtoken"
  environment = "production"
  origin = "kapacitor"

Send alerts to an Alerta room from a TICKscript

The following TICKscript sends the message, “Hey, check your CPU”, to Alerta whenever idle CPU usage drops below 10% using the .alerta() event handler and default Alerta settings defined in the kapacitor.conf.

alerta-cpu-alert.tick

stream
  |from()
    .measurement('cpu')
  |alert()
    .crit(lambda: "usage_idle" < 10)
    .stateChangesOnly()
    .message('Hey, check your CPU')
    .alerta()
      .resource('{{ .Name }}')
      .event('{{ .ID }}')
      .correlated('{{ .Name }}')

Send alerts to an Alerta room from a defined handler

The following setup sends an alert to the cpu topic with the message, “Hey, check your CPU”. An Alerta handler is added that subscribes to the cpu topic and publishes all alert messages to Alerta using default settings defined in the kapacitor.conf.

Create a TICKscript that publishes alert messages to a topic. The TICKscript below sends an alert message to the cpu topic any time idle CPU usage drops below 10%.

cpu_alert.tick

stream
  |from()
    .measurement('cpu')
  |alert()
    .crit(lambda: "usage_idle" < 10)
    .stateChangesOnly()
    .message('Hey, check your CPU')
    .topic('cpu')

Add and enable the TICKscript:

kapacitor define cpu_alert -tick cpu_alert.tick
kapacitor enable cpu_alert

Create a handler file that subscribes to the cpu topic and uses the Alerta event handler to send alerts to the alerts channel in Alerta.

alerta_cpu_handler.yaml

id: alerta-cpu-alert
topic: cpu
kind: alerta
options:
  resource: '{{ .Name }}'
  event: '{{ .ID }}'
  origin: 'kapacitor'
  correlate: ['service1', 'service2']

Add the handler:

kapacitor define-topic-handler alerta_cpu_handler.yaml