Definitions for terms specific to this guide are defined below:
Standalone event handler
“Standalone” event handlers (also known as “topic handlers”) are those added using an event handler file that subscribes to a specified alert topic. As alerts are published to that topic, the standalone event handler sends alert data to a given endpoint. How the handler works and to what endpoint data is sent depends on the particular type of handler used (defined by the
kind attribute in your handler file).
Inline event handler
“Inline” event handlers are those used in TICKscripts that send alert data directly from the running task.
Currently, Kapacitor Enterprise is only partially cluster-aware. For example, members in a Kapacitor cluster aren’t aware of tasks and TICKscripts running on other members in the cluster. Tasks need to be defined on each member on which you wish the task to run. However, “standalone” alert handlers only need to be defined on one member. Kapacitor will manage the replication of that handler across other members of the cluster.
Alert handlers in a cluster
Because members of a Kapacitor Enterprise cluster are not aware of tasks running on other members, alert data should not be sent directly from TICKscripts. Doing so will result in duplicate notifications as each member running the same task will send the same alert.
Instead, publish alert data to a topic and create a standalone alert handler that subscribes to the topic. Kapacitor Enterprise handles the replication of standalone handlers across the cluster. As multiple members publish alerts a topic, Kapacitor deduplicates the alerts so the standalone handler will not send duplicate alerts.
Add alerts to a Kapacitor cluster
To add alerts to your Kapacitor Enterprise cluster without worrying about duplicate alerts, do the following:
1. Publish alert data to a topic
In your TICKscript AlertNodes, publish alert
data to a topic using the
TICKscript - Publish alerts to a topic
|alert() // ... .topic('topic-name')
2. Add the task to nodes in your cluster
Since tasks are not replicated across members in you cluster, tasks need to be defined on each member. Login to each member you wish to run a given task and define a new task using your TICKscript:
# Pattern kapacitor define <task-name> -tick <path-to-tickscript-file> # Example kapacitor define cpu-alert -tick cpu_alert.tick
InfluxData recommends defining a given task on the number members set in the
redundancyconfiguration option of the
[alert]section of your
kapacitor.conf. For example, if you set
redundancyto 2, then each task that generates alerts should be defined on 2 members of the cluster.
3. Add an alert handler that subscribes to your topic
id: handler-id topic: topic-name kind: slack options: workspace: 'workspace.slack.com' channel: '#alerts' username: 'kapacitor'
Standalone handlers only need to be defined on one member of your cluster. Kapacitor Enterprise manages the replication of standalone handlers across the cluster. Login to one of the members in your cluster and define your alert handler:
# Pattern kapacitor define-topic-handler <path/to/handler.yml> # From inside a running Kapacitor instance kapacitor define-topic-handler ./slack-handler.yml