Using Kapacitor template tasks

Kapacitor has a template system that lets you define a template and reuse it for multiple tasks. Each task can define its own value for various vars within the template. Templates can be consumed via the CLI and API.

The following is a simple example that defines a template that computes the mean of a field and triggers an alert.

// Which measurement to consume
var measurement string
// Optional where filter
var where_filter = lambda: TRUE
// Optional list of group by dimensions
var groups = [*]
// Which field to process
var field string
// Warning criteria, has access to 'mean' field
var warn lambda
// Critical criteria, has access to 'mean' field
var crit lambda
// How much data to window
var window = 5m
// The slack channel for alerts
var slack_channel = '#alerts'

stream
    |from()
        .measurement(measurement)
        .where(where_filter)
        .groupBy(groups)
    |window()
        .period(window)
        .every(window)
    |mean(field)
    |alert()
         .warn(warn)
         .crit(crit)
         .slack()
         .channel(slack_channel)

Notice how everything about the task is a var except for the structure of the task pipeline itself. This allows substantial flexibility in customization of the template.

To use this template, first define the template like so:

kapacitor define-template generic_mean_alert -tick path/to/above/script.tick -type stream

At this point you can run show-template to see more information about our template.

kapacitor show-template generic_mean_alert

You should see a list of vars for the template like this:

Vars:
Name             Type      Default Value   Description
crit             lambda    <required>      Critical criteria, has access to 'mean' field
field            string    <required>      Which field to process
groups           list      [*]             Optional list of group by dimensions
measurement      string    <required>      Which measurement to consume
slack_channel    string    #alerts         The Slack channel for alerts
warn             lambda    <required>      Warning criteria, has access to 'mean' field
where_filter     lambda    TRUE            Optional where filter
window           duration  5m0s            How much data to window

A task will gain its type and TICKscript properties from the template. The specific values of vars and set of database and retention policies for a task are unique per task.

Now you can define a task that uses the template to alert on cpu usage. Create a file cpu_vars.json with these contents.

{
    "measurement": {"type" : "string", "value" : "cpu" },
    "where_filter": {"type": "lambda", "value": "\"cpu\" == 'cpu-total'"},
    "groups": {"type": "list", "value": [{"type":"string", "value":"host"},{"type":"string", "value":"dc"}]},
    "field": {"type" : "string", "value" : "usage_idle" },
    "warn": {"type" : "lambda", "value" : "\"mean\" < 30.0" },
    "crit": {"type" : "lambda", "value" : "\"mean\" < 10.0" },
    "window": {"type" : "duration", "value" : "1m" },
    "slack_channel": {"type" : "string", "value" : "#alerts_testing" }
}

Now define the task using the vars for the task.

kapacitor define cpu_alert -template generic_mean_alert -vars cpu_vars.json -dbrp telegraf.autogen

The show command will display the vars associated with this task.

kapacitor show cpu_alert

Example output:

Vars:
Name                Type      Value
crit                lambda    "mean" < 10.0
field               string    usage_idle
groups              list      [host,dc]
measurement         string    cpu
slack_channel       string    #alerts_testing
warn                lambda    "mean" < 30.0
where_filter        lambda    "cpu" == 'cpu-total'
window              duration  1m0s

We can also create a task for a memory based alert, using the same template. Create a mem_vars.json and use this snippet.

{
    "measurement": {"type" : "string", "value" : "mem" },
    "groups": {"type": "list", "value": [{"type":"star", "value":"*"}]},
    "field": {"type" : "string", "value" : "used_percent" },
    "warn": {"type" : "lambda", "value" : "\"mean\" > 80.0" },
    "crit": {"type" : "lambda", "value" : "\"mean\" > 90.0" },
    "window": {"type" : "duration", "value" : "10m" },
    "slack_channel": {"type" : "string", "value" : "#alerts_testing" }
}

kapacitor define mem_alert -template generic_mean_alert -vars mem_vars.json -dbrp telegraf.autogen

Running show will display the vars associated with this task, which are unique to our mem_alert task.

kapacitor show mem_alert

And again the vars output:

Vars:
Name              Type      Value
crit              lambda    "mean" > 90.0
field             string    used_percent
groups            list      [*]
measurement       string    mem
slack_channel     string    #alerts_testing
warn              lambda    "mean" > 80.0
window            duration  10m0s

You can define any number of tasks that use the same template.

NOTE: Updates to the template will update all associated tasks and reload them if necessary.

Using var declarations

Vars work with normal tasks as well and can be used to overwrite any defaults in the script. Since at any point a TICKscript could come in handy as a template we recommend always using var declarations in your scripts. This way your normal tasks work and if you decide that you want to create another similar task its now trivial to define a template and then multiple tasks.