AlertNode

Constructor

Chaining Method Description
alert ( ) Create an alert node, which can trigger alerts.

Property Methods

Setters Description
alerta ( ) Send the alert to Alerta.
all ( ) Indicates an alert should trigger only if all points in a batch match the criteria. Does not apply to stream alerts.
crit ( value ast.LambdaNode) Filter expression for the CRITICAL alert level. An empty value indicates the level is invalid and is skipped.
critReset ( value ast.LambdaNode) Filter expression for reseting the CRITICAL alert level to lower level.
details ( value string) Template for constructing a detailed HTML message for the alert. The same template data is available as the AlertNode.Message property, in addition to a Message field that contains the rendered Message value.
durationField ( value string) Optional field key to add the alert duration to the data. The duration is always in units of nanoseconds.
email ( to ...string) Email the alert data.
exec ( executable stringargs ...string) Execute a command whenever an alert is triggered and pass the alert data over STDIN in JSON format.
flapping ( low float64high float64) Perform flap detection on the alerts. The method used is similar method to Nagios: https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/3/en/flapping.html
hipChat ( ) Send the alert to HipChat. For step-by-step instructions on setting up Kapacitor with HipChat, see the Event Handler Setup Guide (https://docs.influxdata.com//kapacitor/latest/guides/event-handler-setup/#hipchat-setup). To allow Kapacitor to post to HipChat, go to the URL https://www.hipchat.com/docs/apiv2 for information on how to get your room id and tokens.
history ( value int64) Number of previous states to remember when computing flapping levels and checking for state changes. Minimum value is 2 in order to keep track of current and previous states.
id ( value string) Template for constructing a unique ID for a given alert.
idField ( value string) Optional field key to add to the data, containing the alert ID as a string.
idTag ( value string) Optional tag key to use when tagging the data with the alert ID.
info ( value ast.LambdaNode) Filter expression for the INFO alert level. An empty value indicates the level is invalid and is skipped.
infoReset ( value ast.LambdaNode) Filter expression for reseting the INFO alert level to lower level.
levelField ( value string) Optional field key to add to the data, containing the alert level as a string.
levelTag ( value string) Optional tag key to use when tagging the data with the alert level.
log ( filepath string) Log JSON alert data to file. One event per line. Must specify the absolute path to the log file. It will be created if it does not exist. Example: stream
message ( value string) Template for constructing a meaningful message for the alert.
messageField ( value string) Optional field key to add to the data, containing the alert message.
mqtt ( topic string) Send alert to an MQTT broker
noRecoveries ( ) Do not send recovery alerts.
opsGenie ( ) Send alert to OpsGenie. To use OpsGenie alerting you must first enable the ‘Alert Ingestion API’ in the ‘Integrations’ section of OpsGenie. Then place the API key from the URL into the ‘opsgenie’ section of the Kapacitor configuration.
pagerDuty ( ) Send the alert to PagerDuty. To use PagerDuty alerting you must first follow the steps to enable a new ‘Generic API’ service.
post ( urls ...string) HTTP POST JSON alert data to a specified URL.
pushover ( ) Send the alert to Pushover. Register your application with Pushover at https://pushover.net/apps/build to get a Pushover token.
sensu ( ) Send the alert to Sensu.
slack ( ) Send the alert to Slack. To allow Kapacitor to post to Slack, go to the URL https://slack.com/services/new/incoming-webhook and create a new incoming webhook and place the generated URL in the ‘slack’ configuration section.
snmpTrap ( trapOid string) Send the alert using SNMP traps. To allow Kapacitor to post SNMP traps,
stateChangesOnly ( maxInterval ...time.Duration) Only sends events where the state changed. Each different alert level OK, INFO, WARNING, and CRITICAL are considered different states.
talk ( ) Send the alert to Talk. To use Talk alerting you must first follow the steps to create a new incoming webhook.
tcp ( address string) Send JSON alert data to a specified address over TCP.
telegram ( ) Send the alert to Telegram. For step-by-step instructions on setting up Kapacitor with Telegram, see the Event Handler Setup Guide (https://docs.influxdata.com//kapacitor/latest/guides/event-handler-setup/#telegram-setup). To allow Kapacitor to post to Telegram,
topic ( value string) Topic specifies the name of an alert topic to which, alerts will be published. Alert handlers can be configured per topic, see the API documentation.
victorOps ( ) Send alert to VictorOps. To use VictorOps alerting you must first enable the ‘Alert Ingestion API’ in the ‘Integrations’ section of VictorOps. Then place the API key from the URL into the ‘victorops’ section of the Kapacitor configuration.
warn ( value ast.LambdaNode) Filter expression for the WARNING alert level. An empty value indicates the level is invalid and is skipped.
warnReset ( value ast.LambdaNode) Filter expression for reseting the WARNING alert level to lower level.

Chaining Methods

Alert, Bottom, Combine, Count, CumulativeSum, Deadman, Default, Delete, Derivative, Difference, Distinct, Elapsed, Eval, First, Flatten, GroupBy, HoltWinters, HoltWintersWithFit, HttpOut, HttpPost, InfluxDBOut, Join, K8sAutoscale, KapacitorLoopback, Last, Max, Mean, Median, Min, Mode, MovingAverage, Percentile, Sample, Shift, Sideload, Spread, StateCount, StateDuration, Stats, Stddev, Sum, SwarmAutoscale, Top, Union, Where, Window



Description

An AlertNode can trigger an event of varying severity levels, and pass the event to alert handlers. The criteria for triggering an alert is specified via a lambda expression. See AlertNode.Info, AlertNode.Warn, and AlertNode.Crit below.

Different event handlers can be configured for each AlertNode. Some handlers like Email, HipChat, Sensu, Slack, OpsGenie, VictorOps, PagerDuty, Telegram and Talk have a configuration option ‘global’ that indicates that all alerts implicitly use the handler.

Available event handlers:

  • log – log alert data to file.
  • post – HTTP POST data to a specified URL.
  • tcp – Send data to a specified address via raw TCP.
  • email – Send and email with alert data.
  • exec – Execute a command passing alert data over STDIN.
  • HipChat – Post alert message to HipChat room.
  • Alerta – Post alert message to Alerta.
  • Sensu – Post alert message to Sensu client.
  • Slack – Post alert message to Slack channel.
  • SNMPTraps – Trigger SNMP traps.
  • OpsGenie – Send alert to OpsGenie.
  • VictorOps – Send alert to VictorOps.
  • PagerDuty – Send alert to PagerDuty.
  • Pushover – Send alert to Pushover.
  • Talk – Post alert message to Talk client.
  • Telegram – Post alert message to Telegram client.
  • MQTT – Post alert message to MQTT.

See below for more details on configuring each handler.

Each event that gets sent to a handler contains the following alert data:

  • ID – the ID of the alert, user defined.
  • Message – the alert message, user defined.
  • Details – the alert details, user defined HTML content.
  • Time – the time the alert occurred.
  • Duration – the duration of the alert in nanoseconds.
  • Level – one of OK, INFO, WARNING or CRITICAL.
  • Data – influxql.Result containing the data that triggered the alert.

Events are sent to handlers if the alert is in a state other than ‘OK’ or the alert just changed to the ‘OK’ state from a non ‘OK’ state (a.k.a. the alert recovered). Using the AlertNode.StateChangesOnly property events will only be sent to handlers if the alert changed state.

It is valid to configure multiple alert handlers, even with the same type.

Example:

   stream
           .groupBy('service')
       |alert()
           .id('kapacitor/{{ index .Tags "service" }}')
           .message('{{ .ID }} is {{ .Level }} value:{{ index .Fields "value" }}')
           .info(lambda: "value" > 10)
           .warn(lambda: "value" > 20)
           .crit(lambda: "value" > 30)
           .post("http://example.com/api/alert")
           .post("http://another.example.com/api/alert")
           .tcp("exampleendpoint.com:5678")
           .email('oncall@example.com')

Each expression maintains its own state. The order of execution for the expressions is not considered to be deterministic. For each point an expression may or may not be evaluated. If no expression is true, then the alert is considered to be in the OK state.

Kapacitor supports alert reset expressions. This way when an alert enters a state, it can only be lowered in severity if its reset expression evaluates to true.

Example:

   stream
       |from()
           .measurement('cpu')
           .where(lambda: "host" == 'serverA')
           .groupBy('host')
       |alert()
           .info(lambda: "value" > 60)
           .infoReset(lambda: "value" < 50)
           .warn(lambda: "value" > 70)
           .warnReset(lambda: "value" < 60)
           .crit(lambda: "value" > 80)
           .critReset(lambda: "value" < 70)

For example given the following values: 61 73 64 85 62 56 47 The corresponding alert states are: INFO WARNING WARNING CRITICAL INFO INFO OK

Available Statistics:

  • alerts_triggered – Total number of alerts triggered
  • oks_triggered – Number of OK alerts triggered
  • infos_triggered – Number of informational (.info) alerts triggered
  • warns_triggered – Number of warning (.warn) alerts triggered
  • crits_triggered – Number of critical (crit) alerts triggered

^

Properties

Property methods modify state on the calling node. They do not add another node to the pipeline, and always return a reference to the calling node. Property methods are marked using the . operator.

Alerta

Send the alert to Alerta.

Example:

    [alerta]
      enabled = true
      url = "https://alerta.yourdomain"
      token = "9hiWoDOZ9IbmHsOTeST123ABciWTIqXQVFDo63h9"
      environment = "Production"
      origin = "Kapacitor"

In order to not post a message every alert interval use AlertNode.StateChangesOnly so that only events where the alert changed state are sent to Alerta.

Send alerts to Alerta. The resource and event properties are required.

Example:

    stream
         |alert()
             .alerta()
                 .resource('Hostname or service')
                 .event('Something went wrong')

Alerta also accepts optional alert information.

Example:

    stream
         |alert()
             .alerta()
                 .resource('Hostname or service')
                 .event('Something went wrong')
                 .environment('Development')
                 .group('Dev. Servers')
                 .timeout(5m)

NOTE: Alerta cannot be configured globally because of its required properties.

alert.alerta()

^

Alerta Environment

Alerta environment. Can be a template and has access to the same data as the AlertNode.Details property. Default is set from the configuration.

alert.alerta()
      .environment(value string)

^

Alerta Event

Alerta event. Can be a template and has access to the same data as the idInfo property. Default: {{ .ID }}

alert.alerta()
      .event(value string)

^

Alerta Group

Alerta group. Can be a template and has access to the same data as the AlertNode.Details property. Default: {{ .Group }}

alert.alerta()
      .group(value string)

^

Alerta Origin

Alerta origin. If empty, uses the origin from the configuration.

alert.alerta()
      .origin(value string)

^

Alerta Resource

Alerta resource. Can be a template and has access to the same data as the AlertNode.Details property. Default: {{ .Name }}

alert.alerta()
      .resource(value string)

^

Alerta Services

List of effected services. If not specified defaults to the Name of the stream.

alert.alerta()
      .services(service ...string)

^

Alerta Timeout

Alerta timeout. Default: 24h

alert.alerta()
      .timeout(value time.Duration)

^

Alerta Token

Alerta authentication token. If empty, uses the token from the configuration.

alert.alerta()
      .token(value string)

^

Alerta Value

Alerta value. Can be a template and has access to the same data as the AlertNode.Details property. Default is an empty string.

alert.alerta()
      .value(value string)

^

All

Indicates an alert should trigger only if all points in a batch match the criteria. Does not apply to stream alerts.

alert.all()

^

Crit

Filter expression for the CRITICAL alert level. An empty value indicates the level is invalid and is skipped.

alert.crit(value ast.LambdaNode)

^

CritReset

Filter expression for reseting the CRITICAL alert level to lower level.

alert.critReset(value ast.LambdaNode)

^

Details

Template for constructing a detailed HTML message for the alert. The same template data is available as the AlertNode.Message property, in addition to a Message field that contains the rendered Message value.

The intent is that the Message property be a single line summary while the Details property is a more detailed message possibly spanning multiple lines, and containing HTML formatting.

This template is rendered using the html/template package in Go so that safe and valid HTML can be generated.

The json method is available within the template to convert any variable to a valid JSON string.

Example:

    |alert()
       .id('{{ .Name }}')
       .details('''
<h1>{{ .ID }}</h1>
<b>{{ .Message }}</b>
Value: {{ index .Fields "value" }}
''')
       .email()

Default: {{ json . }}

alert.details(value string)

^

DurationField

Optional field key to add the alert duration to the data. The duration is always in units of nanoseconds.

alert.durationField(value string)

^

Email

Email the alert data.

If the To list is empty, the To addresses from the configuration are used. The email subject is the AlertNode.Message property. The email body is the AlertNode.Details property. The emails are sent as HTML emails and so the body can contain html markup.

If the ‘smtp’ section in the configuration has the option: global = true then all alerts are sent via email without the need to explicitly state it in the TICKscript.

Example:

    |alert()
       .id('{{ .Name }}')
       // Email subject
       .message('{{ .ID }}:{{ .Level }}')
       //Email body as HTML
       .details('''
<h1>{{ .ID }}</h1>
<b>{{ .Message }}</b>
Value: {{ index .Fields "value" }}
''')
       .email()

Send an email with custom subject and body.

Example:

     [smtp]
       enabled = true
       host = "localhost"
       port = 25
       username = ""
       password = ""
       from = "kapacitor@example.com"
       to = ["oncall@example.com"]
       # Set global to true so all alert trigger emails.
       global = true
       state-changes-only =  true

Example:

    stream
         |alert()

Send email to ‘oncall@example.com’ from ‘kapacitor@example.com’

alert.email(to ...string)

^

Email To

Define the To addresses for the email alert. Multiple calls append to the existing list of addresses. If empty, uses the addresses from the configuration.

Example:

    |alert()
       .id('{{ .Name }}')
       // Email subject
       .message('{{ .ID }}:{{ .Level }}')
       //Email body as HTML
       .details('''
<h1>{{ .ID }}</h1>
<b>{{ .Message }}</b>
Value: {{ index .Fields "value" }}
''')
       .email('admin@example.com')
         .to('oncall@example.com')
         .to('support@example.com')

All three email addresses will receive the alert message.

Passing addresses to the email property directly or using the email.to property is the same.

alert.email(to ...string)
      .to(to ...string)

^

Exec

Execute a command whenever an alert is triggered and pass the alert data over STDIN in JSON format.

alert.exec(executable string, args ...string)

^

Flapping

Perform flap detection on the alerts. The method used is similar method to Nagios: https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/3/en/flapping.html

Each different alerting level is considered a different state. The low and high thresholds are inverted thresholds of a percentage of state changes. Meaning that if the percentage of state changes goes above the high threshold, the alert enters a flapping state. The alert remains in the flapping state until the percentage of state changes goes below the low threshold. Typical values are low: 0.25 and high: 0.5. The percentage values represent the number state changes over the total possible number of state changes. A percentage change of 0.5 means that the alert changed state in half of the recorded history, and remained the same in the other half of the history.

alert.flapping(low float64, high float64)

^

HipChat

Send the alert to HipChat. For step-by-step instructions on setting up Kapacitor with HipChat, see the Event Handler Setup Guide (https://docs.influxdata.com//kapacitor/latest/guides/event-handler-setup/#hipchat-setup). To allow Kapacitor to post to HipChat, go to the URL https://www.hipchat.com/docs/apiv2 for information on how to get your room id and tokens.

Example:

    [hipchat]
      enabled = true
      url = "https://orgname.hipchat.com/v2/room"
      room = "4189212"
      token = "9hiWoDOZ9IbmHsOTeST123ABciWTIqXQVFDo63h9"

In order to not post a message every alert interval use AlertNode.StateChangesOnly so that only events where the alert changed state are posted to the room.

Example:

    stream
         |alert()
             .hipChat()

Send alerts to HipChat room in the configuration file.

Example:

    stream
         |alert()
             .hipChat()
             .room('Kapacitor')

Send alerts to HipChat room ‘Kapacitor’

If the ‘hipchat’ section in the configuration has the option: global = true then all alerts are sent to HipChat without the need to explicitly state it in the TICKscript.

Example:

    [hipchat]
      enabled = true
      url = "https://orgname.hipchat.com/v2/room"
      room = "Test Room"
      token = "9hiWoDOZ9IbmHsOTeST123ABciWTIqXQVFDo63h9"
      global = true
      state-changes-only = true

Example:

    stream
         |alert()

Send alert to HipChat using default room ‘Test Room’.

alert.hipChat()

^

HipChat Room

HipChat room in which to post messages. If empty, uses the channel from the configuration.

alert.hipChat()
      .room(value string)

^

HipChat Token

HipChat authentication token. If empty, uses the token from the configuration.

alert.hipChat()
      .token(value string)

^

History

Number of previous states to remember when computing flapping levels and checking for state changes. Minimum value is 2 in order to keep track of current and previous states.

Default: 21

alert.history(value int64)

^

Id

Template for constructing a unique ID for a given alert.

Available template data:

  • Name – Measurement name.
  • TaskName – The name of the task
  • Group – Concatenation of all GROUP BY tags of the form [key=value,]+. If no .groupBy is performed, then it is equal to literal 'nil'.
  • Tags – Map of tags. Use '{{ index .Tags "key" }}' to get a specific tag value.
  • ServerInfo – Information about the running server. Available nested fields are: Hostname, ClusterID, and ServerID.

Example:

   stream
       |from()
           .measurement('cpu')
           .groupBy('cpu')
       |alert()
           .id('kapacitor/{{ .Name }}/{{ .Group }}')

ID: kapacitor/cpu/cpu=cpu0,

Example:

   stream
       |from()
           .measurement('cpu')
           .groupBy('service')
       |alert()
           .id('kapacitor/{{ index .Tags "service" }}')

ID: kapacitor/authentication

Example:

   stream
       |from()
           .measurement('cpu')
           .groupBy('service', 'host')
       |alert()
           .id('kapacitor/{{ index .Tags "service" }}/{{ index .Tags "host" }}')

ID: kapacitor/authentication/auth001.example.com

Default: {{ .Name }}:{{ .Group }}

alert.id(value string)

^

IdField

Optional field key to add to the data, containing the alert ID as a string.

alert.idField(value string)

^

IdTag

Optional tag key to use when tagging the data with the alert ID.

alert.idTag(value string)

^

Info

Filter expression for the INFO alert level. An empty value indicates the level is invalid and is skipped.

alert.info(value ast.LambdaNode)

^

InfoReset

Filter expression for reseting the INFO alert level to lower level.

alert.infoReset(value ast.LambdaNode)

^

LevelField

Optional field key to add to the data, containing the alert level as a string.

alert.levelField(value string)

^

LevelTag

Optional tag key to use when tagging the data with the alert level.

alert.levelTag(value string)

^

Log

Log JSON alert data to file. One event per line. Must specify the absolute path to the log file. It will be created if it does not exist. Example:

    stream
         |alert()
             .log('/tmp/alert')

Example:

    stream
         |alert()
             .log('/tmp/alert')
             .mode(0644)
alert.log(filepath string)

^

Log Mode

File’s mode and permissions, default is 0600 NOTE: The leading 0 is required to interpret the value as an octal integer.

alert.log(filepath string)
      .mode(value int64)

^

Message

Template for constructing a meaningful message for the alert.

Available template data:

  • ID – The ID of the alert.
  • Name – Measurement name.
  • TaskName – The name of the task
  • Group – Concatenation of all GROUP BY tags of the form [key=value,]+. If no groupBy is performed equal to literal ‘nil’.
  • Tags – Map of tags. Use {{ index .Tags "key" }} to get a specific tag value.
  • Level – Alert level. Valid values are: INFO, WARNING, CRITICAL.
  • Fields – Map of fields. Use {{ index .Fields "key" }} to get a specific field value.
  • Time – The time of the point that triggered the event.
    • Duration – The duration of the alert. The duration is printed in user-friendly format (e.g., 1000000000ns is printed as 1s).

Example:

   stream
       |from()
           .measurement('cpu')
           .groupBy('service', 'host')
       |alert()
           .id('{{ index .Tags "service" }}/{{ index .Tags "host" }}')
           .message('{{ .ID }} is {{ .Level}} value: {{ index .Fields "value" }}')

Message: authentication/auth001.example.com is CRITICAL value:42

Default: {{ .ID }} is {{ .Level }}

alert.message(value string)

^

MessageField

Optional field key to add to the data, containing the alert message.

alert.messageField(value string)

^

Mqtt

Send alert to an MQTT broker.

alert.mqtt(topic string)

^

Mqtt BrokerName

BrokerName is the name of the configured MQTT broker to use when publishing the alert. If empty, this defaults to the configured default broker.

alert.mqtt(topic string)
      .brokerName(value string)

^

Mqtt Qos

The Qos that will be used to deliver the alerts.

Valid values are:

  • 0 - At most once delivery
  • 1 - At least once delivery
  • 2 - Exactly once delivery
alert.mqtt(topic string)
      .qos(value int64)

^

Mqtt Retained

Retained indicates whether this alert should be delivered to clients that were not connected to the broker at the time of the alert.

alert.mqtt(topic string)
      .retained(value bool)

^

Mqtt Topic

The MQTT topic where alerts will be dispatched to.

alert.mqtt(topic string)
      .topic(value string)

^

NoRecoveries

Do not send recovery alerts.

alert.noRecoveries()

^

OpsGenie

Send alert to OpsGenie. To use OpsGenie alerting you must first enable the ‘Alert Ingestion API’ in the ‘Integrations’ section of OpsGenie. Then place the API key from the URL into the [opsgenie]` section of the Kapacitor configuration.

Example:

    [opsgenie]
      enabled = true
      api-key = "xxxxx"
      teams = ["everyone"]
      recipients = ["jim", "bob"]

With the correct configuration you can now use OpsGenie in TICKscripts.

Example:

    stream
         |alert()
             .opsGenie()

Send alerts to OpsGenie using the teams and recipients in the configuration file.

Example:

    stream
         |alert()
             .opsGenie()
             .teams('team_rocket','team_test')

Send alerts to OpsGenie with team set to ‘team_rocket’ and ‘team_test’.

If the [opsgenie] section in the configuration has the option global = true, then all alerts are sent to OpsGenie without the need to explicitly state it in the TICKscript.

Example:

    [opsgenie]
      enabled = true
      api-key = "xxxxx"
      recipients = ["johndoe"]
      global = true

Example:

    stream
         |alert()

Send alert to OpsGenie using the default recipients, found in the configuration.

alert.opsGenie()

^

OpsGenie Recipients

The list of recipients to be alerted. If empty defaults to the recipients from the configuration.

alert.opsGenie()
      .recipients(recipients ...string)

^

OpsGenie Teams

The list of teams to be alerted. If empty defaults to the teams from the configuration.

alert.opsGenie()
      .teams(teams ...string)

^

PagerDuty

Send the alert to PagerDuty. To use PagerDuty alerting you must first follow the steps to enable a new ‘Generic API’ service.

From https://developer.pagerduty.com/documentation/integration/events

  1. In your account, under the Services tab, click “Add New Service”.
  2. Enter a name for the service and select an escalation policy. Then, select “Generic API” for the Service Type.
  3. Click the “Add Service” button.
  4. Once the service is created, you’ll be taken to the service page. On this page, you’ll see the “Service key”, which is needed to access the API

Place the ‘service key’ into the ‘pagerduty’ section of the Kapacitor configuration as the option ‘service-key’.

Example:

    [pagerduty]
      enabled = true
      service-key = "xxxxxxxxx"

With the correct configuration you can now use PagerDuty in TICKscripts.

Example:

    stream
         |alert()
             .pagerDuty()

If the [pagerduty] section in the configuration has the option global = true, then all alerts are sent to PagerDuty without the need to explicitly state it in the TICKscript.

Example:

    [pagerduty]
      enabled = true
      service-key = "xxxxxxxxx"
      global = true

Example:

    stream
         |alert()

Send alert to PagerDuty.

alert.pagerDuty()

^

PagerDuty ServiceKey

The service key to use for the alert. Defaults to the value in the configuration if empty.

alert.pagerDuty()
      .serviceKey(value string)

^

Post

Send JSON alert data to a specified URL using HTTP POST method.

Example:

    stream
         |alert()
             .post()
                 .endpoint('example')

Example:

    stream
         |alert()
             .post('http://example.com')
alert.post(urls ...string)

^

Post CaptureResponse

CaptureResponse indicates that the HTTP response should be read and logged if the status code was not an 2xx code.

alert.post(urls ...string)
      .captureResponse()

^

Post Endpoint

Name of the endpoint to be used, as is defined in the configuration file.

alert.post(urls ...string)
      .endpoint(value string)

^

Post Header

Set a header key and value on the post request. Setting the Authenticate header is not allowed from within TICKscripts – use the configuration file to specify sensitive headers.

Example:

    stream
         |alert()
             .post()
                 .endpoint('example')
                 .header('a','b')
alert.post(urls ...string)
      .header(k string, v string)

^

Post Timeout

Timeout for HTTP Post

alert.post(urls ...string)
      .timeout(value time.Duration)

^

Pushover

Send the alert to Pushover. Register your application with Pushover at https://pushover.net/apps/build to get a Pushover token.

Alert Level Mapping: OK - Sends a -2 priority level. Info - Sends a -1 priority level. Warning - Sends a 0 priority level. Critical - Sends a 1 priority level.

Example:

    [pushover]
      enabled = true
      token = "9hiWoDOZ9IbmHsOTeST123ABciWTIqXQVFDo63h9"
      user_key = "Pushover"

Example:

    stream
         |alert()
             .pushover()
              .sound('siren')
              .user_key('other user')
              .device('mydev')
              .title('mytitle')
              .URL('myurl')
              .URLTitle('mytitle')

Send alerts to Pushover.

alert.pushover()

^

Pushover Device

Users device name to send message directly to that device, rather than all of a user’s devices (multiple device names may be separated by a comma)

alert.pushover()
      .device(value string)

^

Pushover Sound

The name of one of the sounds supported by the device clients to override the user’s default sound choice.

alert.pushover()
      .sound(value string)

^

Pushover Title

Your message’s title, otherwise your apps name is used.

alert.pushover()
      .title(value string)

^

Pushover URL

A supplementary URL to show with your message.

alert.pushover()
      .uRL(value string)

^

Pushover URLTitle

A title for your supplementary URL, otherwise just URL is shown.

alert.pushover()
      .uRLTitle(value string)

^

Pushover UserKey

User/Group key of your user (or you), viewable when logged into the Pushover dashboard. Often referred to as USER_KEY in the Pushover documentation. If empty, uses the user from the configuration.

alert.pushover()
      .userKey(value string)

^

Sensu

Send the alert to Sensu.

Example:

    [sensu]
      enabled = true
      addr = "10.56.4.205:3030"
      source = "Kapacitor"
      handlers = ["sns","slack"]

Example:

    stream
         |alert()
             .sensu()

Send alerts to Sensu client.

Example:

    stream
         |alert()
             .sensu()
             .handlers('sns','slack')

Send alerts to Sensu specifying the handlers

alert.sensu()

^

Sensu Handlers

List of effected services. If not specified defaults to the Name of the stream.

alert.sensu()
      .handlers(handlers ...string)

^

Sensu Source

Sensu source in which to post messages. If empty, uses the Source from the configuration.

alert.sensu()
      .source(value string)

^

Slack

Send the alert to Slack. To allow Kapacitor to post to Slack, go to the URL https://slack.com/services/new/incoming-webhook and create a new incoming webhook and place the generated URL in the ‘slack’ configuration section.

Example:

    [slack]
      enabled = true
      url = "https://hooks.slack.com/services/xxxxxxxxx/xxxxxxxxx/xxxxxxxxxxxxxxxxxxxxxxxx"
      channel = "#general"

In order to not post a message every alert interval use AlertNode.StateChangesOnly so that only events where the alert changed state are posted to the channel.

Example:

    stream
         |alert()
             .slack()

Send alerts to Slack channel in the configuration file.

Example:

    stream
         |alert()
             .slack()
             .channel('#alerts')

Send alerts to Slack channel ‘#alerts’

Example:

    stream
         |alert()
             .slack()
             .channel('@jsmith')

Send alert to user ‘@jsmith’

If the ‘slack’ section in the configuration has the option: global = true then all alerts are sent to Slack without the need to explicitly state it in the TICKscript.

Example:

    [slack]
      enabled = true
      url = "https://hooks.slack.com/services/xxxxxxxxx/xxxxxxxxx/xxxxxxxxxxxxxxxxxxxxxxxx"
      channel = "#general"
      global = true
      state-changes-only = true

Example:

    stream
         |alert()

Send alert to Slack using default channel ‘#general’.

alert.slack()

^

Slack Channel

Slack channel in which to post messages. If empty, uses the Channel from the configuration.

alert.slack()
      .channel(value string)

^

Slack IconEmoji

IconEmoji is an emoji name surrounded in ‘:’ characters. The emoji image will replace the normal user icon for the slack bot.

alert.slack()
      .iconEmoji(value string)

^

Slack Username

Username of the Slack bot. If empty, uses the username from the configuration.

alert.slack()
      .username(value string)

^

SnmpTrap

Send the alert using SNMP traps. To allow Kapacitor to post SNMP traps,

Example:

    [snmptrap]
      enabled = true
      addr = "127.0.0.1:9162"
      community = "public"

Example:

    stream
         |alert()
             .snmpTrap('1.1.1.1')
                 .data('1.3.6.1.2.1.1.7', 'i', '{{ index .Field "value" }}')

Send alerts to target-ip:target-port on OID ‘1.3.6.1.2.1.1.7’

alert.snmpTrap(trapOid string)

^

SnmpTrap Data

Define Data for SNMP Trap alert. Multiple calls append to the existing list of data.

Available types:

Abbreviation Datatype
c Counter
i Integer
n Null
s String
t Time ticks

Example:

    |alert()
       .message('{{ .ID }}:{{ .Level }}')
       .snmpTrap('1.3.6.1.4.1.1')
          .data('1.3.6.1.4.1.1.5', 's', '{{ .Level }}' )
          .data('1.3.6.1.4.1.1.6', 'i', '50' )
          .data('1.3.6.1.4.1.1.7', 'c', '{{ index .Fields "num_requests" }}' )
          .data('1.3.6.1.4.1.1.8', 's', '{{ .Message }}' )
alert.snmpTrap(trapOid string)
      .data(oid string, typ string, value string)

^

StateChangesOnly

Only sends events where the state changed. Each different alert level OK, INFO, WARNING, and CRITICAL are considered different states.

Example:

   stream
       |from()
           .measurement('cpu')
       |window()
            .period(10s)
            .every(10s)
       |alert()
           .crit(lambda: "value" > 10)
           .stateChangesOnly()
           .slack()

If the “value” is greater than 10 for a total of 60s, then only two events will be sent. First, when the value crosses the threshold, and second, when it falls back into an OK state. Without stateChangesOnly, the alert would have triggered 7 times: 6 times for each 10s period where the condition was met and once more for the recovery.

An optional maximum interval duration can be provided. An event will not be ignore (aka trigger an alert) if more than the maximum interval has elapsed since the last alert.

Example:

   stream
       |from()
           .measurement('cpu')
       |window()
            .period(10s)
            .every(10s)
       |alert()
           .crit(lambda: "value" > 10)
           .stateChangesOnly(10m)
           .slack()

The above usage will only trigger alerts to slack on state changes or at least every 10 minutes.

alert.stateChangesOnly(maxInterval ...time.Duration)

^

Talk

Send the alert to Talk. To use Talk alerting you must first follow the steps to create a new incoming webhook.

  1. Go to the URL https:/account.jianliao.com/signin.
  2. Sign in with you account. under the Team tab, click “Integrations”.
  3. Select “Customize service”, click incoming Webhook “Add” button.
  4. After choose the topic to connect with “xxx”, click “Confirm Add” button.
  5. Once the service is created, you’ll see the “Generate Webhook url”.

Place the ‘Generate Webhook url’ into the ‘Talk’ section of the Kapacitor configuration as the option ‘url’.

Example:

    [talk]
      enabled = true
      url = "https://jianliao.com/v2/services/webhook/uuid"
      author_name = "Kapacitor"

Example:

    stream
         |alert()
             .talk()

Send alerts to Talk client.

alert.talk()

^

Tcp

Send JSON alert data to a specified address over TCP.

alert.tcp(address string)

^

Tcp Address

The endpoint address.

alert.tcp(address string)
      .address(value string)

^

Telegram

Send the alert to Telegram. For step-by-step instructions on setting up Kapacitor with Telegram, see the Event Handler Setup Guide (https://docs.influxdata.com//kapacitor/latest/guides/event-handler-setup/#telegram-setup). To allow Kapacitor to post to Telegram,

Example:

    [telegram]
      enabled = true
      token = "123456789:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      chat-id = "xxxxxxxxx"
      parse-mode = "Markdown"
	disable-web-page-preview = true
	disable-notification = false

In order to not post a message every alert interval use AlertNode.StateChangesOnly so that only events where the alert changed state are posted to the chat-id.

Example:

    stream
         |alert()
             .telegram()

Send alerts to Telegram chat-id in the configuration file.

Example:

    stream
         |alert()
             .telegram()
             .chatId('xxxxxxx')

Send alerts to Telegram user/group ‘xxxxxx’

If the ‘telegram’ section in the configuration has the option: global = true then all alerts are sent to Telegram without the need to explicitly state it in the TICKscript.

Example:

    [telegram]
      enabled = true
      token = "123456789:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      chat-id = "xxxxxxxxx"
      global = true
      state-changes-only = true

Example:

    stream
         |alert()

Send alert to Telegram using default chat-id ‘xxxxxxxx’.

alert.telegram()

^

Telegram ChatId

Telegram user/group ID to post messages to. If empty, uses chati-d from the configuration.

alert.telegram()
      .chatId(value string)

^

Telegram DisableNotification

Disables the Notification. If empty, defaults to the configuration.

alert.telegram()
      .disableNotification()

^

Telegram DisableWebPagePreview

Disables the WebPagePreview. If empty, defaults to the configuration.

alert.telegram()
      .disableWebPagePreview()

^

Telegram ParseMode

Parse node, defaults to Markdown. If empty, uses the parse-mode from the configuration.

alert.telegram()
      .parseMode(value string)

^

Topic

Topic specifies the name of an alert topic to which alerts will be published. Alert handlers can be configured per topic – see the API documentation.

alert.topic(value string)

^

VictorOps

Send alert to VictorOps. To use VictorOps alerting you must first enable the ‘Alert Ingestion API’ in the ‘Integrations’ section of VictorOps. Then place the API key from the URL into the ‘victorops’ section of the Kapacitor configuration.

Example:

    [victorops]
      enabled = true
      api-key = "xxxxx"
      routing-key = "everyone"

With the correct configuration you can now use VictorOps in TICKscripts.

Example:

    stream
         |alert()
             .victorOps()

Send alerts to VictorOps using the routing key (routing-key)) specified in the configuration file.

Example:

    stream
         |alert()
             .victorOps()
             .routingKey('team_rocket')

Send alerts to VictorOps with routing key ‘team_rocket’

If the VictorOps section (victorops) in the configuration file has the option global = true, then all alerts are sent to VictorOps without the need to explicitly state it in the TICKscript.

Example:

    [victorops]
      enabled = true
      api-key = "xxxxx"
      routing-key = "everyone"
      global = true

Example:

    stream
         |alert()

Send alert to VictorOps using the default routing key, found in the configuration.

alert.victorOps()

^

VictorOps RoutingKey

The routing key to use for the alert. If empty, defaults to the value in the configuration.

alert.victorOps()
      .routingKey(value string)

^

Warn

Filter expression for the WARNING alert level. An empty value indicates the level is invalid and is skipped.

alert.warn(value ast.LambdaNode)

^

WarnReset

Filter expression for resetting the WARNING alert level to lower level.

alert.warnReset(value ast.LambdaNode)

^

Chaining Methods

Chaining methods create a new node in the pipeline as a child of the calling node. They do not modify the calling node. Chaining methods are marked using the | operator.

Alert

Create an alert node which can trigger alerts.

alert|alert()

Returns: AlertNode

^

Bottom

Select the bottom num points for field and sort by any extra tags or fields.

alert|bottom(num int64, field string, fieldsAndTags ...string)

Returns: InfluxQLNode

^

Combine

Combine this node with itself. The data are combined on timestamp.

alert|combine(expressions ...ast.LambdaNode)

Returns: CombineNode

^

Count

Count the number of points.

alert|count(field string)

Returns: InfluxQLNode

^

CumulativeSum

Compute a cumulative sum of each point that is received. A point is emitted for every point collected.

alert|cumulativeSum(field string)

Returns: InfluxQLNode

^

Deadman

Helper function for creating an alert on low throughput, a.k.a. deadman’s switch.

  • Threshold: Trigger alert if throughput drops below threshold in points/interval.
  • Interval: How often to check the throughput.
  • Expressions: Optional list of expressions to also evaluate. Useful for time of day alerting.

Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    data
        |deadman(100.0, 10s)
    //Do normal processing of data
    data...

The above is equivalent to this.

Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    data
        |stats(10s)
            .align()
        |derivative('emitted')
            .unit(10s)
            .nonNegative()
        |alert()
            .id('node \'stream0\' in task \'{{ .TaskName }}\'')
            .message('{{ .ID }} is {{ if eq .Level "OK" }}alive{{ else }}dead{{ end }}: {{ index .Fields "emitted" | printf "%0.3f" }} points/10s.')
            .crit(lambda: "emitted" <= 100.0)
    //Do normal processing of data
    data...

The id and message alert properties can be configured globally via the ‘deadman’ configuration section.

Since the AlertNode is the last piece it can be further modified as usual. Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    data
        |deadman(100.0, 10s)
            .slack()
            .channel('#dead_tasks')
    //Do normal processing of data
    data...

You can specify additional lambda expressions to further constrain when the deadman’s switch is triggered. Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    // Only trigger the alert if the time of day is between 8am-5pm.
    data
        |deadman(100.0, 10s, lambda: hour("time") >= 8 AND hour("time") <= 17)
    //Do normal processing of data
    data...
alert|deadman(threshold float64, interval time.Duration, expr ...ast.LambdaNode)

Returns: AlertNode

^

Default

Create a node that can set defaults for missing tags or fields.

alert|default()

Returns: DefaultNode

^

Delete

Create a node that can delete tags or fields.

alert|delete()

Returns: DeleteNode

^

Derivative

Create a new node that computes the derivative of adjacent points.

alert|derivative(field string)

Returns: DerivativeNode

^

Difference

Compute the difference between points independent of elapsed time.

alert|difference(field string)

Returns: InfluxQLNode

^

Distinct

Produce batch of only the distinct points.

alert|distinct(field string)

Returns: InfluxQLNode

^

Elapsed

Compute the elapsed time between points

alert|elapsed(field string, unit time.Duration)

Returns: InfluxQLNode

^

Eval

Create an eval node that will evaluate the given transformation function to each data point. A list of expressions may be provided and will be evaluated in the order they are given. The results are available to later expressions.

alert|eval(expressions ...ast.LambdaNode)

Returns: EvalNode

^

First

Select the first point.

alert|first(field string)

Returns: InfluxQLNode

^

Flatten

Flatten points with similar times into a single point.

alert|flatten()

Returns: FlattenNode

^

GroupBy

Group the data by a set of tags.

Can pass literal * to group by all dimensions. Example:

    |groupBy(*)
alert|groupBy(tag ...interface{})

Returns: GroupByNode

^

HoltWinters

Compute the holt-winters (https://docs.influxdata.com/influxdb/latest/query_language/functions/#holt-winters) forecast of a data set.

alert|holtWinters(field string, h int64, m int64, interval time.Duration)

Returns: InfluxQLNode

^

HoltWintersWithFit

Compute the holt-winters (https://docs.influxdata.com/influxdb/latest/query_language/functions/#holt-winters) forecast of a data set. This method also outputs all the points used to fit the data in addition to the forecasted data.

alert|holtWintersWithFit(field string, h int64, m int64, interval time.Duration)

Returns: InfluxQLNode

^

HttpOut

Create an HTTP output node that caches the most recent data it has received. The cached data are available at the given endpoint. The endpoint is the relative path from the API endpoint of the running task. For example, if the task endpoint is at /kapacitor/v1/tasks/<task_id> and endpoint is top10, then the data can be requested from /kapacitor/v1/tasks/<task_id>/top10.

alert|httpOut(endpoint string)

Returns: HTTPOutNode

^

HttpPost

Creates an HTTP Post node that POSTS received data to the provided HTTP endpoint. HttpPost expects 0 or 1 arguments. If 0 arguments are provided, you must specify an endpoint property method.

alert|httpPost(url ...string)

Returns: HTTPPostNode

^

InfluxDBOut

Create an influxdb output node that will store the incoming data into InfluxDB.

alert|influxDBOut()

Returns: InfluxDBOutNode

^

Join

Join this node with other nodes. The data are joined on timestamp.

alert|join(others ...Node)

Returns: JoinNode

^

K8sAutoscale

Create a node that can trigger autoscale events for a kubernetes cluster.

alert|k8sAutoscale()

Returns: K8sAutoscaleNode

^

KapacitorLoopback

Create an kapacitor loopback node that will send data back into Kapacitor as a stream.

alert|kapacitorLoopback()

Returns: KapacitorLoopbackNode

^

Last

Select the last point.

alert|last(field string)

Returns: InfluxQLNode

^

Max

Select the maximum point.

alert|max(field string)

Returns: InfluxQLNode

^

Mean

Compute the mean of the data.

alert|mean(field string)

Returns: InfluxQLNode

^

Median

Compute the median of the data. Note, this method is not a selector, if you want the median point use .percentile(field, 50.0).

alert|median(field string)

Returns: InfluxQLNode

^

Min

Select the minimum point.

alert|min(field string)

Returns: InfluxQLNode

^

Mode

Compute the mode of the data.

alert|mode(field string)

Returns: InfluxQLNode

^

MovingAverage

Compute a moving average of the last window points. No points are emitted until the window is full.

alert|movingAverage(field string, window int64)

Returns: InfluxQLNode

^

Percentile

Select a point at the given percentile. This is a selector function, no interpolation between points is performed.

alert|percentile(field string, percentile float64)

Returns: InfluxQLNode

^

Sample

Create a new node that samples the incoming points or batches.

One point will be emitted every count or duration specified.

alert|sample(rate interface{})

Returns: SampleNode

^

Shift

Create a new node that shifts the incoming points or batches in time.

alert|shift(shift time.Duration)

Returns: ShiftNode

^

Sideload

Create a node that can load data from external sources

alert|sideload()

Returns: SideloadNode

^

Spread

Compute the difference between min and max points.

alert|spread(field string)

Returns: InfluxQLNode

^

StateCount

Create a node that tracks number of consecutive points in a given state.

alert|stateCount(expression ast.LambdaNode)

Returns: StateCountNode

^

StateDuration

Create a node that tracks duration in a given state.

alert|stateDuration(expression ast.LambdaNode)

Returns: StateDurationNode

^

Stats

Create a new stream of data that contains the internal statistics of the node. The interval represents how often to emit the statistics based on real time. This means the interval time is independent of the times of the data points the source node is receiving.

alert|stats(interval time.Duration)

Returns: StatsNode

^

Stddev

Compute the standard deviation.

alert|stddev(field string)

Returns: InfluxQLNode

^

Sum

Compute the sum of all values.

alert|sum(field string)

Returns: InfluxQLNode

^

SwarmAutoscale

Create a node that can trigger autoscale events for a docker swarm cluster.

alert|swarmAutoscale()

Returns: SwarmAutoscaleNode

^

Top

Select the top num points for field and sort by any extra tags or fields.

alert|top(num int64, field string, fieldsAndTags ...string)

Returns: InfluxQLNode

^

Union

Perform the union of this node and all other given nodes.

alert|union(node ...Node)

Returns: UnionNode

^

Where

Create a new node that filters the data stream by a given expression.

alert|where(expression ast.LambdaNode)

Returns: WhereNode

^

Window

Create a new node that windows the stream by time.

NOTE: Window can only be applied to stream edges.

alert|window()

Returns: WindowNode

^

This documentation is open source. See a typo? Please, open an issue.


Need help getting up and running? Get Support