BatchNode

Constructor

Chaining Method Description
batch Has no constructor signature.

Property Methods

This node has no properties that can be set.

Chaining Methods

Deadman, Query, Stats



Description

A node that handles creating several child QueryNodes. Each call to query creates a child batch node that can further be configured. See QueryNode The batch variable in batch tasks is an instance of a BatchNode.

Example:

     var errors = batch
                      |query('SELECT value from errors')
                      ...
     var views = batch
                      |query('SELECT value from views')
                      ...

Available Statistics:

  • query_errors – number of errors when querying
  • batches_queried – number of batches returned from queries
  • points_queried – total number of points in batches

^

Chaining Methods

Chaining methods create a new node in the pipeline as a child of the calling node. They do not modify the calling node. Chaining methods are marked using the | operator.

Deadman

Helper function for creating an alert on low throughput, a.k.a. deadman’s switch.

  • Threshold – trigger alert if throughput drops below threshold in points/interval.
  • Interval – how often to check the throughput.
  • Expressions – optional list of expressions to also evaluate. Useful for time of day alerting.

Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    data
        |deadman(100.0, 10s)
    //Do normal processing of data
    data...

The above is equivalent to this Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    data
        |stats(10s)
            .align()
        |derivative('emitted')
            .unit(10s)
            .nonNegative()
        |alert()
            .id('node \'stream0\' in task \'{{ .TaskName }}\'')
            .message('{{ .ID }} is {{ if eq .Level "OK" }}alive{{ else }}dead{{ end }}: {{ index .Fields "emitted" | printf "%0.3f" }} points/10s.')
            .crit(lambda: "emitted" <= 100.0)
    //Do normal processing of data
    data...

The id and message alert properties can be configured globally via the ‘deadman’ configuration section.

Since the AlertNode is the last piece it can be further modified as usual. Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    data
        |deadman(100.0, 10s)
            .slack()
            .channel('#dead_tasks')
    //Do normal processing of data
    data...

You can specify additional lambda expressions to further constrain when the deadman’s switch is triggered. Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    // Only trigger the alert if the time of day is between 8am-5pm.
    data
        |deadman(100.0, 10s, lambda: hour("time") >= 8 AND hour("time") <= 17)
    //Do normal processing of data
    data...
batch|deadman(threshold float64, interval time.Duration, expr ...ast.LambdaNode)

Returns: AlertNode

^

Query

The query to execute. Must not contain a time condition in the WHERE clause or contain a GROUP BY clause. The time conditions are added dynamically according to the period, offset and schedule. The GROUP BY clause is added dynamically according to the dimensions passed to the groupBy method.

batch|query(q string)

Returns: QueryNode

^

Stats

Create a new stream of data that contains the internal statistics of the node. The interval represents how often to emit the statistics based on real time. This means the interval time is independent of the times of the data points the source node is receiving.

batch|stats(interval time.Duration)

Returns: StatsNode

^

This documentation is open source. See a typo? Please, open an issue.


Need help getting up and running? Get Support