Flux Profiler package

The Flux Profiler package provides performance profiling tools for Flux queries and operations. Import the profiler package:

import "profiler"

Each enabled profiler appends a table to the output stream of tables containing data returned by the profiler.

Options

The Profiler package includes the following options:

enabledProfilers

Enable Flux profilers.

Data type: Array of strings

import "profiler"

option profiler.enabledProfilers = ["query", "operator"]

// Query to profile

Available profilers

• query
• operator

query

The query profiler provides statistics about the execution of an entire Flux script. When enabled, results returned by yield() include a table with the following columns:

• TotalDuration: total query duration in nanoseconds.
• CompileDuration: number of nanoseconds spent compiling the query.
• QueueDuration: number of nanoseconds spent queueing.
• RequeueDuration: number fo nanoseconds spent requeueing.
• PlanDuration: number of nanoseconds spent planning the query.
• ExecuteDuration: number of nanoseconds spent executing the query.
• Concurrency: number of goroutines allocated to process the query.
• MaxAllocated: maximum number of bytes the query allocated.
• TotalAllocated: total number of bytes the query allocated (includes memory that was freed and then used again).
• RuntimeErrors: error messages returned during query execution.
• flux/query-plan: Flux query plan.
• influxdb/scanned-values: value scanned by InfluxDB.
• influxdb/scanned-bytes: number of bytes scanned by InfluxDB.

operator

The operator profiler output statistics about each operation in a query. Operations executed in the storage tier return as a single operation. When the operator profile is enabled, results returned by yield() include a table with a row for each operation and the following columns:

• Type: operation type
• Label: operation name
• Count: total number of times the operation executed
• MinDuration: minimum duration of the operation in nanoseconds
• MaxDuration: maximum duration of the operation in nanoseconds
• DurationSum: total duration of all operation executions in nanoseconds
• MeanDuration: average duration of all operation executions in nanoseconds

Examples

• Measure query and operator performance for InfluxDB
• Measure query and operator performance of array.from()

Measure query and operator performance for InfluxDB

The example below uses the NOAA water sample data.

import "profiler"

option profiler.enabledProfilers = ["query", "operator"]

from(bucket: "noaa")
  |> range(start: 2019-08-17T00:00:00Z, stop: 2019-08-17T00:30:00Z)
  |> filter(fn: (r) =>
    r._measurement == "h2o_feet" and
    r._field == "water_level" and
    r.location == "coyote_creek"
  )
  |> map(fn: (r) => ({ r with
    _value: r._value * 12.0,
    _measurement: "h2o_inches"
  }))
  |> drop(columns: ["_start", "_stop"])

Output tables

Measure query and operator performance of array.from()

import "profiler"
import "array"

option profiler.enabledProfilers = ["query", "operator"]

arrData = [
  {_time: 2021-01-01T00:00:00Z, _field: "foo", _value: 1.2},
  {_time: 2021-01-02T00:00:00Z, _field: "foo", _value: 2.5},
  {_time: 2021-01-03T00:00:00Z, _field: "foo", _value: 2.6},
  {_time: 2021-01-04T00:00:00Z, _field: "foo", _value: 8.1},
  {_time: 2021-01-05T00:00:00Z, _field: "foo", _value: 4.9}
]

array.from(rows: arrData)

Output tables