Documentation

Flux Profiler package

The Flux Profiler package provides performance profiling tools for Flux queries and operations. Import the profiler package:

import "profiler"

Each enabled profiler appends a table to the output stream of tables containing data returned by the profiler.

Options

The Profiler package includes the following options:

enabledProfilers

Enable Flux profilers.

Data type: Array of strings

import "profiler"

option profiler.enabledProfilers = ["query", "operator"]

// Query to profile

Available profilers

query

The query profiler provides statistics about the execution of an entire Flux script. When enabled, results returned by yield() include a table with the following columns:

  • TotalDuration: total query duration in nanoseconds.
  • CompileDuration: number of nanoseconds spent compiling the query.
  • QueueDuration: number of nanoseconds spent queueing.
  • RequeueDuration: number fo nanoseconds spent requeueing.
  • PlanDuration: number of nanoseconds spent planning the query.
  • ExecuteDuration: number of nanoseconds spent executing the query.
  • Concurrency: number of goroutines allocated to process the query.
  • MaxAllocated: maximum number of bytes the query allocated.
  • TotalAllocated: total number of bytes the query allocated (includes memory that was freed and then used again).
  • RuntimeErrors: error messages returned during query execution.
  • flux/query-plan: Flux query plan.
  • influxdb/scanned-values: value scanned by InfluxDB.
  • influxdb/scanned-bytes: number of bytes scanned by InfluxDB.

operator

The operator profiler output statistics about each operation in a query. Operations executed in the storage tier return as a single operation. When the operator profile is enabled, results returned by yield() include a table with a row for each operation and the following columns:

  • Type: operation type
  • Label: operation name
  • Count: total number of times the operation executed
  • MinDuration: minimum duration of the operation in nanoseconds
  • MaxDuration: maximum duration of the operation in nanoseconds
  • DurationSum: total duration of all operation executions in nanoseconds
  • MeanDuration: average duration of all operation executions in nanoseconds

Examples

Measure query and operator performance for InfluxDB

The example below uses the NOAA water sample data.

import "profiler"

option profiler.enabledProfilers = ["query", "operator"]

from(bucket: "noaa")
  |> range(start: 2019-08-17T00:00:00Z, stop: 2019-08-17T00:30:00Z)
  |> filter(fn: (r) =>
    r._measurement == "h2o_feet" and
    r._field == "water_level" and
    r.location == "coyote_creek"
  )
  |> map(fn: (r) => ({ r with
    _value: r._value * 12.0,
    _measurement: "h2o_inches"
  }))
  |> drop(columns: ["_start", "_stop"])

Output tables

_time_measurementlocation_field_value
2019-08-17T00:00:00Zh2o_inchescoyote_creekwater_level97.44
2019-08-17T00:06:00Zh2o_inchescoyote_creekwater_level96.06
2019-08-17T00:12:00Zh2o_inchescoyote_creekwater_level94.64
2019-08-17T00:18:00Zh2o_inchescoyote_creekwater_level93.14
2019-08-17T00:24:00Zh2o_inchescoyote_creekwater_level91.62
_measurementTotalDurationCompileDurationQueueDurationPlanDurationRequeueDurationExecuteDurationConcurrencyMaxAllocatedTotalAllocatedRuntimeErrorsflux/query-planinfluxdb/scanned-valuesinfluxdb/scanned-bytes
profiler/query12464554725978174460011706028017280“digraph {…}”00
_measurementTypeLabelCountMinDurationMaxDurationDurationSumMeanDuration
profiler/operator*universe.schemaMutationTransformationdrop4114145141451414514145
profiler/operator*universe.mapTransformationmap31250831250831250831250831
profiler/operator*influxdb.readFilterSourcemerged_ReadRange5_filter21529282529282529282529282

Measure query and operator performance of array.from()

import "profiler"
import "array"

option profiler.enabledProfilers = ["query", "operator"]

arrData = [
  {_time: 2021-01-01T00:00:00Z, _field: "foo", _value: 1.2},
  {_time: 2021-01-02T00:00:00Z, _field: "foo", _value: 2.5},
  {_time: 2021-01-03T00:00:00Z, _field: "foo", _value: 2.6},
  {_time: 2021-01-04T00:00:00Z, _field: "foo", _value: 8.1},
  {_time: 2021-01-05T00:00:00Z, _field: "foo", _value: 4.9}
]

array.from(rows: arrData)

Output tables

_time_field_value
2021-01-01T00:00:00Zfoo1.2
2021-01-02T00:00:00Zfoo2.5
2021-01-03T00:00:00Zfoo2.6
2021-01-04T00:00:00Zfoo8.1
2021-01-05T00:00:00Zfoo4.9
_measurementTotalDurationCompileDurationQueueDurationPlanDurationRequeueDurationExecuteDurationConcurrencyMaxAllocatedTotalAllocatedRuntimeErrorsflux/query-plan
profiler/query67648775826621762100613643408320“digraph {…}”
_measurementTypeLabelCountMinDurationMaxDurationDurationSumMeanDuration
profiler/operator*array.tableSourcearray.from0156361563615636156361

Select your region

Upgrade to InfluxDB Cloud or InfluxDB 2.0!

InfluxDB Cloud and InfluxDB OSS 2.0 ready for production.