Documentation

Flux Profiler package

The Flux Profiler package provides performance profiling tools for Flux queries and operations. Import the profiler package:

import "profiler"

Each enabled profiler appends a table to the output stream of tables containing data returned by the profiler.

Options

The Profiler package includes the following options:

enabledProfilers

Enable Flux profilers.

Data type: Array of strings

import "profiler"

option profiler.enabledProfilers = ["query", "operator"]

// Query to profile

Available profilers

query

The query profiler provides statistics about the execution of an entire Flux script. When enabled, results returned by yield() include a table with the following columns:

  • TotalDuration: total query duration in nanoseconds.
  • CompileDuration: number of nanoseconds spent compiling the query.
  • QueueDuration: number of nanoseconds spent queueing.
  • RequeueDuration: number fo nanoseconds spent requeueing.
  • PlanDuration: number of nanoseconds spent planning the query.
  • ExecuteDuration: number of nanoseconds spent executing the query.
  • Concurrency: number of goroutines allocated to process the query.
  • MaxAllocated: maximum number of bytes the query allocated.
  • TotalAllocated: total number of bytes the query allocated (includes memory that was freed and then used again).
  • RuntimeErrors: error messages returned during query execution.
  • flux/query-plan: Flux query plan.
  • influxdb/scanned-values: value scanned by InfluxDB.
  • influxdb/scanned-bytes: number of bytes scanned by InfluxDB.

operator

The operator profiler output statistics about each operation in a query. Operations executed in the storage tier return as a single operation. When the operator profile is enabled, results returned by yield() include a table with a row for each operation and the following columns:

  • Type: operation type
  • Label: operation name
  • Count: total number of times the operation executed
  • MinDuration: minimum duration of the operation in nanoseconds
  • MaxDuration: maximum duration of the operation in nanoseconds
  • DurationSum: total duration of all operation executions in nanoseconds
  • MeanDuration: average duration of all operation executions in nanoseconds

Examples

Measure query and operator performance for InfluxDB

The example below uses the NOAA water sample data.

import "profiler"

option profiler.enabledProfilers = ["query", "operator"]

from(bucket: "noaa")
  |> range(start: 2019-08-17T00:00:00Z, stop: 2019-08-17T00:30:00Z)
  |> filter(fn: (r) =>
    r._measurement == "h2o_feet" and
    r._field == "water_level" and
    r.location == "coyote_creek"
  )
  |> map(fn: (r) => ({ r with
    _value: r._value * 12.0,
    _measurement: "h2o_inches"
  }))
  |> drop(columns: ["_start", "_stop"])

Output tables

_time_measurementlocation_field_value
2019-08-17T00:00:00Zh2o_inchescoyote_creekwater_level97.44
2019-08-17T00:06:00Zh2o_inchescoyote_creekwater_level96.06
2019-08-17T00:12:00Zh2o_inchescoyote_creekwater_level94.64
2019-08-17T00:18:00Zh2o_inchescoyote_creekwater_level93.14
2019-08-17T00:24:00Zh2o_inchescoyote_creekwater_level91.62
_measurementTotalDurationCompileDurationQueueDurationPlanDurationRequeueDurationExecuteDurationConcurrencyMaxAllocatedTotalAllocatedRuntimeErrorsflux/query-planinfluxdb/scanned-valuesinfluxdb/scanned-bytes
profiler/query12464554725978174460011706028017280“digraph {…}”00
_measurementTypeLabelCountMinDurationMaxDurationDurationSumMeanDuration
profiler/operator*universe.schemaMutationTransformationdrop4114145141451414514145
profiler/operator*universe.mapTransformationmap31250831250831250831250831
profiler/operator*influxdb.readFilterSourcemerged_ReadRange5_filter21529282529282529282529282

Measure query and operator performance of array.from()

import "profiler"
import "array"

option profiler.enabledProfilers = ["query", "operator"]

arrData = [
  {_time: 2021-01-01T00:00:00Z, _field: "foo", _value: 1.2},
  {_time: 2021-01-02T00:00:00Z, _field: "foo", _value: 2.5},
  {_time: 2021-01-03T00:00:00Z, _field: "foo", _value: 2.6},
  {_time: 2021-01-04T00:00:00Z, _field: "foo", _value: 8.1},
  {_time: 2021-01-05T00:00:00Z, _field: "foo", _value: 4.9}
]

array.from(rows: arrData)

Output tables

_time_field_value
2021-01-01T00:00:00Zfoo1.2
2021-01-02T00:00:00Zfoo2.5
2021-01-03T00:00:00Zfoo2.6
2021-01-04T00:00:00Zfoo8.1
2021-01-05T00:00:00Zfoo4.9
_measurementTotalDurationCompileDurationQueueDurationPlanDurationRequeueDurationExecuteDurationConcurrencyMaxAllocatedTotalAllocatedRuntimeErrorsflux/query-plan
profiler/query67648775826621762100613643408320“digraph {…}”
_measurementTypeLabelCountMinDurationMaxDurationDurationSumMeanDuration
profiler/operator*array.tableSourcearray.from0156361563615636156361

Cloud or OSS?

InfluxDB OSS 2.0 now generally available!

InfluxDB OSS 2.0 is now generally available and ready for production use. See the InfluxDB OSS 2.0 release notes.

For information about upgrading to InfluxDB OSS 2.0, see: