Documentation

Transform data with mathematic operations

This page documents an earlier version of InfluxDB. InfluxDB v2.6 is the latest stable version. View this page in the v2.6 documentation.

Flux, InfluxData’s data scripting and query language, supports mathematic expressions in data transformations. This article describes how to use Flux arithmetic operators to “map” over data and transform values using mathematic operations.

If you’re just getting started with Flux queries, check out the following:

Basic mathematic operations
// Examples executed using the Flux REPL
> 9 + 9
18
> 22 - 14
8
> 6 * 5
30
> 21 / 7
3

See Flux Read-Eval-Print Loop (REPL).

Operands must be the same type

Operands in Flux mathematic operations must be the same data type. For example, integers cannot be used in operations with floats. Otherwise, you will get an error similar to:

Error: type error: float != int

To convert operands to the same type, use type-conversion functions or manually format operands. The operand data type determines the output data type. For example:

100 // Parsed as an integer
100.0 // Parsed as a float

// Example evaluations
> 20 / 8
2

> 20.0 / 8.0
2.5

Custom mathematic functions

Flux lets you create custom functions that use mathematic operations. View the examples below.

Custom multiplication function
multiply = (x, y) => x * y

multiply(x: 10, y: 12)
// Returns 120
Custom percentage function
percent = (sample, total) => (sample / total) * 100.0

percent(sample: 20.0, total: 80.0)
// Returns 25.0

Transform values in a data stream

To transform multiple values in an input stream, your function needs to:

The example multiplyByX() function below includes:

  • A tables parameter that represents the input data stream (<-).
  • An x parameter which is the number by which values in the _value column are multiplied.
  • A map() function that iterates over each row in the input stream. It uses the with operator to preserve existing columns in each row. It also multiples the _value column by x.
multiplyByX = (x, tables=<-) => tables
    |> map(fn: (r) => ({r with _value: r._value * x}))

data
    |> multiplyByX(x: 10)

Examples

Convert bytes to gigabytes

To convert active memory from bytes to gigabytes (GB), divide the active field in the mem measurement by 1,073,741,824.

The map() function iterates over each row in the piped-forward data and defines a new _value by dividing the original _value by 1073741824.

from(bucket: "example-bucket")
    |> range(start: -10m)
    |> filter(fn: (r) => r._measurement == "mem" and r._field == "active")
    |> map(fn: (r) => ({r with _value: r._value / 1073741824}))

You could turn that same calculation into a function:

bytesToGB = (tables=<-) => tables
    |> map(fn: (r) => ({r with _value: r._value / 1073741824}))

data
    |> bytesToGB()

Include partial gigabytes

Because the original metric (bytes) is an integer, the output of the operation is an integer and does not include partial GBs. To calculate partial GBs, convert the _value column and its values to floats using the float() function and format the denominator in the division operation as a float.

bytesToGB = (tables=<-) => tables
    |> map(fn: (r) => ({r with _value: float(v: r._value) / 1073741824.0}))

Calculate a percentage

To calculate a percentage, use simple division, then multiply the result by 100.

> 1.0 / 4.0 * 100.0
25.0

For an in-depth look at calculating percentages, see Calculate percentages.

Pivot vs join

To query and use values in mathematical operations in Flux, operand values must exists in a single row. Both pivot() and join() will do this, but there are important differences between the two:

Pivot is more performant

pivot() reads and operates on a single stream of data. join() requires two streams of data and the overhead of reading and combining both streams can be significant, especially for larger data sets.

Use join for multiple data sources

Use join() when querying data from different buckets or data sources.

Pivot fields into columns for mathematic calculations
data
    |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value")
    |> map(fn: (r) => ({r with _value: (r.field1 + r.field2) / r.field3 * 100.0}))
Join multiple data sources for mathematic calculations
import "sql"
import "influxdata/influxdb/secrets"

pgUser = secrets.get(key: "POSTGRES_USER")
pgPass = secrets.get(key: "POSTGRES_PASSWORD")
pgHost = secrets.get(key: "POSTGRES_HOST")

t1 = sql.from(
    driverName: "postgres",
    dataSourceName: "postgresql://${pgUser}:${pgPass}@${pgHost}",
    query: "SELECT id, name, available FROM example_table",
)

t2 = from(bucket: "example-bucket")
    |> range(start: -1h)
    |> filter(fn: (r) => r._measurement == "example-measurement" and r._field == "example-field")

join(tables: {t1: t1, t2: t2}, on: ["id"])
    |> map(fn: (r) => ({r with _value: r._value_t2 / r.available_t1 * 100.0}))

Was this page helpful?

Thank you for your feedback!


Set your InfluxDB URL

Linux Package Signing Key Rotation

All signed InfluxData Linux packages have been resigned with an updated key. If using Linux, you may need to update your package configuration to continue to download and verify InfluxData software packages.

For more information, see the Linux Package Signing Key Rotation blog post.

InfluxDB Cloud backed by InfluxDB IOx

All InfluxDB Cloud organizations created on or after January 31, 2023 are backed by the new InfluxDB IOx storage engine. Check the right column of your InfluxDB Cloud organization homepage to see which InfluxDB storage engine you’re using.

If powered by IOx, this is the correct documentation.

If powered by TSM, see the TSM-based InfluxDB Cloud documentation.

InfluxDB Cloud backed by InfluxDB TSM

All InfluxDB Cloud organizations created on or after January 31, 2023 are backed by the new InfluxDB IOx storage engine which enables nearly unlimited series cardinality and SQL query support. Check the right column of your InfluxDB Cloud organization homepage to see which InfluxDB storage engine you’re using.

If powered by TSM, this is the correct documentation.

If powered by IOx, see the IOx-based InfluxDB Cloud documentation.

State of the InfluxDB Cloud (IOx) documentation

The new documentation for InfluxDB Cloud backed by InfluxDB IOx is a work in progress. We are adding new information and content almost daily. Thank you for your patience!

If there is specific information you’re looking for, please submit a documentation issue.