Documentation

Flux data model

To get the most out of using Flux to process your data, you must understand how Flux structures and operates on data. The Flux data model comprises the following:

Stream of tables

A stream of tables is a collection of zero or more tables. Data sources return results as a stream of tables.

Table

A table is a collection of columns partitioned by group key.

Column

A column is a collection of values of the same basic type that contains one value for each row.

Row

A row is a collection of associated column values.

Group key

A group key defines which columns and specific column values to include in a table. All rows in a table contain the same values in group key columns. All tables in a stream of tables have a unique group key, but group key modifications are applied to a stream of tables.

Example group keys

Group keys contain key-value pairs, where each key represents a column name and each value represents the column value included in the table. The following are examples of group keys in a stream of tables with three separate tables. Each group key represents a table containing data for a unique location:

[_measurement: "production", facility: "us-midwest", _field: "apq"]
[_measurement: "production", facility: "eu-central", _field: "apq"]
[_measurement: "production", facility: "ap-east", _field: "apq"]

An empty group key groups all data in a stream of tables into a single table.

For an example of how group keys work, see the Table grouping example below.

Data sources determine data structure

The Flux data model is separate from the queried data source model. Queried sources return data structured into columnar tables. The table structure and columns included depends on the data source.

For example, InfluxDB returns data grouped by series, so each table in the returned stream of tables represents a unique series. However, SQL data sources return a stream of tables with a single table and an empty group key.

Operate on tables

At its core, Flux operates on tables. Flux transformations take a stream of tables as input, but operate on each table individually. For example, aggregate transformations like sum(), output a stream of tables containing one table for each corresponding input table:

|> sum()

Restructure tables

All tables in a stream of tables are defined by their group key. To change how data is partitioned or grouped into tables, use functions such as group() or window() to modify group keys in a stream of tables.

data
  |> group(columns: ["foo", "bar"], mode: "by")

Table grouping example

The tables below represent data returned from InfluxDB with the following schema:

  • example measurement
  • loc tag with two values
  • sensorID tag with two values
  • temp and hum fields

To modify the group key and see how the sample data is partitioned into new tables, select columns to group by:

data
  |> group(columns: ["_measurement", "loc", "sensorID", "_field"])

Upgrade to InfluxDB Cloud or InfluxDB 2.0!

InfluxDB Cloud and InfluxDB OSS 2.0 ready for production.