Flux data model
To get the most out of using Flux to process your data, you must understand how Flux structures and operates on data. The Flux data model comprises the following:
Stream of tables
A stream of tables is a collection of zero or more tables. Data sources return results as a stream of tables.
A row is a collection of associated column values.
A group key defines which columns and specific column values to include in a table. All rows in a table contain the same values in group key columns. All tables in a stream of tables have a unique group key, but group key modifications are applied to a stream of tables.
Example group keys
Group keys contain key-value pairs, where each key represents a column name and each value represents the column value included in the table. The following are examples of group keys in a stream of tables with three separate tables. Each group key represents a table containing data for a unique location:
[_measurement: "production", facility: "us-midwest", _field: "apq"] [_measurement: "production", facility: "eu-central", _field: "apq"] [_measurement: "production", facility: "ap-east", _field: "apq"]
An empty group key groups all data in a stream of tables into a single table.
For an example of how group keys work, see the Table grouping example below.
Data sources determine data structure
The Flux data model is separate from the queried data source model. Queried sources return data structured into columnar tables. The table structure and columns included depends on the data source.
For example, InfluxDB returns data grouped by series, so each table in the returned stream of tables represents a unique series. However, SQL data sources return a stream of tables with a single table and an empty group key.
Column labels beginning with underscores
Some data sources return column labels prefixed with an underscore (
This is a Flux convention used to identify important or reserved column names.
While the underscore doesn’t change the functionality of the column, many
functions in the Flux standard library expect or require
these specific column names.
Operate on tables
At its core, Flux operates on tables.
Flux transformations take a stream
of tables as input, but operate on each table individually.
For example, aggregate transformations like
output a stream of tables containing one table for each corresponding input table:
All tables in a stream of tables are defined by their group key.
To change how data is partitioned or grouped into tables, use functions such as
to modify group keys in a stream of tables.
data |> group(columns: ["foo", "bar"], mode: "by")
Table grouping example
The tables below represent data returned from InfluxDB with the following schema:
loctag with two values
sensorIDtag with two values
To modify the group key and see how the sample data is partitioned into new tables, select columns to group by:
data |> group(columns: ["_measurement", "loc", "sensorID", "_field"])
Was this page helpful?
Thank you for your feedback!
Support and feedback
Thank you for being part of our community! We welcome and encourage your feedback and bug reports for Flux and this documentation. To find support, use the following resources:
InfluxDB Cloud customers can contact InfluxData Support.