Flux data model
To get the most out of using Flux to process your data, you must understand how Flux structures and operates on data. The Flux data model comprises the following:
Stream of tables
A stream of tables is a collection of zero or more tables. Data sources return results as a stream of tables.
A row is a collection of associated column values.
A group key defines which columns to use to group tables in a stream of tables. Each table in a stream of tables represents a unique group key instance. All rows in a table contain the same values for each group key column.
Example group key
A group key can be represented by an array of column labels.
[_measurement, facility, _field]
Example group key instances
Group key instances (unique to each table) include key-value pairs that identify each column name in the table that has the same value. The following are examples of group key instances in a stream of tables with three separate tables. Each represents a table containing data for a unique location:
[_measurement: "production", facility: "us-midwest", _field: "apq"] [_measurement: "production", facility: "eu-central", _field: "apq"] [_measurement: "production", facility: "ap-east", _field: "apq"]
An empty group key groups all data in a stream of tables into a single table.
For an example of how group keys work, see the Table grouping example below.
Data sources determine data structure
The Flux data model is separate from the queried data source model. Queried sources return data structured into columnar tables. The table structure and columns included depends on the data source.
For example, InfluxDB returns data grouped by series, so each table in the returned stream of tables represents a unique series. However, SQL data sources return a stream of tables with a single table and an empty group key.
Column labels beginning with underscores
Some data sources return column labels prefixed with an underscore (
This is a Flux convention used to identify important or reserved column names.
While the underscore doesn’t change the functionality of the column, many
functions in the Flux standard library expect or require
these specific column names.
Operate on tables
At its core, Flux operates on tables.
Flux transformations take a stream
of tables as input, but operate on each table individually.
For example, aggregate transformations like
output a stream of tables containing one table for each corresponding input table:
All tables in a stream of tables are defined by their group key.
To change how data is partitioned or grouped into tables, use functions such as
to modify group keys in a stream of tables.
data |> group(columns: ["foo", "bar"], mode: "by")
Table grouping example
The tables below represent data returned from InfluxDB with the following schema:
loctag with two values
sensorIDtag with two values
To modify the group key and see how the sample data is partitioned into new tables, select columns to group by:
data |> group(columns: ["_measurement", "loc", "sensorID", "_field"])
Was this page helpful?
Thank you for your feedback!
Support and feedback
Thank you for being part of our community! We welcome and encourage your feedback and bug reports for Flux and this documentation. To find support, use the following resources:
Customers with an annual or support contract can contact InfluxData Support.