experimental.quantile() function

experimental.quantile() returns non-null records with values in the _value column that fall within the specified quantile or represent the specified quantile.

The _value column must contain float values.

Computation methods and behavior

experimental.quantile() behaves like an aggregate function or a selector function depending on the method parameter. The following computation methods are available:

estimate_tdigest

An aggregate method that uses a t-digest data structure to compute an accurate quantile estimate on large data sources. When used, experimental.quantile() outputs non-null records with values that fall within the specified quantile.

exact_mean

An aggregate method that takes the average of the two points closest to the quantile value. When used, experimental.quantile() outputs non-null records with values that fall within the specified quantile.

exact_selector

A selector method that returns the data point for which at least q points are less than. When used, experimental.quantile() outputs the non-null record with the value that represents the specified quantile.

Function type signature

(
    <-tables: stream[{A with _value: float}],
    q: float,
    ?compression: float,
    ?method: string,
) => stream[{A with _value: float}]

Parameters

q

(Required) Quantile to compute ([0 - 1]).

method

Computation method. Default is estimate_tdigest.

Supported methods:

• estimate_tdigest
• exact_mean
• exact_selector

compression

Number of centroids to use when compressing the dataset. Default is 1000.0.

A larger number produces a more accurate result at the cost of increased memory requirements.

tables

Input data. Default is piped-forward data (<-).

Examples

• Return values in the 50th percentile of each input table
• Return a value representing the 50th percentile of each input table

Return values in the 50th percentile of each input table

import "experimental"
import "sampledata"

sampledata.float()
    |> experimental.quantile(q: 0.5)

Return a value representing the 50th percentile of each input table

import "experimental"
import "sampledata"

sampledata.float()
    |> experimental.quantile(q: 0.5, method: "exact_selector")