quantile() function

quantile() returns rows from each input table with values that fall within a specified quantile or returns the row with the value that represents the specified quantile.

quantile() supports columns with float values.

Function behavior

quantile() acts as an aggregate or selector transformation depending on the specified method.

• Aggregate: When using the estimate_tdigest or exact_mean methods, quantile() acts as an aggregate transformation and outputs the average of non-null records with values that fall within the specified quantile.
• Selector: When using the exact_selector method, quantile() acts as a selector selector transformation and outputs the non-null record with the value that represents the specified quantile.

Function type signature

(
    <-tables: stream[A],
    q: float,
    ?column: string,
    ?compression: float,
    ?method: string,
) => stream[A] where A: Record

Parameters

column

Column to use to compute the quantile. Default is _value.

q

(Required) Quantile to compute. Must be between 0.0 and 1.0.

method

Computation method. Default is estimate_tdigest.

Avaialable methods:

• estimate_tdigest: Aggregate method that uses a t-digest data structure to compute an accurate quantile estimate on large data sources.
• exact_mean: Aggregate method that takes the average of the two points closest to the quantile value.
• exact_selector: Selector method that returns the row with the value for which at least q points are less than.

compression

Number of centroids to use when compressing the dataset. Default is 1000.0.

A larger number produces a more accurate result at the cost of increased memory requirements.

tables

Input data. Default is piped-forward data (<-).

Examples

• Quantile as an aggregate
• Quantile as a selector

Quantile as an aggregate

import "sampledata"

sampledata.float()
    |> quantile(q: 0.99, method: "estimate_tdigest")

Quantile as a selector

import "sampledata"

sampledata.float()
    |> quantile(q: 0.5, method: "exact_selector")