difference() function

difference() returns the difference between subsequent values.

Subtraction rules for numeric types

• The difference between two non-null values is their algebraic difference; or null, if the result is negative and nonNegative: true;
• null minus some value is always null;
• Some value v minus null is v minus the last non-null value seen before v; or null if v is the first non-null value seen.
• If nonNegative and initialZero are set to true, difference() returns the difference between 0 and the subsequent value. If the subsequent value is less than zero, difference() returns null.

Output tables

For each input table with n rows, difference() outputs a table with n - 1 rows.

Function type signature

(
    <-tables: stream[A],
    ?columns: [string],
    ?initialZero: bool,
    ?keepFirst: bool,
    ?nonNegative: bool,
) => stream[B] where A: Record, B: Record

Parameters

nonNegative

Disallow negative differences. Default is false.

When true, if a value is less than the previous value, the function assumes the previous value should have been a zero.

columns

List of columns to operate on. Default is ["_value"].

keepFirst

Keep the first row in each input table. Default is false.

If true, the difference of the first row of each output table is null.

initialZero

Use zero (0) as the initial value in the difference calculation when the subsequent value is less than the previous value and nonNegative is true. Default is false.

tables

Input data. Default is piped-forward data (<-).

Examples

• Calculate the difference between subsequent values
• Calculate the non-negative difference between subsequent values
• Calculate the difference between subsequent values with null values
• Keep the first value when calculating the difference between values

Calculate the difference between subsequent values

import "sampledata"

sampledata.int()
    |> difference()

Calculate the non-negative difference between subsequent values

import "sampledata"

sampledata.int()
    |> difference(nonNegative: true)

Calculate the difference between subsequent values with null values

import "sampledata"

sampledata.int(includeNull: true)
    |> difference()

Keep the first value when calculating the difference between values

import "sampledata"

sampledata.int()
    |> difference(keepFirst: true)