River auto profiler plugin

A zero-config InfluxDB 3 Enterprise Processing Engine plugin that incrementally profiles incoming time-series data, classifies data patterns, and writes recommended anomaly detection parameters per series. This is the “zero-config” enabler — it learns about your data so the anomaly detector can auto-tune.

Features

• Incremental profiling: Uses River ML’s streaming statistics to build profiles one observation at a time
• Pattern classification: Automatically classifies data as stable, noisy, trending, seasonal, or bursty
• Adaptive recommendations: Recommends detector mode, threshold, and fading factors based on data characteristics
• Calibrated EW stats: Collects initial observations to determine optimal fading factor before creating EW statistics, then replays calibration data
• Exceedance-calibrated thresholds: Automatically adjusts anomaly thresholds based on observed data distribution (targets ~1% anomaly rate)
• Per-series profiles: Each unique combination of table + tags + field gets its own profile
• Low overhead: Only writes profiles every N observations (default 50) or every 5 minutes for slow data
• LRU eviction: Configurable limit on tracked series to control memory usage

How It Works

On every write, the profiler iterates over each numeric field in each row and updates streaming statistics for a unique series, keyed by table + tag values + field name. Tracked stats include EW mean/variance, skewness, kurtosis, write interval, and seasonal variance buckets (hourly or weekly).

Each series starts in a short calibration phase (30 observations). During calibration values are buffered; once it ends, the profiler uses the observed write interval to pick an appropriate fading factor and replays the buffered values through the EW stats so the profile starts off already informed by early history.

After calibration, every profile_write_interval observations (or every 5 minutes for slow streams) the plugin writes a profile snapshot to _meta.series_profiles — pattern label, recommended detector mode, threshold, and fading factors. Profiles with fewer than min_observations are flagged profile_mature=false, so downstream consumers can fall back to safe defaults until the profile has seen enough data.

Prerequisites

• InfluxDB 3 Enterprise with Processing Engine enabled
• River ML library (river>=0.23.0)

Quick Start

1. Install River (if not already installed)

influxdb3 install package river

2. Create the profiler trigger

influxdb3 create trigger \
  --database mydb \
  --plugin-filename river_auto_profiler.py \
  --trigger-spec "all_tables" \
  auto_profiler

3. Query profiles

SELECT * FROM "_meta.series_profiles" ORDER BY time DESC LIMIT 10

Trigger Arguments

Arguments can be passed inline (space-separated lists) or via TOML config file (native TOML lists). Parameter names are the same in both modes.

Inline example

influxdb3 create trigger \
  --database mydb \
  --plugin-filename river_auto_profiler.py \
  --trigger-spec "all_tables" \
  --trigger-arguments 'include_fields=temperature humidity' 'exclude_tables=debug_info system_logs' \
  auto_profiler

TOML config example

influxdb3 create trigger \
  --database mydb \
  --plugin-filename river_auto_profiler.py \
  --trigger-spec "all_tables" \
  --trigger-arguments config_file_path=river_auto_profiler_config.toml \
  auto_profiler

Or with an absolute path (no PLUGIN_DIR needed):

influxdb3 create trigger \
  --database mydb \
  --plugin-filename river_auto_profiler.py \
  --trigger-spec "all_tables" \
  --trigger-arguments config_file_path=/etc/influxdb3/river_auto_profiler_config.toml \
  auto_profiler

See river_auto_profiler_config.toml for an example configuration.

Column Classification

The plugin classifies each column in incoming data:

1. time — skipped
2. String columns listed in string_fields — skipped (not a tag, not numeric)
3. Other string columns — treated as tags (used in series key)
4. Numeric columns (int, float, not bool) — treated as fields to profile
   • Filtered by include_fields (if set, only listed fields are profiled)
   • Filtered by exclude_fields (excluded fields are skipped)

Table Filtering

All tables starting with _ are automatically skipped (for example, _meta.*, _anomalies.*, _system.*, _forecasts.*). Additional tables can be excluded via exclude_tables.

Output Schema

Profiles are written to _meta.series_profiles.

Tags: source_table, field_name, plus all original tags from the source data.

Fields:

Integration with Anomaly Detector

The recommended parameters written to _meta.series_profiles are consumed by the companion river_anomaly_detector plugin when it runs with auto_tune=true (default). Deploying both triggers on the same database lets the detector auto-tune per series:

influxdb3 create trigger \
  --database mydb \
  --plugin-filename river_auto_profiler.py \
  --trigger-spec "all_tables" \
  auto_profiler

influxdb3 create trigger \
  --database mydb \
  --plugin-filename river_anomaly_detector.py \
  --trigger-spec "all_tables" \
  anomaly_detector

Until a profile matures (profile_mature=true, i.e. at least min_observations observations), the anomaly detector falls back to conservative defaults.

Pattern Classification

The profiler classifies each series into one of five patterns based on streaming statistics:

Seasonality Detection

Uses hourly (24 buckets) or weekly (168 buckets) variance buckets. If the coefficient of variation across bucket variances is high, the data is seasonal. Requires at least 25% of buckets with 2+ observations each. The seasonality_ready field in the output indicates whether enough data has been collected for reliable seasonality detection.

Trend Detection

Uses EW mean drift — compares a fast EW mean (fading factor 0.3) against a slow EW mean (fading factor 0.05). Large divergence relative to the standard deviation indicates trending data. Requires at least 20 observations.

Tuning Rules

Threshold Recommendation (exceedance-calibrated)

The threshold is calibrated automatically using exceedance rate tracking. The profiler monitors what fraction of observations fall outside mean ± threshold × std and adjusts the threshold to target a ~1% anomaly rate. The threshold is bounded to [2.5, 10.0] and calibrated every 200 observations. Initial value is set from pattern classification:

Fading Factor Recommendation (based on write frequency)

Seasonal Fading Factor Recommendation

Model Persistence

The plugin automatically checkpoints and restores models so they survive server restarts.

Checkpoint

Every checkpoint_interval_seconds (default: 1800 = 30 minutes), the plugin pickles each per-series profile, compresses with zlib, base64-encodes, and writes to _system.model_checkpoints. Large models are automatically chunked across multiple rows (60KB chunks) to stay within InfluxDB’s string field limit.

Profiles with fewer than min_checkpoint_observations observations (default: 10) are skipped — there is no point persisting cold-start state that would be rebuilt from scratch on restore anyway.

Restore

On the first invocation after a server restart (detected by an empty profile cache), the plugin queries _system.model_checkpoints for the latest checkpoint per series. Checkpoints older than max_checkpoint_age_hours (default: 24) are ignored. Restored profiles resume profiling immediately with their full statistical history.

Checkpoint Schema (_system.model_checkpoints)

Logging

Logs are stored in the _internal database (or the database where the trigger is created) in the system.processing_engine_logs table. To view logs:

influxdb3 query --database _internal "SELECT * FROM system.processing_engine_logs WHERE trigger_name = 'your_trigger_name'"

Log columns:

• event_time: Timestamp of the log event
• trigger_name: Name of the trigger that generated the log
• log_level: Severity level (INFO, WARN, ERROR)
• log_text: Message describing the action or error

Report an issue

For plugin issues, see the Plugins repository issues page.

Find support for InfluxDB 3 Enterprise

The InfluxDB Discord server is the best place to find support for InfluxDB 3 Core and InfluxDB 3 Enterprise. For other InfluxDB versions, see the Support and feedback options.