Size WAL retention

Write-ahead log (WAL) retention on the source InfluxDB 3 Enterprise server determines whether EDR recovers a missed window from WAL files (fast, precise) or falls back to slower compacted-file recovery: an outage or disconnection longer than the retention window forces the fallback. WAL files are compact (about 3x smaller than gen0), so generous retention is cheap—a source ingesting 1 MB/s needs about 86 GB for 24 hours of WAL retention.

This page covers sizing that retention and, if the resulting WAL buildup becomes a storage concern, using EDR’s optional WAL cleanup to bound it.

Why retention matters

The InfluxDB flag --wal-snapshots-to-keep (spelled --pt-wal-snapshots-to-keep on InfluxDB 3.10.x) controls how many WAL snapshots are retained before deletion:

retention_window ~= snapshots_to_keep x snapshot_interval (~10-20s)

The InfluxDB default (5) retains only about one to two minutes of data. Any agent restart or network interruption longer than that sends EDR to compacted-file recovery. EDR is correct either way—delivery is at-least-once regardless, and data that outlives WAL retention is recovered from the compacted files (see Historic fill and Gap fill). But EDR replicates faster with sized retention: WAL replication is precise and cheap, while recovery from cv2 is slower and replicates more bytes than strictly necessary.

If you see gaps regularly (gaps_pending in metrics), increase --wal-snapshots-to-keep on the source InfluxDB—recovery from compacted files is a safety net, not the intended steady state.

WAL cleanup for production

Generous WAL retention also means WAL files linger on the source object store long after EDR has replicated them. The source server owns their lifecycle and deletes them once they’re rolled into gen0 (governed by --wal-snapshots-to-keep). If you raise that retention so EDR has a wide replication window, WAL files can accumulate well past the point EDR has already replicated them.

For production, set server WAL retention high enough for your outage window and enable the opt-in --wal-cleanup-enabled sweep to bound the replicated tail’s storage cost. The agent deletes already-replicated WAL below four safety floors. Cleanup is off by default because it deletes source WAL. Server-negotiated retention holds are not part of EDR 1.0.0.

influxdb3-edr ... --wal-cleanup-enabled \
  --wal-cleanup-interval-secs 300 \
  --wal-cleanup-snapshot-margin 100

When enabled, every --wal-cleanup-interval-secs (default 5 minutes) the agent deletes WAL files for each source node only below the lowest of four floors, so it can never strand anything that still needs them:

1. The replication cursor—only files EDR has already replicated. (This also means cleanup can never make a not-yet-replicated file look “missing” and trigger a false gap.)
2. The snapshot boundary minus --wal-cleanup-snapshot-margin—WAL above the source’s last snapshot is not yet in gen0, so the source node itself still needs it to recover. The margin (default 100 files) keeps a safety buffer below that boundary.
3. The historic-fill consumed floor—historic fill reads older WAL during backfill; cleanup stays below what it still needs (and does nothing until historic fill’s plan exists).
4. The agent’s own pipeline floor—the oldest WAL file still queued or in flight in the agent, plus any range still unresolved in the gap ledger. A failed WAL file’s skip receipt holds this floor until the gap ledger records the failure. The replication cursor alone is not a safe bound for the agent’s own input: skip receipts can advance it past still-queued files, and without this floor a sufficiently-lagged agent could delete its own unprocessed backlog—a silent, unrecoverable shortfall.

With multiple destinations the first, third, and fourth floors are the min across every destination—WAL is retained for the most-behind consumer. A destination that is down therefore pins WAL indefinitely by default; --wal-cleanup-max-destination-hold bounds that: a destination whose cursor hasn’t advanced within the hold stops holding the floor (one loud WARN naming it), and when it returns it recovers the evicted range from gen0 and cv2 files through gap fill—the standard over-replication trade, never loss.

Deletion is by WAL-id range (no object-store listing) and resumes from a persisted per-node watermark, so steady-state sweeps only touch the small new increment. Normal sweeps log at debug; a sweep that deletes files logs one info line per node (wal cleanup: deleted replicated WAL files, with the watermark and the four floors); delete errors warn and the watermark is held (retried next sweep).

After enabling WAL cleanup, confirm gaps_pending and wal_files_lost_total stay flat—cleanup is designed never to create gaps, so a rise is a signal to investigate.