Monitor the performance upgrade preview

InfluxDB 3 Enterprise provides system tables and a query telemetry endpoint to monitor file status, query execution, and overall performance when using the performance upgrade preview.

System tables

The upgraded storage engine exposes internal state through system tables that you can query with SQL.

system.pt_ingest_wal

View WAL files and their shards:

SELECT * FROM system.pt_ingest_wal;

Example output:

Use this table to monitor:

• WAL accumulation: Track the number and size of unmerged WAL files
• Shard distribution: See how data is distributed across shards
• Time coverage: Verify data time ranges
• Merge status: Identify WAL files where is_merged is false

Monitor WAL backlog

Check for WAL accumulation that may indicate merging is falling behind:

SELECT
  COUNT(*) as wal_file_count,
  SUM(size_bytes) / 1024 / 1024 as total_size_mb,
  MIN(min_time) as oldest_data,
  MAX(max_time) as newest_data
FROM system.pt_ingest_wal;

system.pt_ingest_files

View Gen0 files with metadata:

SELECT * FROM system.pt_ingest_files;

Example output:

Use this table to monitor:

• File counts per generation: Track compaction progress
• File sizes: Verify files are within configured limits
• Time ranges: Identify Gen0 files that span multiple compaction windows

Monitor file distribution

Check file distribution and compaction status:

SELECT
  generation,
  COUNT(*) as file_count,
  SUM(row_count) as total_rows,
  SUM(size_bytes) / 1024 / 1024 as total_size_mb,
  AVG(size_bytes) / 1024 / 1024 as avg_file_size_mb
FROM system.pt_ingest_files
GROUP BY generation
ORDER BY generation;

Compaction tables

The following system tables expose the state of the pacha-tree compaction subsystem.

system.pt_compaction_active_jobs

View currently running compaction jobs:

SELECT * FROM system.pt_compaction_active_jobs;

Key columns:

system.pt_compaction_ingest_nodes

View per-ingest-node compaction lag:

SELECT * FROM system.pt_compaction_ingest_nodes;

Key columns:

Use compaction_lag and deferred_snapshot_count as the primary health indicators. A non-zero deferred_snapshot_count means snapshots failed to compact and are accumulating; check system.pt_compaction_deferred_snapshots for details.

system.pt_compaction_nodes

View compaction node state:

SELECT * FROM system.pt_compaction_nodes;

system.pt_compaction_run_sets

View pending compaction work grouped by time window and shard:

SELECT * FROM system.pt_compaction_run_sets;

system.pt_compaction_deferred_snapshots

View snapshots that failed to compact:

SELECT * FROM system.pt_compaction_deferred_snapshots;

A growing list here indicates a persistent compaction failure. Check error_message for the root cause.

Parquet upgrade status

If you upgraded from Parquet, use these system tables to monitor migration progress.

system.upgrade_parquet_node

View per-node upgrade status:

SELECT * FROM system.upgrade_parquet_node;

Monitor this table to confirm each node reaches completed status. During the upgrade, nodes progress through detection, conversion, and finalization stages.

system.upgrade_parquet

View per-file migration progress:

SELECT * FROM system.upgrade_parquet;

Use this table to track individual file conversions during the migration. The status updates on a polling interval (default 5 seconds, configurable with --pt-upgrade-poll-interval).

Query telemetry

The query telemetry endpoint provides detailed execution statistics for analyzing query performance.

Enable query telemetry

Query the telemetry endpoint after executing a query:

curl -X GET "http://localhost:8181/api/v3/query_sql_telemetry" \
  -H "Authorization: Bearer AUTH_TOKEN"

Replace AUTH_TOKEN with your authentication token.

Telemetry response

The response includes:

Example telemetry output

{
  "query_id": "q_12345",
  "execution_time_us": 4523,
  "chunks": [
    {
      "chunk_id": "c_1",
      "files_scanned": 3,
      "blocks_processed": 12,
      "rows_read": 24000,
      "rows_returned": 150,
      "bytes_read": 1234567
    }
  ],
  "cache_stats": {
    "gen0_hits": 5,
    "gen0_misses": 1,
    "compacted_hits": 8,
    "compacted_misses": 2
  }
}

Performance analysis

Query performance metrics

Track these key metrics for query performance:

Ingest performance metrics

Monitor these metrics for write performance:

Monitor with SQL

Create a performance summary query:

-- File generation summary
SELECT
  'Gen0 files' as metric,
  COUNT(*) as count,
  SUM(size_bytes) / 1024 / 1024 as size_mb
FROM system.pt_ingest_files
WHERE generation = 0

UNION ALL

SELECT
  'Compacted files' as metric,
  COUNT(*) as count,
  SUM(size_bytes) / 1024 / 1024 as size_mb
FROM system.pt_ingest_files
WHERE generation > 0

UNION ALL

SELECT
  'WAL files' as metric,
  COUNT(*) as count,
  SUM(size_bytes) / 1024 / 1024 as size_mb
FROM system.pt_ingest_wal;

Troubleshooting

High WAL file count

Symptom: system.pt_ingest_wal shows many accumulated files.

Possible causes:

• Merge operations falling behind write rate
• Insufficient flush concurrency
• Object storage latency

Solutions:

1. Increase flush concurrency:

   --pt-wal-flush-concurrency 8

2. Increase WAL flush interval to create larger, fewer files:

   --pt-wal-flush-interval 5s

3. Increase the WAL buffer size so each flush produces a larger file:

   --pt-wal-max-buffer-size 30MB

4. Check object storage performance and connectivity.

High cache miss rate

Symptom: cache_stats shows >40% miss rate.

Possible causes:

• Cache size too small for working set
• Cache recency window too narrow
• Random access patterns across time ranges

Solutions:

1. Increase cache size:

   --pt-file-cache-size 16GB

2. Extend cache recency window:

   --pt-file-cache-recency 24h

3. Extend eviction timeout:

   --pt-file-cache-evict-after 48h

Slow compaction

Symptom: Gen0 file count continues to grow.

Possible causes:

• Compaction budget too low for write volume
• High write rate overwhelming compaction
• Snapshot size too large, creating oversized Gen0 files

Solutions:

1. Increase the compaction input size budget:

   --pt-compactor-input-size-budget 12GB

2. Reduce snapshot size to create smaller, more frequent Gen0 files:

   --pt-snapshot-size 125MB

3. For distributed deployments, add a dedicated compactor node:

   influxdb3 serve \
     # ...
     --use-pacha-tree \
     --mode compact

Query node lag

Symptom: Query nodes return stale data.

Possible causes:

• Replication falling behind
• Network latency to object storage
• Insufficient replica concurrency

Solutions:

For a full list of replication options, see Replication (query nodes).

1. Increase replication concurrency:

   --pt-wal-replica-steady-concurrency 8

2. Reduce the replication polling interval:

   --pt-wal-replication-interval 100ms

3. Increase replica queue size:

   --pt-wal-replica-queue-size 200