InfluxDB 3 Enterprise configuration options

InfluxDB 3 Enterprise lets you customize your server configuration by using influxdb3 serve command options or by setting environment variables.

Configure your server

Pass configuration options to the influxdb serve server using either command options or environment variables. Command options take precedence over environment variables.

Example influxdb3 serve command options

influxdb3 serve \
  --object-store file \
  --data-dir ~/.influxdb3 \
  --writer-id my-host \
  --log-filter info \
  --max-http-request-size 20971520 \
  --aws-allow-http

Example environment variables

export INFLUXDB3_OBJECT_STORE=file
export INFLUXDB3_DB_DIR=~/.influxdb3
export INFLUXDB3_WRITER_IDENTIFIER_PREFIX=my-host
export LOG_FILTER=info
export INFLUXDB3_MAX_HTTP_REQUEST_SIZE=20971520
export AWS_ALLOW_HTTP=true

influxdb3 serve

Server configuration options

• General
  • object-store
  • data-dir
  • writer-id
  • mode
• AWS
  • aws-access-key-id
  • aws-secret-access-key
  • aws-default-region
  • aws-endpoint
  • aws-session-token
  • aws-allow-http
  • aws-skip-signature
• Google Cloud Service
  • google-service-account
• Microsoft Azure
  • azure-storage-account
  • azure-storage-access-key
• Object Storage
  • bucket
  • object-store-connection-limit
  • object-store-http2-only
  • object-store-http2-max-frame-size
  • object-store-max-retries
  • object-store-retry-timeout
  • object-store-cache-endpoint
• Logs
  • log-filter
  • log-destination
  • log-format
  • query-log-size
• Traces
  • traces-exporter
  • traces-exporter-jaeger-agent-host
  • traces-exporter-jaeger-agent-port
  • traces-exporter-jaeger-service-name
  • traces-exporter-jaeger-trace-context-header-name
  • traces-jaeger-debug-name
  • traces-jaeger-tags
  • traces-jaeger-max-msgs-per-second
• DataFusion
  • datafusion-num-threads
  • datafusion-runtime-type
  • datafusion-runtime-disable-lifo-slot
  • datafusion-runtime-event-interval
  • datafusion-runtime-global-queue-interval
  • datafusion-runtime-max-blocking-threads
  • datafusion-runtime-max-io-events-per-tick
  • datafusion-runtime-thread-keep-alive
  • datafusion-runtime-thread-priority
  • datafusion-max-parquet-fanout
  • datafusion-use-cached-parquet-loader
  • datafusion-config
• HTTP
  • max-http-request-size
  • http-bind
  • bearer-token
• Memory
  • ram-pool-data-bytes
  • exec-mem-pool-bytes
  • buffer-mem-limit-mb
  • force-snapshot-mem-threshold
• Write-Ahead Log (WAL)
  • wal-flush-interval
  • wal-snapshot-size
  • wal-max-write-buffer-size
  • snapshotted-wal-files-to-keep
• Replication
  • read-from-writer-ids
  • replication-interval
• Compaction
  • compactor-id
  • compact-from-writer-ids
  • run-compactions
  • compaction-row-limit
  • compaction-max-num-files-per-plan
  • compaction-gen2-duration
  • compaction-multipliers
  • gen1-duration
• Caching
  • preemptive-cache-age
  • parquet-mem-cache-size-mb
  • parquet-mem-cache-prune-percentage
  • parquet-mem-cache-prune-interval
  • disable-parquet-mem-cache
  • last-cache-eviction-interval
  • distinct-cache-eviction-interval
• Plugins
  • plugin-dir

General

• object-store
• bucket
• data-dir
• writer-id
• mode

object-store

Specifies which object storage to use to store Parquet files. This option supports the following values:

• memory (default)
• memory-throttled
• file
• s3
• google
• azure

data-dir

Defines the location InfluxDB 3 Enterprise uses to store files locally.

writer-id

Specifies the writer identifier used as a prefix in all object store file paths. This should be unique for any hosts sharing the same object store configuration–for example, the same bucket.

mode

Sets the mode to start the server in.

This option supports the following values:

• read
• read_write (default)
• compactor

Default: read_write

AWS

• aws-access-key-id
• aws-secret-access-key
• aws-default-region
• aws-endpoint
• aws-session-token
• aws-allow-http
• aws-skip-signature

aws-access-key-id

When using Amazon S3 as the object store, set this to an access key that has permission to read from and write to the specified S3 bucket.

aws-secret-access-key

When using Amazon S3 as the object store, set this to the secret access key that goes with the specified access key ID.

aws-default-region

When using Amazon S3 as the object store, set this to the region that goes with the specified bucket if different from the fallback value.

Default: us-east-1

aws-endpoint

When using an Amazon S3 compatibility storage service, set this to the endpoint.

aws-session-token

When using Amazon S3 as an object store, set this to the session token. This is handy when using a federated login or SSO and fetching credentials via the UI.

aws-allow-http

Allows unencrypted HTTP connections to AWS.

aws-skip-signature

If enabled, S3 object stores do not fetch credentials and do not sign requests.

Google Cloud Service

• google-service-account

google-service-account

When using Google Cloud Storage as the object store, set this to the path to the JSON file that contains the Google credentials.

Microsoft Azure

• azure-storage-account
• azure-storage-access-key

azure-storage-account

When using Microsoft Azure as the object store, set this to the name you see when navigating to All Services > Storage accounts > [name].

azure-storage-access-key

When using Microsoft Azure as the object store, set this to one of the Key values in the Storage account’s Settings > Access keys.

Object Storage

• bucket
• object-store-connection-limit
• object-store-http2-only
• object-store-http2-max-frame-size
• object-store-max-retries
• object-store-retry-timeout
• object-store-cache-endpoint

bucket

Sets the name of the object storage bucket to use. Must also set --object-store to a cloud object storage for this option to take effect.

object-store-connection-limit

When using a network-based object store, limits the number of connections to this value.

Default: 16

object-store-http2-only

Forces HTTP/2 connections to network-based object stores.

object-store-http2-max-frame-size

Sets the maximum frame size (in bytes/octets) for HTTP/2 connections.

object-store-max-retries

Defines the maximum number of times to retry a request.

object-store-retry-timeout

Specifies the maximum length of time from the initial request after which no further retries are be attempted.

object-store-cache-endpoint

Sets the endpoint of an S3-compatible, HTTP/2-enabled object store cache.

Logs

• log-filter
• log-destination
• log-format
• query-log-size

log-filter

Sets the filter directive for logs.

log-destination

Specifies the destination for logs.

Default: stdout

log-format

Defines the message format for logs.

This option supports the following values:

• full (default)

Default: full

query-log-size

Defines the size of the query log. Up to this many queries remain in the log before older queries are evicted to make room for new ones.

Default: 1000

Traces

• traces-exporter
• traces-exporter-jaeger-agent-host
• traces-exporter-jaeger-agent-port
• traces-exporter-jaeger-service-name
• traces-exporter-jaeger-trace-context-header-name
• traces-jaeger-debug-name
• traces-jaeger-tags
• traces-jaeger-max-msgs-per-second

traces-exporter

Sets the type of tracing exporter.

Default: none

traces-exporter-jaeger-agent-host

Specifies the Jaeger agent network hostname for tracing.

Default: 0.0.0.0

traces-exporter-jaeger-agent-port

Defines the Jaeger agent network port for tracing.

Default: 6831

traces-exporter-jaeger-service-name

Sets the Jaeger service name for tracing.

Default: iox-conductor

traces-exporter-jaeger-trace-context-header-name

Specifies the header name used for passing trace context.

Default: uber-trace-id

traces-jaeger-debug-name

Specifies the header name used for force sampling in tracing.

Default: jaeger-debug-id

traces-jaeger-tags

Defines a set of key=value pairs to annotate tracing spans with.

traces-jaeger-max-msgs-per-second

Specifies the maximum number of messages sent to a Jaeger service per second.

Default: 1000

DataFusion

• datafusion-num-threads
• datafusion-runtime-type
• datafusion-runtime-disable-lifo-slot
• datafusion-runtime-event-interval
• datafusion-runtime-global-queue-interval
• datafusion-runtime-max-blocking-threads
• datafusion-runtime-max-io-events-per-tick
• datafusion-runtime-thread-keep-alive
• datafusion-runtime-thread-priority
• datafusion-max-parquet-fanout
• datafusion-use-cached-parquet-loader
• datafusion-config

datafusion-num-threads

Sets the maximum number of DataFusion runtime threads to use.

datafusion-runtime-type

Specifies the DataFusion tokio runtime type.

This option supports the following values:

• current-thread
• multi-thread (default)
• multi-thread-alt

Default: multi-thread

datafusion-runtime-disable-lifo-slot

Disables the LIFO slot of the DataFusion runtime.

This option supports the following values:

• true
• false

datafusion-runtime-event-interval

Sets the number of scheduler ticks after which the scheduler of the DataFusion tokio runtime polls for external events–for example: timers, I/O.

datafusion-runtime-global-queue-interval

Sets the number of scheduler ticks after which the scheduler of the DataFusion runtime polls the global task queue.

datafusion-runtime-max-blocking-threads

Specifies the limit for additional threads spawned by the DataFusion runtime.

datafusion-runtime-max-io-events-per-tick

Configures the maximum number of events processed per tick by the tokio DataFusion runtime.

datafusion-runtime-thread-keep-alive

Sets a custom timeout for a thread in the blocking pool of the tokio DataFusion runtime.

datafusion-runtime-thread-priority

Sets the thread priority for tokio DataFusion runtime workers.

Default: 10

datafusion-max-parquet-fanout

When multiple parquet files are required in a sorted way (deduplication for example), specifies the maximum fanout.

Default: 1000

datafusion-use-cached-parquet-loader

Uses a cached parquet loader when reading parquet files from the object store.

datafusion-config

Provides custom configuration to DataFusion as a comma-separated list of key:value pairs.

HTTP

• max-http-request-size
• http-bind
• bearer-token

max-http-request-size

Specifies the maximum size of HTTP requests.

Default: 10485760

http-bind

Defines the address on which InfluxDB serves HTTP API requests.

Default: 0.0.0.0:8181

bearer-token

Specifies the bearer token to be set for requests.

Memory

• ram-pool-data-bytes
• exec-mem-pool-bytes
• buffer-mem-limit-mb
• force-snapshot-mem-threshold

ram-pool-data-bytes

Specifies the size of the RAM cache used to store data, in bytes.

Default: 1073741824

exec-mem-pool-bytes

Specifies the size of the memory pool used during query execution, in bytes.

Default: 8589934592

buffer-mem-limit-mb

Specifies the size limit of the buffered data in MB. If this limit is exceeded, the server forces a snapshot.

Default: 5000

force-snapshot-mem-threshold

Specifies the threshold for the internal memory buffer. Supports either a percentage (portion of available memory)of or absolute value (total bytes)–for example: 70% or 100000.

Default: 70%

Write-Ahead Log (WAL)

• wal-flush-interval
• wal-snapshot-size
• wal-max-write-buffer-size
• snapshotted-wal-files-to-keep

wal-flush-interval

Specifies the interval to flush buffered data to a WAL file. Writes that wait for WAL confirmation take up to this interval to complete.

Default: 1s

wal-snapshot-size

Defines the number of WAL files to attempt to remove in a snapshot. This, multiplied by the interval, determines how often snapshots are taken.

Default: 600

wal-max-write-buffer-size

Specifies the maximum number of write requests that can be buffered before a flush must be executed and succeed.

Default: 100000

snapshotted-wal-files-to-keep

Specifies the number of snapshotted WAL files to retain in the object store. Flushing the WAL files does not clear the WAL files immediately; they are deleted when the number of snapshotted WAL files exceeds this number.

Default: 300

Replication

• read-from-writer-ids
• replication-interval

read-from-writer-ids

Specifies a comma-separated list of writer identifier prefixes (writer-ids) to read WAL files from. [env: =]

replication-interval

Defines the interval at which each replica specified in the read-from-writer-ids option is replicated.

Default: 250ms

Compaction

• compactor-id
• compact-from-writer-ids
• run-compactions
• compaction-row-limit
• compaction-max-num-files-per-plan
• compaction-gen2-duration
• compaction-multipliers
• gen1-duration

compactor-id

Specifies the prefix in the object store where all compacted data is written. Provide this option only if this server should handle compaction for its own write buffer and any replicas it manages.

compact-from-writer-ids

Defines a comma-separated list of writer identifier prefixes from which data is compacted.

run-compactions

Indicates that the server should run compactions. Only a single server should run compactions for a given compactor-id. This option is only applicable if a compactor-id is set.

compaction-row-limit

Specifies the soft limit for the number of rows per file that the compactor writes. The compactor may write more rows than this limit.

Default: 1000000

compaction-max-num-files-per-plan

Sets the maximum number of files included in any compaction plan.

Default: 500

compaction-gen2-duration

Specifies the duration of the first level of compaction (gen2). Later levels of compaction are multiples of this duration. This value should be equal to or greater than the gen1 duration.

Default: 20m

compaction-multipliers

Specifies a comma-separated list of multiples defining the duration of each level of compaction. The number of elements in the list determines the number of compaction levels. The first element specifies the duration of the first level (gen3); subsequent levels are multiples of the previous level.

Default: 3,4,6,5

gen1-duration

Specifies the duration that Parquet files are arranged into. Data timestamps land each row into a file of this duration. Supported durations are 1m, 5m, and 10m. These files are known as “generation 1” files, which the compactor can merge into larger generations.

Default: 10m

Caching

• preemptive-cache-age
• parquet-mem-cache-size-mb
• parquet-mem-cache-prune-percentage
• parquet-mem-cache-prune-interval
• disable-parquet-mem-cache
• last-cache-eviction-interval
• distinct-cache-eviction-interval

preemptive-cache-age

Specifies the interval to prefetch into the Parquet cache during compaction.

Default: 3d

parquet-mem-cache-size-mb

Defines the size of the in-memory Parquet cache in megabytes (MB).

Default: 1000

parquet-mem-cache-prune-percentage

Specifies the percentage of entries to prune during a prune operation on the in-memory Parquet cache.

Default: 0.1

parquet-mem-cache-prune-interval

Sets the interval to check if the in-memory Parquet cache needs to be pruned.

Default: 1s

disable-parquet-mem-cache

Disables the in-memory Parquet cache. By default, the cache is enabled.

last-cache-eviction-interval

Specifies the interval to evict expired entries from the Last-N-Value cache, expressed as a human-readable time–for example: 20s, 1m, 1h.

Default: 10s

distinct-cache-eviction-interval

Specifies the interval to evict expired entries from the distinct value cache, expressed as a human-readable time–for example: 20s, 1m, 1h.

Default: 10s

Plugins

• plugin-dir

plugin-dir

Specifies the local directory that contains Python plugins and their test files.