Get started with InfluxDB 3 Enterprise

What’s in this guide

This guide covers Enterprise as well as InfluxDB 3 Core, including the following topics:

• Install and startup

• Authentication and authorization

• Data Model

• Tools to use

• Write data

• Query data

• Last values cache

• Distinct values cache

• Python plugins and the processing engine

• Multi-server setups

Install and startup

InfluxDB 3 Enterprise runs on Linux, macOS, and Windows.

curl -O https://www.influxdata.com/d/install_influxdb3.sh \
&& sh install_influxdb3.sh enterprise

docker pull influxdb:3-enterprise

Build artifacts and images update with every merge into the InfluxDB 3 Enterprise main branch.

Verify the install

After you have installed InfluxDB 3 Enterprise, enter the following command to verify that it completed successfully:

influxdb3 --version

If your system doesn’t locate influxdb3, then source the configuration file (for example, .bashrc, .zshrc) for your shell–for example:

source ~/.zshrc

Start InfluxDB

To start your InfluxDB instance, use the influxdb3 serve command and provide the following:

• --object-store: Specifies the type of object store to use. InfluxDB supports the following: local file system (file), memory, S3 (and compatible services like Ceph or Minio) (s3), Google Cloud Storage (google), and Azure Blob Storage (azure). The default is file. Depending on the object store type, you may need to provide additional options for your object store configuration.

• --node-id: A string identifier that distinguishes individual server instances within the cluster. This forms the final part of the storage path: <CONFIGURED_PATH>/<CLUSTER_ID>/<NODE_ID>. In a multi-node setup, this ID is used to reference specific nodes.

• --cluster-id: A string identifier that determines part of the storage path hierarchy. All nodes within the same cluster share this identifier. The storage path follows the pattern <CONFIGURED_PATH>/<CLUSTER_ID>/<NODE_ID>. In a multi-node setup, this ID is used to reference the entire cluster.

The following examples show how to start InfluxDB 3 Enterprise with different object store configurations.

Filesystem object store

Store data in a specified directory on the local filesystem. This is the default object store type.

Replace the following with your values:

# Filesystem object store
# Provide the filesystem directory
influxdb3 serve \
  --node-id host01 \
  --cluster-id cluster01 \
  --object-store file \
  --data-dir ~/.influxdb3

To run the Docker image and persist data to the file system, mount a volume for the object store-for example, pass the following options:

• -v /path/on/host:/path/in/container: Mounts a directory from your file system to the container
• --object-store file --data-dir /path/in/container: Uses the mount for server storage

# File system object store with Docker 
# Create a mount
# Provide the mount path
docker run -it \
 -v /path/on/host:/path/in/container \
 influxdb:3-enterprise influxdb3 serve \
 --node-id my_host \
 --cluster-id my_cluster \
 --object-store file \
 --data-dir /path/in/container

S3 object store

Store data in an S3-compatible object store. This is useful for production deployments that require high availability and durability. Provide your bucket name and credentials to access the S3 object store.

# S3 object store (default is the us-east-1 region)
# Specify the object store type and associated options
influxdb3 serve \
  --node-id host01 \
  --cluster-id cluster01 \
  --object-store s3 \
  --bucket OBJECT_STORE_BUCKET \
  --aws-access-key AWS_ACCESS_KEY_ID \
  --aws-secret-access-key AWS_SECRET_ACCESS_KEY

# Minio or other open source object store
# (using the AWS S3 API with additional parameters)
# Specify the object store type and associated options
influxdb3 serve \
  --node-id host01 \
  --cluster-id cluster01 \
  --object-store s3 \
  --bucket OBJECT_STORE_BUCKET \
  --aws-access-key-id AWS_ACCESS_KEY_ID \
  --aws-secret-access-key AWS_SECRET_ACCESS_KEY \
  --aws-endpoint ENDPOINT \
  --aws-allow-http

Memory object store

Store data in RAM without persisting it on shutdown. It’s useful for rapid testing and development.

# Memory object store
# Stores data in RAM; doesn't persist data
influxdb3 serve \
--node-id host01 \
--cluster-id cluster01 \
--object-store memory

For more information about server options, use the CLI help or view the InfluxDB 3 CLI reference:

influxdb3 serve --help

Licensing

When first starting a new instance, InfluxDB 3 Enterprise prompts you to select a license type.

InfluxDB 3 Enterprise licenses authorize the use of the InfluxDB 3 Enterprise software and apply to a single cluster. Licenses are primarily based on the number of CPUs InfluxDB can use, but there are other limitations depending on the license type. The following InfluxDB 3 Enterprise license types are available:

• Trial: 30-day trial license with full access to InfluxDB 3 Enterprise capabilities.
• At-Home: For at-home hobbyist use with limited access to InfluxDB 3 Enterprise capabilities.
• Commercial: Commercial license with full access to InfluxDB 3 Enterprise capabilities.

For more information, see how to Manage your license.

Authentication and authorization

InfluxDB 3 Enterprise uses token-based authentication and authorization, which is enabled by default when you start the server.

With authentication enabled, you must provide a token with influxdb3 CLI commands and HTTP API requests.

InfluxDB 3 Enterprise supports the following types of tokens:

• admin token: Grants access to all CLI actions and API endpoints. A server can have one admin token.

• resource tokens: Tokens that grant read and write access to specific resources (databases and system information endpoints) on the server.

  • A database token grants access to write and query data in a database
  • A system token grants read access to system information endpoints and metrics for the server

For more information about tokens and authorization, see Manage tokens.

Create an operator token

After you start the server, create your first admin token. The first admin token you create is the operator token for the server.

Use the influxdb3 CLI or the HTTP API to create your operator token.

influxdb3 create token --admin

# With Docker — in a new terminal:
docker exec -it 
CONTAINER_NAME
 influxdb3 create token --admin

The command returns a token string for authenticating CLI commands and API requests. Store your token securely—you cannot retrieve it from the database later.

Set your token for authentication

Use your operator token to authenticate server actions in InfluxDB 3 Enterprise, such as creating additional tokens, performing administrative tasks, and writing and querying data.

Use one of the following methods to provide your token and authenticate influxdb3 CLI commands.

In your command, replace YOUR_AUTH_TOKEN with your token string (for example, the operator token from the previous step).

export INFLUXDB3_AUTH_TOKEN=
YOUR_AUTH_TOKEN

influxdb3 show databases --token AUTH_TOKEN

For HTTP API requests, include your token in the Authorization header–for example:

curl "http://localhost:8181/api/v3/configure/database" \
  --header "Authorization: Bearer 
AUTH_TOKEN
"

Learn more about tokens and permissions

• Manage admin tokens - Understand and manage operator and named admin tokens

• Manage resource tokens - Create, list, and delete resource tokens

• Authentication - Understand authentication, authorizations, and permissions in InfluxDB 3 Enterprise

Data model

The database server contains logical databases, which have tables, which have columns. Compared to previous versions of InfluxDB you can think of a database as a bucket in v2 or as a db/retention_policy in v1. A table is equivalent to a measurement, which has columns that can be of type tag (a string dictionary), int64, float64, uint64, bool, or string and finally every table has a time column that is a nanosecond precision timestamp.

In InfluxDB 3, every table has a primary key–the ordered set of tags and the time–for its data. This is the sort order used for all Parquet files that get created. When you create a table, either through an explicit call or by writing data into a table for the first time, it sets the primary key to the tags in the order they arrived. This is immutable. Although InfluxDB is still a schema-on-write database, the tag column definitions for a table are immutable.

Tags should hold unique identifying information like sensor_id, or building_id or trace_id. All other data should be kept in fields. You will be able to add fast last N value and distinct value lookups later for any column, whether it is a field or a tag.

Tools to use

The following table compares tools that you can use to interact with InfluxDB 3 Enterprise. This tutorial covers many of the recommended tools.

Write data

InfluxDB is a schema-on-write database. You can start writing data and InfluxDB creates the logical database, tables, and their schemas on the fly. After a schema is created, InfluxDB validates future write requests against it before accepting the data. Subsequent requests can add new fields on-the-fly, but can’t add new tags.

Write data in line protocol syntax

InfluxDB 3 Enterprise accepts data in line protocol syntax. The following code block is an example of time series data in line protocol syntax:

• cpu: the table name.
• host, region, applications: the tags. A tag set is an ordered, comma-separated list of key/value pairs where the values are strings.
• val, usage_percent, status: the fields. A field set is a comma-separated list of key/value pairs.
• timestamp: If you don’t specify a timestamp, InfluxData uses the time when data is written. The default precision is a nanosecond epoch. To specify a different precision, pass the precision parameter in your CLI command or API request.

cpu,host=Alpha,region=us-west,application=webserver val=1i,usage_percent=20.5,status="OK"
cpu,host=Bravo,region=us-east,application=database val=2i,usage_percent=55.2,status="OK"
cpu,host=Charlie,region=us-west,application=cache val=3i,usage_percent=65.4,status="OK"
cpu,host=Bravo,region=us-east,application=database val=4i,usage_percent=70.1,status="Warn"
cpu,host=Bravo,region=us-central,application=database val=5i,usage_percent=80.5,status="OK"
cpu,host=Alpha,region=us-west,application=webserver val=6i,usage_percent=25.3,status="Warn"

Write data using the CLI

To quickly get started writing data, you can use the influxdb3 CLI.

Example: write data using the influxdb3 CLI

Use the influxdb3 write command to write data to a database.

In the code samples, replace the following placeholders with your values:

• DATABASE_NAME: the name of the database to write to.
• AUTH_TOKEN: your database token with permission to write to the specified database

Write data via stdin

Pass data as quoted line protocol via standard input (stdin)–for example:

influxdb3 write \
  --database 
DATABASE_NAME
 \
  --token 
AUTH_TOKEN
 \
  --precision ns \
  --accept-partial \
'cpu,host=Alpha,region=us-west,application=webserver val=1i,usage_percent=20.5,status="OK"
cpu,host=Bravo,region=us-east,application=database val=2i,usage_percent=55.2,status="OK"
cpu,host=Charlie,region=us-west,application=cache val=3i,usage_percent=65.4,status="OK"
cpu,host=Bravo,region=us-east,application=database val=4i,usage_percent=70.1,status="Warn"
cpu,host=Bravo,region=us-central,application=database val=5i,usage_percent=80.5,status="OK"
cpu,host=Alpha,region=us-west,application=webserver val=6i,usage_percent=25.3,status="Warn"'

Write data from a file

Pass the --file option to write line protocol you have saved to a file–for example, save the sample line protocol to a file named server_data and then enter the following command:

influxdb3 write \
  --database 
DATABASE_NAME
 \
  --token 
AUTH_TOKEN
 \
  --precision ns \ 
  --accept-partial \
  --file path/to/server_data 

Replace the following placeholders with your values:

• DATABASE_NAME: the name of the database to write to.
• AUTH_TOKEN: your database token with permission to write to the specified database

Write data using the HTTP API

InfluxDB 3 Enterprise provides three write API endpoints that respond to HTTP POST requests. The /api/v3/write_lp endpoint is the recommended endpoint for writing data and provides additional options for controlling write behavior.

If you need to write data using InfluxDB v1.x or v2.x tools, use the compatibility API endpoints. Compatibility APIs work with Telegraf, InfluxDB v2.x and v1.x API client libraries, and other tools that support the v1.x or v2.x APIs.

POST /api/v3/write_lp?db=mydb&precision=nanosecond&accept_partial=true&no_sync=false

curl -v "http://localhost:8181/api/v3/write_lp?db=sensors&precision=auto" \
  --header 'Authorization: Bearer apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0==' \
  --data-raw 'home,room=Sunroom temp=96
home,room=Sunroom temp="hi"'

< HTTP/1.1 400 Bad Request
...
{
  "error": "partial write of line protocol occurred",
  "data": [
    {
      "original_line": "home,room=Sunroom temp=hi",
      "line_number": 2,
      "error_message": "invalid column type for column 'temp', expected iox::column_type::field::float, got iox::column_type::field::string"
    }
  ]
}

curl -v "http://localhost:8181/api/v3/write_lp?db=sensors&precision=auto&accept_partial=false" \
  --header 'Authorization: Bearer apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0==' \
  --data-raw 'home,room=Sunroom temp=96
home,room=Sunroom temp="hi"'

< HTTP/1.1 400 Bad Request
...
{
  "error": "parsing failed for write_lp endpoint",
  "data": {
    "original_line": "home,room=Sunroom temp=hi",
    "line_number": 2,
    "error_message": "invalid column type for column 'temp', expected iox::column_type::field::float, got iox::column_type::field::string"
  }
}

POST /api/v2/write?bucket=mydb&precision=ns

POST /write?db=mydb&precision=ns

Write responses

By default, InfluxDB acknowledges writes after flushing the WAL file to the object store (occurring every second). For high write throughput, you can send multiple concurrent write requests.

Use no_sync for immediate write responses

To reduce the latency of writes, use the no_sync write option, which acknowledges writes before WAL persistence completes. When no_sync=true, InfluxDB validates the data, writes the data to the WAL, and then immediately responds to the client, without waiting for persistence to the object store.

Using no_sync=true is best when prioritizing high-throughput writes over absolute durability.

• Default behavior (no_sync=false): Waits for data to be written to the object store before acknowledging the write. Reduces the risk of data loss, but increases the latency of the response.
• With no_sync=true: Reduces write latency, but increases the risk of data loss in case of a crash before WAL persistence.

Immediate write using the HTTP API

The no_sync parameter controls when writes are acknowledged–for example:

curl "http://localhost:8181/api/v3/write_lp?db=sensors&precision=auto&no_sync=true" \
  --header 'Authorization: Bearer apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0==' \
  --data-raw "home,room=Sunroom temp=96"

Create a database or table

To create a database without writing data, use the create subcommand–for example:

influxdb3 create database 
DATABASE_NAME
 \
  --token 
AUTH_TOKEN
 

Replace the following placeholders with your values:

• DATABASE_NAME: the name of the database to create
• AUTH_TOKEN: the admin token for your InfluxDB 3 Enterprise server

To learn more about a subcommand, use the -h, --help flag or view the InfluxDB 3 CLI reference:

influxdb3 create -h

Query data

InfluxDB 3 supports native SQL for querying, in addition to InfluxQL, an SQL-like language customized for time series queries.

The quickest way to get started querying is to use the influxdb3 CLI (which uses the Flight SQL API over HTTP2).

The query subcommand includes options to help ensure that the right database is queried with the correct permissions. Only the --database option is required, but depending on your specific setup, you may need to pass other options, such as host, port, and token.

Example: query “SHOW TABLES” on the servers database:

$ influxdb3 query --database servers "SHOW TABLES"
+---------------+--------------------+--------------+------------+
| table_catalog | table_schema       | table_name   | table_type |
+---------------+--------------------+--------------+------------+
| public        | iox                | cpu          | BASE TABLE |
| public        | information_schema | tables       | VIEW       |
| public        | information_schema | views        | VIEW       |
| public        | information_schema | columns      | VIEW       |
| public        | information_schema | df_settings  | VIEW       |
| public        | information_schema | schemata     | VIEW       |
+---------------+--------------------+--------------+------------+

Example: query the cpu table, limiting to 10 rows:

$ influxdb3 query --database servers "SELECT DISTINCT usage_percent, time FROM cpu LIMIT 10"
+---------------+---------------------+
| usage_percent | time                |
+---------------+---------------------+
| 63.4          | 2024-02-21T19:25:00 |
| 25.3          | 2024-02-21T19:06:40 |
| 26.5          | 2024-02-21T19:31:40 |
| 70.1          | 2024-02-21T19:03:20 |
| 83.7          | 2024-02-21T19:30:00 |
| 55.2          | 2024-02-21T19:00:00 |
| 80.5          | 2024-02-21T19:05:00 |
| 60.2          | 2024-02-21T19:33:20 |
| 20.5          | 2024-02-21T18:58:20 |
| 85.2          | 2024-02-21T19:28:20 |
+---------------+---------------------+

Query using the CLI for InfluxQL

InfluxQL is an SQL-like language developed by InfluxData with specific features tailored for leveraging and working with InfluxDB. It’s compatible with all versions of InfluxDB, making it a good choice for interoperability across different InfluxDB installations.

To query using InfluxQL, enter the influxdb3 query subcommand and specify influxql in the language option–for example:

influxdb3 query \
  --database 
DATABASE_NAME
 \
  --token <
AUTH_TOKEN
> \
  --language influxql \
  "SELECT DISTINCT usage_percent FROM cpu WHERE time >= now() - 1d"

Replace the following placeholders with your values:

• DATABASE_NAME: the name of the database to query
• AUTH_TOKEN: your database token with permission to query the specified database

Query using the API

InfluxDB 3 supports Flight (gRPC) APIs and an HTTP API. To query your database using the HTTP API, send a request to the /api/v3/query_sql or /api/v3/query_influxql endpoints. In the request, specify the database name in the db parameter and a query in the q parameter. You can pass parameters in the query string or inside a JSON object.

Use the format parameter to specify the response format: pretty, jsonl, parquet, csv, and json. Default is json.

Example: Query passing URL-encoded parameters

The following example sends an HTTP GET request with a URL-encoded SQL query:

curl -G "http://localhost:8181/api/v3/query_sql" \
  --header 'Authorization: Bearer 
AUTH_TOKEN
' \
  --data-urlencode "db=
DATABASE_NAME
" \
  --data-urlencode "q=select * from cpu limit 5"

Replace the following placeholders with your values:

• DATABASE_NAME: the name of the database to query
• AUTH_TOKEN: your database token with permission to query the specified database

Example: Query passing JSON parameters

The following example sends an HTTP POST request with parameters in a JSON payload:

curl http://localhost:8181/api/v3/query_sql \
  --data '{"db": "
DATABASE_NAME
", "q": "select * from cpu limit 5"}'

Replace the following placeholders with your values:

• DATABASE_NAME: the name of the database to query
• AUTH_TOKEN: your database token with permission to query the specified database

Query using the Python client

Use the InfluxDB 3 Python library to interact with the database and integrate with your application. We recommend installing the required packages in a Python virtual environment for your specific project.

To get started, install the influxdb3-python package.

pip install influxdb3-python

From here, you can connect to your database with the client library using just the host and **database name:

from influxdb_client_3 import InfluxDBClient3

client = InfluxDBClient3(
    token='
AUTH_TOKEN
',
    host='http://localhost:8181',
    database='
DATABASE_NAME
'
)

Replace the following placeholders with your values:

• DATABASE_NAME: the name of the database to query
• AUTH_TOKEN: your database token with permission to query the specified database

The following example shows how to query using SQL, and then use PyArrow to explore the schema and process results. To authorize the query, the example retrieves the database token from the INFLUXDB3_AUTH_TOKEN environment variable.

from influxdb_client_3 import InfluxDBClient3
import os

client = InfluxDBClient3(
    token=os.environ.get('INFLUXDB3_AUTH_TOKEN'),
    host='http://localhost:8181',
    database='servers'
)

# Execute the query and return an Arrow table
table = client.query(
    query="SELECT * FROM cpu LIMIT 10",
    language="sql"
)

print("\n#### View Schema information\n")
print(table.schema)

print("\n#### Use PyArrow to read the specified columns\n")
print(table.column('usage_active'))
print(table.select(['host', 'usage_active']))
print(table.select(['time', 'host', 'usage_active']))

print("\n#### Use PyArrow compute functions to aggregate data\n")
print(table.group_by('host').aggregate([]))
print(table.group_by('cpu').aggregate([('time_system', 'mean')]))

For more information about the Python client library, see the influxdb3-python repository in GitHub.

Query using InfluxDB 3 Explorer (Beta)

You can use the InfluxDB 3 Explorer web-based interface to query and visualize data, and administer your InfluxDB 3 Enterprise instance. For more information, see how to install InfluxDB 3 Explorer (Beta) using Docker and get started querying your data.

Last values cache

InfluxDB 3 Enterprise supports a last-n values cache which stores the last N values in a series or column hierarchy in memory. This gives the database the ability to answer these kinds of queries in under 10 milliseconds.

You can use the influxdb3 CLI to create a last value cache.

influxdb3 create last_cache \
  --token 
AUTH_TOKEN
  --database 
DATABASE_NAME
 \
  --table 
TABLE_NAME
 \
  
CACHE_NAME

Replace the following placeholders with your values:

• DATABASE_NAME: the name of the database to create the last values cache in
• AUTH_TOKEN: your admin token
• TABLE_NAME: the name of the table to create the last values cache in
• CACHE_NAME: Optionally, a name for the new cache

Consider the following cpu sample table:

The following command creates a last value cache named cpuCache:

influxdb3 create last_cache \
  --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \
  --database servers \
  --table cpu \
  --key-columns host,application \
  --value-columns usage_percent,status \
  --count 5 cpuCache

You can create a last values cache per time series, but be mindful of high cardinality tables that could take excessive memory.

Query a last values cache

To query data from the LVC, use the last_cache() function in your query–for example:

influxdb3 query \
  --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \
  --database servers \
  "SELECT * FROM last_cache('cpu', 'cpuCache') WHERE host = 'Bravo';"

Delete a last values cache

Use the influxdb3 CLI to delete a last values cache

influxdb3 delete last_cache \
  --token AUTH_TOKEN \
  --database 
DATABASE_NAME
 \
  --table TABLE \
  --cache-name 
CACHE_NAME

Replace the following placeholders with your values:

• AUTH_TOKEN: your admin token
• DATABASE_NAME: the name of the database to delete the last values cache from
• TABLE_NAME: the name of the table to delete the last values cache from
• CACHE_NAME: the name of the last values cache to delete

Distinct values cache

Similar to the last values cache, the database can cache in RAM the distinct values for a single column in a table or a hierarchy of columns. This is useful for fast metadata lookups, which can return in under 30 milliseconds. Many of the options are similar to the last value cache.

You can use the influxdb3 CLI to create a distinct values cache.

influxdb3 create distinct_cache \
  --token 
AUTH_TOKEN
 \
  --database 
DATABASE_NAME
 \
  --table TABLE \
  --columns COLUMNS \
  
CACHE_NAME

• DATABASE_NAME: the name of the database to create the last values cache in
• AUTH_TOKEN: your admin token
• TABLE_NAME: the name of the table to create the distinct values cache in
• CACHE_NAME: Optionally, a name for the new cache

Consider the following cpu sample table:

The following command creates a distinct values cache named cpuDistinctCache:

influxdb3 create distinct_cache \
  --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \
  --database servers \
  --table cpu \
  --columns host,application \
  cpuDistinctCache

Query a distinct values cache

To query data from the distinct values cache, use the distinct_cache() function in your query–for example:

influxdb3 query \
  --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \
  --database servers \
  "SELECT * FROM distinct_cache('cpu', 'cpuDistinctCache')"

Delete a distinct values cache

Use the influxdb3 CLI to delete a distinct values cache

influxdb3 delete distinct_cache \
  --token AUTH_TOKEN \
  --database 
DATABASE_NAME
 \
  --table TABLE \
  --cache-name 
CACHE_NAME

Replace the following placeholders with your values:

• AUTH_TOKEN: your admin token
• DATABASE_NAME: the name of the database to delete the distinct values cache from
• TABLE_NAME: the name of the table to delete the distinct values cache from
• CACHE_NAME: the name of the distinct values cache to delete

Python plugins and the processing engine

The InfluxDB 3 processing engine is an embedded Python VM for running code inside the database to process and transform data.

To activate the processing engine, pass the --plugin-dir <PLUGIN_DIR> option when starting the InfluxDB 3 Enterprise server. PLUGIN_DIR is your filesystem location for storing plugin files for the processing engine to run.

Plugin

A plugin is a Python function that has a signature compatible with a Processing engine trigger.

Trigger

When you create a trigger, you specify a plugin, a database, optional arguments, and a trigger-spec, which defines when the plugin is executed and what data it receives.

Trigger types

InfluxDB 3 provides the following types of triggers, each with specific trigger-specs:

• On WAL flush: Sends a batch of written data (for a specific table or all tables) to a plugin (by default, every second).
• On Schedule: Executes a plugin on a user-configured schedule (using a crontab or a duration); useful for data collection and deadman monitoring.
• On Request: Binds a plugin to a custom HTTP API endpoint at /api/v3/engine/<ENDPOINT_PATH>. The plugin receives the HTTP request headers and content, and can then parse, process, and send the data into the database or to third-party services.

Test, create, and trigger plugin code

Example: Python plugin for WAL rows

# This is the basic structure for Python plugin code that runs in the
# InfluxDB 3 Processing engine.

# When creating a trigger, you can provide runtime arguments to your plugin,
# allowing you to write generic code that uses variables such as monitoring
thresholds, environment variables, and host names.
#
# Use the following exact signature to define a function for the WAL flush
# trigger.
# When you create a trigger for a WAL flush plugin, you specify the database
# and tables that the plugin receives written data from on every WAL flush
# (default is once per second).
def process_writes(influxdb3_local, table_batches, args=None):
    # here you can see logging. for now this won't do anything, but soon 
    # we'll capture this so you can query it from system tables
    if args and "arg1" in args:
        influxdb3_local.info("arg1: " + args["arg1"])

    # here we're using arguments provided at the time the trigger was set up 
    # to feed into paramters that we'll put into a query
    query_params = {"host": "foo"}
    # here's an example of executing a parameterized query. Only SQL is supported. 
    # It will query the database that the trigger is attached to by default. We'll 
    # soon have support for querying other DBs.
    query_result = influxdb3_local.query("SELECT * FROM cpu where host = '$host'", query_params)
    # the result is a list of Dict that have the column name as key and value as 
    # value. If you run the WAL test plugin with your plugin against a DB that 
    # you've written data into, you'll be able to see some results
    influxdb3_local.info("query result: " + str(query_result))

    # this is the data that is sent when the WAL is flushed of writes the server 
    # received for the DB or table of interest. One batch for each table (will 
    # only be one if triggered on a single table)
    for table_batch in table_batches:
        # here you can see that the table_name is available.
        influxdb3_local.info("table: " + table_batch["table_name"])

        # example to skip the table we're later writing data into
        if table_batch["table_name"] == "some_table":
            continue

        # and then the individual rows, which are Dict with keys of the column names and values
        for row in table_batch["rows"]:
            influxdb3_local.info("row: " + str(row))

    # this shows building a line of LP to write back to the database. tags must go first and 
    # their order is important and must always be the same for each individual table. Then 
    # fields and lastly an optional time, which you can see in the next example below
    line = LineBuilder("some_table")\
        .tag("tag1", "tag1_value")\
        .tag("tag2", "tag2_value")\
        .int64_field("field1", 1)\
        .float64_field("field2", 2.0)\
        .string_field("field3", "number three")
    
    # this writes it back (it actually just buffers it until the completion of this function
    # at which point it will write everything back that you put in)
    influxdb3_local.write(line)

    # here's another example, but with us setting a nanosecond timestamp at the end
    other_line = LineBuilder("other_table")
    other_line.int64_field("other_field", 1)
    other_line.float64_field("other_field2", 3.14)
    other_line.time_ns(1302)

    # and you can see that we can write to any DB in the server
    influxdb3_local.write_to_db("mytestdb", other_line)

    # just some log output as an example
    influxdb3_local.info("done")

Test a plugin on the server

Test your InfluxDB 3 plugin safely without affecting written data. During a plugin test:

• A query executed by the plugin queries against the server you send the request to.
• Writes aren’t sent to the server but are returned to you.

To test a plugin, do the following:

1. Create a plugin directory–for example, /path/to/.influxdb/plugins

2. Start the InfluxDB server and include the --plugin-dir <PATH> option.

3. Save the example plugin code to a plugin file inside of the plugin directory. If you haven’t yet written data to the table in the example, comment out the lines where it queries.

4. To run the test, enter the following command with the following options:

   • --lp or --file: The line protocol to test
   • Optional: --input-arguments: A comma-delimited list of <KEY>=<VALUE> arguments for your plugin code

influxdb3 test wal_plugin \
--lp 
INPUT_LINE_PROTOCOL
 \
--input-arguments 
INPUT_ARGS
 \
--database 
DATABASE_NAME
 \
--token 
AUTH_TOKEN
 \
PLUGIN_FILENAME

Replace the following placeholders with your values:

• INPUT_LINE_PROTOCOL: the line protocol to test
• Optional: INPUT_ARGS: a comma-delimited list of <KEY>=<VALUE> arguments for your plugin code–for example, arg1=hello,arg2=world
• DATABASE_NAME: the name of the database to test against
• AUTH_TOKEN: the admin token for your InfluxDB 3 Enterprise server
• PLUGIN_FILENAME: the name of the plugin file to test

The command runs the plugin code with the test data, yields the data to the plugin code, and then responds with the plugin result. You can quickly see how the plugin behaves, what data it would have written to the database, and any errors. You can then edit your Python code in the plugins directory, and rerun the test. The server reloads the file for every request to the test API.

For more information, see influxdb3 test wal_plugin or run influxdb3 test wal_plugin -h.

With the plugin code inside the server plugin directory, and a successful test, you’re ready to create a plugin and a trigger to run on the server.

Example: Test, create, and run a plugin

The following example shows how to test a plugin, and then create the plugin and trigger:

# Test and create a plugin
# Requires:
#   - A database named `mydb` with a table named `foo`
#   - A Python plugin file named `test.py`
# Test a plugin
influxdb3 test wal_plugin \
  --lp "my_measure,tag1=asdf f1=1.0 123" \
  --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \
  --database sensors \
  --input-arguments "arg1=hello,arg2=world" \
  test.py

# Create a trigger that runs the plugin
influxdb3 create trigger \
  --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \
  --database sensors \
  --plugin test_plugin \
  --trigger-spec "table:foo" \
  --trigger-arguments "arg1=hello,arg2=world" \
  trigger1

After you have created a plugin and trigger, enter the following command to enable the trigger and have it run the plugin as you write data:

influxdb3 enable trigger \
  --token 
AUTH_TOKEN
 \
  --database 
DATABASE_NAME
 \
  
TRIGGER_NAME

Replace the following placeholders with your values:

• DATABASE_NAME: the name of the database to enable the trigger in
• AUTH_TOKEN: your admin token
• TRIGGER_NAME: the name of the trigger to enable

For example, to enable the trigger named trigger1 in the sensors database:

influxdb3 enable trigger \
  --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \
  --database sensors
  trigger1 

For more information, see Python plugins and the Processing engine.

Multi-server setup

InfluxDB 3 Enterprise is built to support multi-node setups for high availability, read replicas, and flexible implementations depending on use case.

High availability

Enterprise is architecturally flexible, giving you options on how to configure multiple servers that work together for high availability (HA) and high performance. Built on top of the diskless engine and leveraging the Object store, an HA setup ensures that if a node fails, you can still continue reading from, and writing to, a secondary node.

A two-node setup is the minimum for basic high availability, with both nodes having read-write permissions.

In a basic HA setup:

• Two nodes both write data to the same Object store and both handle queries
• Node 1 and Node 2 are read replicas that read from each other’s Object store directories
• One of the nodes is designated as the Compactor node

The following examples show how to configure and start two nodes for a basic HA setup.

• Node 1 is for compaction (passes compact in --mode)
• Node 2 is for ingest and query

## NODE 1

# Example variables
# node-id: 'host01'
# cluster-id: 'cluster01'
# bucket: 'influxdb-3-enterprise-storage'

influxdb3 serve \
  --node-id host01 \
  --cluster-id cluster01 \
  --mode ingest,query,compact \
  --object-store s3 \
  --bucket influxdb-3-enterprise-storage \
  --http-bind localhost:8181 \
  --aws-access-key-id <AWS_ACCESS_KEY_ID> \
  --aws-secret-access-key <AWS_SECRET_ACCESS_KEY>

## NODE 2

# Example variables
# node-id: 'host02'
# cluster-id: 'cluster01'
# bucket: 'influxdb-3-enterprise-storage'

influxdb3 serve \
  --node-id host02 \
  --cluster-id cluster01 \
  --mode ingest,query \
  --object-store s3 \
  --bucket influxdb-3-enterprise-storage \
  --http-bind localhost:8282 \
  --aws-access-key-id AWS_ACCESS_KEY_ID \
  --aws-secret-access-key AWS_SECRET_ACCESS_KEY

After the nodes have started, querying either node returns data for both nodes, and NODE 1 runs compaction. To add nodes to this setup, start more read replicas with the same cluster ID.

High availability with a dedicated Compactor

Data compaction in InfluxDB 3 is one of the more computationally expensive operations. To ensure that your read-write nodes don’t slow down due to compaction work, set up a compactor-only node for consistent and high performance across all nodes.

The following examples show how to set up high availability with a dedicated Compactor node:

1. Start two read-write nodes as read replicas, similar to the previous example.

   ## NODE 1 — Writer/Reader Node #1
   
   # Example variables
   # node-id: 'host01'
   # cluster-id: 'cluster01'
   # bucket: 'influxdb-3-enterprise-storage'
   
   influxdb3 serve \
     --node-id host01 \
     --cluster-id cluster01 \
     --mode ingest,query \
     --object-store s3 \
     --bucket influxdb-3-enterprise-storage \
     --http-bind localhost:8181 \
     --aws-access-key-id <AWS_ACCESS_KEY_ID> \
     --aws-secret-access-key <AWS_SECRET_ACCESS_KEY>
   

   ## NODE 2 — Writer/Reader Node #2
   
   # Example variables
   # node-id: 'host02'
   # cluster-id: 'cluster01'
   # bucket: 'influxdb-3-enterprise-storage'
   
   influxdb3 serve \
     --node-id host02 \
     --cluster-id cluster01 \
     --mode ingest,query \
     --object-store s3 \
     --bucket influxdb-3-enterprise-storage \
     --http-bind localhost:8282 \
     --aws-access-key-id <AWS_ACCESS_KEY_ID> \
     --aws-secret-access-key <AWS_SECRET_ACCESS_KEY>
   

2. Start the dedicated compactor node with the --mode=compact option to ensure the node only runs compaction.

   ## NODE 3 — Compactor Node
   
   # Example variables
   # node-id: 'host03'
   # cluster-id: 'cluster01'
   # bucket: 'influxdb-3-enterprise-storage'
   
   influxdb3 serve \
     --node-id host03 \
     --cluster-id cluster01 \
     --mode compact \
     --object-store s3 \
     --bucket influxdb-3-enterprise-storage \
     --aws-access-key-id <AWS_ACCESS_KEY_ID> \
     --aws-secret-access-key <AWS_SECRET_ACCESS_KEY>
   

High availability with read replicas and a dedicated Compactor

For a robust and effective setup for managing time-series data, you can run ingest nodes alongside read-only nodes and a dedicated Compactor node.

1. Start ingest nodes by assigning them the ingest mode. To achieve the benefits of workload isolation, you’ll send only write requests to these ingest nodes. Later, you’ll configure the read-only nodes.

   ## NODE 1 — Writer Node #1
   
   # Example variables
   # node-id: 'host01'
   # cluster-id: 'cluster01'
   # bucket: 'influxdb-3-enterprise-storage'
   
   influxdb3 serve \
     --node-id host01 \
     --cluster-id cluster01 \
     --mode ingest \
     --object-store s3 \
     --bucket influxdb-3-enterprise-storage \
     --http-bind localhost:8181 \
     --aws-access-key-id <AWS_ACCESS_KEY_ID> \
     --aws-secret-access-key <AWS_SECRET_ACCESS_KEY>
   

## NODE 2 — Writer Node #2

# Example variables
# node-id: 'host02'
# cluster-id: 'cluster01'
# bucket: 'influxdb-3-enterprise-storage'

influxdb3 serve \
  --node-id host02 \
  --cluster-id cluster01 \
  --mode ingest \
  --object-store s3 \
  --bucket influxdb-3-enterprise-storage \
  --http-bind localhost:8282 \
  --aws-access-key-id <AWS_ACCESS_KEY_ID> \
  --aws-secret-access-key <AWS_SECRET_ACCESS_KEY>

2. Start the dedicated Compactor node with compact.

   ## NODE 3 — Compactor Node
   
   # Example variables
   # node-id: 'host03'
   # cluster-id: 'cluster01'
   # bucket: 'influxdb-3-enterprise-storage'
   
   influxdb3 serve \
    --node-id host03 \
    --cluster-id cluster01 \
    --mode compact \
    --object-store s3 \
    --bucket influxdb-3-enterprise-storage \
    --aws-access-key-id <AWS_ACCESS_KEY_ID> \
     <AWS_SECRET_ACCESS_KEY>
   

3. Finally, start the query nodes as read-only with --mode query.

   ## NODE 4 — Read Node #1
   
   # Example variables
   # node-id: 'host04'
   # cluster-id: 'cluster01'
   # bucket: 'influxdb-3-enterprise-storage'
   
   influxdb3 serve \
     --node-id host04 \
     --cluster-id cluster01 \
     --mode query \
     --object-store s3 \
     --bucket influxdb-3-enterprise-storage \
     --http-bind localhost:8383 \
     --aws-access-key-id <AWS_ACCESS_KEY_ID> \
     --aws-secret-access-key <AWS_SECRET_ACCESS_KEY>
   

   ## NODE 5 — Read Node #2
   
   # Example variables
   # node-id: 'host05'
   # cluster-id: 'cluster01'
   # bucket: 'influxdb-3-enterprise-storage'
   
   influxdb3 serve \
    --node-id host05 \
    --cluster-id cluster01 \
    --mode query \
    --object-store s3 \
    --bucket influxdb-3-enterprise-storage \
    --http-bind localhost:8484 \
    --aws-access-key-id <AWS_ACCESS_KEY_ID> \
     <AWS_SECRET_ACCESS_KEY>
   

Congratulations, you have a robust setup for workload isolation using InfluxDB 3 Enterprise.

Writing and querying for multi-node setups

You can use the default port 8181 for any write or query, without changing any of the commands.

# Example querying a specific host
# HTTP-bound Port: 8585
influxdb3 query \
  --host 
http://localhost:8585
  --token 
AUTH_TOKEN
 \
  --database 
DATABASE_NAME
 "
QUERY
" 

Replace the following placeholders with your values:

• http://localhost:8585: the host and port of the node to query
• AUTH_TOKEN: your database tokenwith permission to query the specified database
• DATABASE_NAME: the name of the database to query
• QUERY: the SQL or InfluxQL query to run against the database

File index settings

To accelerate performance on specific queries, you can define non-primary keys to index on, which helps improve performance for single-series queries. This feature is only available in InfluxDB 3 Enterprise and is not available in Core.

Create a file index

# Example variables on a query
# HTTP-bound Port: 8585

influxdb3 create file_index \
  --host http://localhost:8585 \
  --token 
AUTH_TOKEN
 \
  --database 
DATABASE
_NAME \
  --table 
TABLE
_NAME \
  
COLUMNS

Delete a file index

influxdb3 delete file_index \
  --host http://localhost:8585 \
  --database 
DATABASE
_NAME \
  --table 
TABLE
_NAME \

Replace the following placeholders with your values:

• AUTH_TOKEN: your admin token
• DATABASE_NAME: the name of the database to create the file index in
• TABLE_NAME: the name of the table to create the file index in
• COLUMNS: a comma-separated list of columns to index on, for example, host,application