Enforce a schema

By default, InfluxDB 3 Enterprise databases use implicit schema mode: a write that references a new table or column creates it. Explicit schema mode inverts that: tables and columns must be declared through the configuration API before you can write to them, and a write that references anything undeclared is rejected.

Explicit schema mode is available in InfluxDB 3 Enterprise only. InfluxDB 3 Core rejects a request to create a database with explicit schema mode with HTTP status 400 (explicit schema mode is only available in InfluxDB 3 Enterprise).

• Create a database with explicit schema mode
• Declare tables and columns
• Write to an explicit database
• Partial writes
• Evolve a declared schema
• Propagation across a cluster
• Limits
• Upgrade considerations

Create a database with explicit schema mode

Schema mode is set when you create a database and can’t be changed afterward.

influxdb3 create database \
  --schema-mode explicit \
  --token 
AUTH_TOKEN
 \
  
DATABASE_NAME

or

curl --request POST "http://localhost:8181/api/v3/configure/database" \
  --header "Content-Type: application/json" \
  --header "Authorization: Bearer 
AUTH_TOKEN
" \
  --data '{
    "db": "
DATABASE_NAME
",
    "schema_mode": "explicit"
  }'

Replace the following:

• DATABASE_NAME: the name of the database to create
• AUTH_TOKEN: your admin token

retention_period still works alongside schema_mode. If you omit schema_mode or set it to implicit, the database uses the default, unenforced behavior. As a result, existing scripts and clients keep working unchanged.

Declare tables and columns

Use the same influxdb3 create table command you’d use in an implicit database to declare a table’s tag and field columns:

influxdb3 create table \
  --database 
DATABASE_NAME
 \
  --token 
AUTH_TOKEN
 \
  --tags host,region \
  --fields usage:float64,online:bool \
  
TABLE_NAME

or

curl --request POST "http://localhost:8181/api/v3/configure/table" \
  --header "Authorization: Bearer 
AUTH_TOKEN
" \
  --header "Content-Type: application/json" \
  --data '{
    "db": "
DATABASE_NAME
",
    "table": "
TABLE_NAME
",
    "tags": ["host", "region"],
    "fields": [
      {"name": "usage", "type": "float64"},
      {"name": "online", "type": "bool"}
    ]
  }'

This is the same POST /api/v3/configure/table endpoint that declares tables in an implicit database. Explicit mode only changes what happens when a write later references a table or column that isn’t declared. A declared table always gets its time column automatically.

For field type options, see Field data types.

Write to an explicit database

Line protocol that matches the declared schema is accepted as usual:

influxdb3 write --database 
DATABASE_NAME
 '
TABLE_NAME
,host=a,region=west usage=1.0,online=true'

A write that names an undeclared table or column is rejected. An undeclared column error names the database, the table, the column, and the column type. An undeclared table error names the database and the table:

column 'rack' (iox::column_type::tag) is not defined in table 'TABLE_NAME' of database 'DATABASE_NAME',
which uses explicit schemas; add the column with the /api/v3/configure/table API before writing to it

table 'other_table' is not defined in database 'DATABASE_NAME', which uses explicit schemas;
create the table with the /api/v3/configure/table API before writing to it

Both errors return HTTP status 400.

A few things to know about rejections:

• A declared column written at the wrong type is rejected the same way it is in an implicit database, with the existing InvalidColumnType error. Explicit mode adds nothing here. The error has the following form:

  invalid column type for column '<column>', expected <expected>, got <got>

• A declared tag written as a field, or a field written as a tag, is a wrong type on a declared column. It gets the same invalid column type error, not the undeclared-column error.

• Every write endpoint rejects the line with status 400. The v1 (/write), v2 (/api/v2/write), and v3 (/api/v3/write_lp) endpoints all enforce the declared schema. The endpoint changes only the error format and whether other lines in the batch are stored. See Partial writes.

Partial writes

A rejection is a line-protocol error like any other. As a result, the accept_partial parameter governs the rest of the batch on the /api/v3/write_lp endpoint:

• With accept_partial=false, the whole request fails and nothing is written.
• With accept_partial=true, the offending lines are reported and the rest of the batch is written.

The /api/v3/write_lp endpoint defaults accept_partial to true. A client that sends a batch with one undeclared column and no accept_partial parameter gets a 400 response and has its other lines stored.

The legacy /write and /api/v2/write endpoints fail the whole request when any line references an undeclared column, and they store nothing. Setting accept_partial=true doesn’t change this.

Other ways data arrives

The schema check also applies to data that doesn’t come through a line protocol write endpoint.

Every rejection message names the table or column and says to add it with the /api/v3/configure/table API.

For a schedule trigger, a failed write() doesn’t stop the trigger. For details, see Writes to explicit schema databases. For bulk import, declare every column in the file before you import. See Import data.

Declared columns written with the wrong type

A line that writes a declared column with the wrong type is a type error, not an undeclared-column error, so declaring more columns doesn’t fix it. The following results come from requests with accept_partial=false and one line per request. Each request returns HTTP status 400.

A declared tag written as a field (t,region=west host="x") returns the following message:

invalid column type for column 'host', expected iox::column_type::tag, got iox::column_type::field::string

A declared field written as a tag (t,host=a,usage=5 …) returns the following message:

invalid column type for column 'usage', expected iox::column_type::field::float, got iox::column_type::tag

A declared field written with the wrong type (usage="str" for a float64 field) returns the following message:

invalid column type for column 'usage', expected iox::column_type::field::float, got iox::column_type::field::string

Evolve a declared schema

Add tag and field columns to a declared table with the influxdb3 update table command or a PATCH request to /api/v3/configure/table:

influxdb3 update table \
  --database 
DATABASE_NAME
 \
  --token 
AUTH_TOKEN
 \
  --tags rack \
  --fields temp:float64 \
  
TABLE_NAME

or

curl --request PATCH "http://localhost:8181/api/v3/configure/table" \
  --header "Authorization: Bearer 
AUTH_TOKEN
" \
  --header "Content-Type: application/json" \
  --data '{
    "db": "
DATABASE_NAME
",
    "table": "
TABLE_NAME
",
    "tags": ["rack"],
    "fields": [{"name": "temp", "type": "float64"}]
  }'

The write that was rejected above is accepted once you declare the column. Schema evolution is add-only: you can declare new columns, but you can’t remove, rename, or retype an existing one. Declaring a column that already exists at its declared type is a no-op. Declaring it at a different type returns an error.

For more information, see Add columns to a table.

Propagation across a cluster

In a multi-node cluster, nodes poll the object store for catalog updates at --catalog-sync-interval (default 1s). Enforcement reads each node’s local view of the catalog. That view can lag the node that accepted a declaration by about one interval, and sometimes longer.

A client that declares a column on one node and immediately writes it to a different node can have that write rejected. The reason is that the second node hasn’t yet seen the declaration. The rejection is a per-line error the client can retry. It self-corrects once the writing node’s catalog catches up.

A node’s catalog advances synchronously with a declaration made on that node. That’s why a column declared on the same node you write to is never rejected this way.

Add-only evolution is what keeps this safe. A lagging node’s view of a declared column is never ahead of the catalog, only behind. As a result, the lag can reject a write that should have been accepted. It can never accept a write that should have been rejected.

Limits

• Explicit schema mode is fixed at creation. No API changes it afterward. An implicit database can’t be made explicit. An explicit database can’t be relaxed to implicit. If you need a different mode, create a new database and migrate your data.
• Deleting and recreating a database resets the mode. The mode belongs to the database that was created. As a result, recreating a database without schema_mode creates an implicit one.
• The _internal database is always implicit and can’t be created, deleted, or patched through the API.
• Schema evolution is add-only. Column removal, rename, and type change are out of scope.
• Value-level validation, per-column permissions, and retention rules are separate concerns and aren’t part of schema mode.

Upgrade considerations