---
title: Processing engine reference
description: The InfluxDB 3 Processing engine is an embedded Python virtual machine that runs inside InfluxDB 3 Core to execute Python code in response to triggers you define without requiring external application servers or middleware.
url: https://docs.influxdata.com/influxdb3/core/reference/processing-engine/
estimated_tokens: 2253
product: InfluxDB 3 Core
version: core
---

# Processing engine reference

The Processing engine is an embedded Python virtual machine that runs inside an InfluxDB 3 Core database server. It executes Python code in response to triggers and database events without requiring external application servers or middleware.

## Enable and disable the Processing Engine

The Processing Engine activates when [`--plugin-dir`](/influxdb3/core/reference/config-options/#plugin-dir) or `INFLUXDB3_PLUGIN_DIR` is configured. When not configured, the Python environment and PyO3 bindings aren’t initialized, and the server runs without Processing Engine functionality.

### Default behavior by deployment type

| Deployment | Default state | Configuration |
| --- | --- | --- |
| Docker images | Enabled | INFLUXDB3_PLUGIN_DIR=/plugins |
| DEB/RPM packages | Enabled | plugin-dir="/var/lib/influxdb3/plugins" |
| Binary/source | Disabled | No plugin-dir configured |

### Disable in Docker deployments

Docker images set `INFLUXDB3_PLUGIN_DIR=/plugins` by default.

Setting `INFLUXDB3_PLUGIN_DIR=""` (empty string) does **not** disable the Processing Engine. You must unset the variable, not set it to empty.

Use a custom entrypoint that unsets the variable:

```bash
docker run --entrypoint /bin/sh influxdb:3-core -c 'unset INFLUXDB3_PLUGIN_DIR && exec influxdb3 serve --object-store memory'
```

### Disable in systemd deployments (DEB/RPM)

The post-install script sets `plugin-dir="/var/lib/influxdb3/plugins"` in the TOML configuration. To disable the Processing Engine:

1. Edit the configuration file:
    
    ```bash
    sudo nano /etc/influxdb3/influxdb3-core.conf
    ```
    
2. Comment out or remove the `plugin-dir` line:
    
    ```toml
    # plugin-dir="/var/lib/influxdb3/plugins"
    ```
    
    Do not set `plugin-dir=""` (empty string)—you must remove or comment out the line.
    
3. Restart the service:
    
    ```bash
    sudo systemctl restart influxdb3-core
    ```
    

The `/var/lib/influxdb3/plugins` directory can remain on disk. The Processing Engine only activates based on the `plugin-dir` configuration, not directory existence.

### Benefits of disabling

When the Processing Engine is disabled:

-   The Python environment and PyO3 bindings are not initialized
-   Plugin-related operations return a “No plugin directory configured” error
-   The server runs with reduced resource usage

This is useful for deployments that don’t require plugin functionality and want a minimal server footprint.

## How it works

### Architecture

The Processing engine runs Python code directly within a InfluxDB 3 Core server process. This design provides high performance and direct access to database resources.

-   **Embedded execution**: Code runs in the same process space as the database server
-   **Direct data access**: Zero-copy access to data
-   **Event-driven**: Responds to database writes, scheduled events, and HTTP requests
-   **Cache integration**: Access to system caches including Last values and Distinct values

The Processing engine runs all plugins in the same Python process. Changes made by one plugin can affect other plugins.

### Event processing flow

When specific events occur in the database, the Processing engine handles them through a consistent sequence:

1. A **trigger** specific to the event type activates its plugin. The event types include:
    -   Data writes to specific tables or all tables
    -   Scheduled events (time-based or cron expressions)
    -   HTTP requests to configured endpoints
2. The engine loads the associated **plugin** specified in the trigger configuration
3. The plugin receives context data specific to the trigger type:
    -   Write triggers: the written data and table information
    -   Schedule triggers: the scheduled call time
    -   HTTP triggers: the request object with methods, headers, and body
4. The plugin processes the received data, can query the database, call external tools, and write and cache data in the database
5. Execution completes and the engine returns to waiting state

## Key components

### Trigger system

Triggers connect database events to Python code execution based on specific conditions:

-   **Data write triggers**: Execute on WAL flush events, when data is written to the object store, for a specific table or all tables in a database
-   **Scheduled triggers**: Run at intervals or according to cron expressions
-   **HTTP triggers**: Respond to HTTP requests to custom endpoints

### Plugin registry

The registry manages all Python code available to the Processing engine:

-   Indexes plugins by filename and location
-   Tracks which plugins are used by which triggers
-   Manages plugin versioning and dependencies

### Memory management

The Processing engine implements specialized memory handling to ensure stability and performance:

-   **Execution isolation**: Each plugin runs in its own context
-   **Cache system**: Maintains state between executions
-   **Resource limits**: Controls memory usage and execution time

## Performance characteristics

The Processing engine is designed for high-performance operation with minimal overhead:

-   **Low latency**: Activates triggers in sub-millisecond time
-   **Efficient access**: Accesses database directly without network overhead
-   **Controlled resources**: Limits memory and CPU usage through configuration
-   **Execution policies**: Offers synchronous or asynchronous processing options

## Reliability features

The Processing engine includes multiple features to ensure consistent and dependable execution:

-   **Error handling**: Configures behaviors for failure scenarios (log, retry, or disable)
-   **Execution tracking**: Tracks plugin performance and resource usage
-   **State persistence**: Persists cache state across server restarts

## Extension capabilities

Extend and customize the Processing engine through several built-in mechanisms:

-   **Package management**: Installs custom Python dependencies
-   **Plugin distribution**: Distributes plugins via Git repositories
-   **Shared API**: Provides consistent interface for database operations

For a step-by-step guide to setting up and using the Processing engine, see the [Getting started with plugins](/influxdb3/core/plugins/) documentation.

[plugin](/influxdb3/core/tags/plugin/) [database](/influxdb3/core/tags/database/) [python](/influxdb3/core/tags/python/)