Documentation

Manage Kapacitor subscriptions

Kapacitor is tightly integrated with InfluxDB through the use of InfluxDB subscriptions, local or remote endpoints to which all data written to InfluxDB is copied. Kapacitor subscribes to InfluxDB allowing it to capture, manipulate, and act on your data.

How Kapacitor subscriptions work

Kapacitor allows you to manipulate and act on data as it is written into InfluxDB. Rather than querying InfluxDB for data (except when using the BatchNode), all data is copied to your Kapacitor server or cluster through an InfluxDB subscription. This reduces the query load on InfluxDB and isolates overhead associated with data manipulation to your Kapacitor server or cluster.

On startup, Kapacitor checks for a subscription in InfluxDB with a name matching the Kapacitor server or cluster ID. This ID is stored inside of /var/lib/kapacitor/. If the ID file doesn’t exist on startup, Kapacitor creates it. If a subscription matching the Kapacitor ID doesn’t exist in InfluxDB, Kapacitor creates a new subscription in InfluxDB. This process ensures that when Kapacitor stops, it reconnects to the same subscription on restart as long as the contents of /var/lib/kapacitor/ remain intact.

InfluxDB user must have admin privileges

The InfluxDB user used to create subscriptions for Kapacitor must have admin privileges. Configure the InfluxDB user to use with the [influxdb].username setting in your Kapacitor configuration file.

The directory in which Kapacitor stores its ID can be configured with the data-dir root configuration option in the kapacitor.conf.

Kapacitor IDs in containerized or ephemeral filesystems

In containerized environments, filesystems are considered ephemeral and typically do not persist between container stops and restarts. If /var/lib/kapacitor/ is not persisted, Kapacitor will create a new InfluxDB subscription on startup, resulting in unnecessary “duplicate” subscriptions. You will then need to manually drop the unnecessary subscriptions.

To avoid this, persist the /var/lib/kapacitor directory. Many persistence strategies are available and which to use depends on your specific architecture and containerization technology.

Configure Kapacitor subscriptions

Kapacitor subscription configuration options are available under the [[influxdb]] section in the kapacitor.conf. Below is an example of subscription-specific configuration options followed by a description of each.

Example Kapacitor subscription configuration

[[influxdb]]

  # ...

  disable-subscriptions = false
  subscription-mode = "server"
  subscription-protocol = "http"
  subscriptions-sync-interval = "1m0s"

  # ...

  [influxdb.subscriptions]
    my_database1 = [ "default", "longterm" ]
  [influxdb.excluded-subscriptions]
    my_database2 = [ "default", "shortterm" ]

disable-subscriptions

Set to true to disable all subscriptions.

subscription-mode

Defines the subscription mode of Kapacitor. Available options:

  • "server"
  • "cluster"

The default setting is cluster.

subscription-protocol

Defines which protocol to use for subscriptions. Available options:

  • "udp"
  • "http"
  • "https"

The default setting is http.

[influxdb.subscriptions]

Defines a set of databases and retention policies to subscribe to. If empty, Kapacitor will subscribe to all databases and retention policies except for those listed in [influxdb.excluded-subscriptions].

[influxdb.subscriptions]
  # Pattern:
  db_name = <list of retention policies>

  # Example:
  my_database = [ "default", "longterm" ]

[influxdb.excluded-subscriptions]

Defines a set of databases and retention policies to exclude from subscriptions.

[influxdb.excluded-subscriptions]
  # Pattern:
  db_name = <list of retention policies>

  # Example:
  my_database = [ "default", "longterm" ]

Only one of [influxdb.subscriptions] or [influxdb.excluded-subscriptions] need be defined. They essentially fulfill the same purpose in different ways, but specific use cases do lend themselves to one or the other.

Troubleshooting

View the Kapacitor server or cluster ID

There are two ways to view your Kapacitor server or cluster ID:

  1. View the contents of /var/lib/kapacitor/server.id or /var/lib/kapacitor/cluster.id.

    The location of ID files depends on your operating system and the data-dir setting in your kapacitor.conf.

  2. Run the following command:

    kapacitor stats general
    

    The server and cluster IDs are included in the output.

Duplicate Kapacitor subscriptions

Duplicate Kapacitor subscriptions are often caused by the contents of /var/lib/kapacitor not persisting between restarts as described above. The solution is to ensure the contents of this director are persisted. Any duplicate Kapacitor subscriptions already created will need to be manually removed.


Was this page helpful?

Thank you for your feedback!


The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Read more

InfluxDB v3 enhancements and InfluxDB Clustered is now generally available

New capabilities, including faster query performance and management tooling advance the InfluxDB v3 product line. InfluxDB Clustered is now generally available.

InfluxDB v3 performance and features

The InfluxDB v3 product line has seen significant enhancements in query performance and has made new management tooling available. These enhancements include an operational dashboard to monitor the health of your InfluxDB cluster, single sign-on (SSO) support in InfluxDB Cloud Dedicated, and new management APIs for tokens and databases.

Learn about the new v3 enhancements


InfluxDB Clustered general availability

InfluxDB Clustered is now generally available and gives you the power of InfluxDB v3 in your self-managed stack.

Talk to us about InfluxDB Clustered