Documentation

Set up prerequisites

InfluxDB Clustered requires the following prerequisite external dependencies:

  • kubectl command line tool
  • Kubernetes cluster
  • kubecfg kubit operator
  • Kubernetes ingress controller
  • Object storage: AWS S3 or S3-compatible storage (including Google Cloud Storage or Azure Blob Storage) to store the InfluxDB Parquet files.
  • PostgreSQL-compatible database (AWS Aurora, hosted PostgreSQL, etc.): Stores the InfluxDB Catalog.
  • Local or attached storage: Stores the Write-Ahead Log (WAL) for InfluxDB Ingesters.

The following walks through preparing these prerequisites.

Install kubectl

Kubernetes provides the kubectl command line tool for communicating with a Kubernetes cluster’s control plane. kubectl is used to manage your InfluxDB cluster.

Follow instructions to install kubectl on your local machine:

InfluxDB Clustered Kubernetes deployments require kubectl 1.27 or higher.

Set up your Kubernetes cluster

  1. Deploy a Kubernetes cluster. The deployment process depends on your Kubernetes environment or Kubernetes cloud provider. Refer to the Kubernetes documentation or your cloud provider’s documentation for information about deploying a Kubernetes cluster.

  2. Ensure kubectl can connect to your Kubernetes cluster. Your Manage kubeconfig file defines cluster connection credentials.

  3. Create two namespaces–influxdb and kubit. Use kubectl create namespace to create the namespaces:

    kubectl create namespace influxdb && \
    kubectl create namespace kubit
    
  4. Install an ingress controller in the cluster and a mechanism to obtain a valid TLS certificate (for example: cert-manager or provide the certificate PEM manually out of band). To use the InfluxDB-specific ingress controller, install Ingress NGINX.

  5. Ensure your Kubernetes cluster can access the InfluxDB container registry, or, if running in an air-gapped environment, a local container registry to which you can copy the InfluxDB images.

Cluster sizing recommendation

As a starting point for a production workload, InfluxData recommends the following sizing for InfluxDB Clustered components:

  • Catalog (PostgreSQL-compatible database) (x1):
  • Ingesters and Routers (x3):
    • EC2 m6i.2xlarge (8 CPU, 32 GB RAM)
    • Local storage: mimumum of 2 GB (high-speed SSD)
  • Queriers (x3):
    • EC2 m6i.2xlarge (8 CPU, 32 GB RAM)
  • Compactors (x1):
    • EC2 m6i.2xlarge (8 CPU, 32 GB RAM)
  • Kubernetes Control Plane (x1):
    • EC2 t3.large (2 CPU, 8 GB RAM)
  • Catalog (PostgreSQL-compatible database) (x1):
  • Ingesters and Routers (x3):
    • GCE c2-standard-8 (8 CPU, 32 GB RAM)
    • Local storage: mimumum of 2 GB (high-speed SSD)
  • Queriers (x3):
    • GCE c2-standard-8 (8 CPU, 32 GB RAM)
  • Compactors (x1):
    • GCE c2-standard-8 (8 CPU, 32 GB RAM)
  • Kubernetes Control Plane (x1):
    • GCE c2d-standard-2 (2 CPU, 8 GB RAM)
  • Catalog (PostgreSQL-compatible database) (x1):
  • Ingesters and Routers (x3):
    • Standard_D8s_v3 (8 CPU, 32 GB RAM)
    • Local storage: mimumum of 2 GB (high-speed SSD)
  • Queriers (x3):
    • Standard_D8s_v3 (8 CPU, 32 GB RAM)
  • Compactors (x1):
    • Standard_D8s_v3 (8 CPU, 32 GB RAM)
  • Kubernetes Control Plane (x1):
    • Standard_B2ms (2 CPU, 8 GB RAM)
  • Catalog (PostgreSQL-compatible database) (x1):
    • CPU: 4-8 cores
    • RAM: 16-32 GB
  • Ingesters and Routers (x3):
    • CPU: 8 cores
    • RAM: 32 GB
    • Local storage: 2 GB (high-speed SSD)
  • Queriers (x3):
    • CPU: 8 cores
    • RAM: 32 GB
  • Compactors (x1):
    • CPU: 8 cores
    • RAM: 32 GB
  • Kubernetes Control Plane (x1):
    • CPU: 2 cores
    • RAM: 8 GB

Your sizing may need to be different based on your environment, cloud provider, and workload, but this is a reasonable starting size for your initial testing.

Install the kubecfg kubit operator

The kubecfg kubit operator (maintained by InfluxData) simplifies the installation and management of the InfluxDB Clustered package. It manages the application of the jsonnet templates used to install, manage, and update an InfluxDB cluster.

The InfluxDB Clustered Helm chart includes the kubit operator

If using the InfluxDB Clustered Helm chart to deploy your InfluxDB cluster, you do not need to install the kubit operator separately. The Helm chart installs the kubit operator.

Use kubectl to install the kubecfg kubit operator v0.0.18 or later.

kubectl apply -k 'https://github.com/kubecfg/kubit//kustomize/global?ref=v0.0.19'

Set up a Kubernetes ingress controller

Kubernetes ingress routes HTTP/S requests to services within the cluster and requires deploying an ingress controller. You can provide your own ingress or you can install Nginx Ingress Controller to use the InfluxDB-defined ingress.

InfluxDB Clustered components use gRPC/HTTP2 protocols. If using an external load balancer, you may need to explicitly enable these protocols on your load balancers.

Set up your object store

InfluxDB Clustered supports AWS S3 or S3-compatible storage (including Google Cloud Storage, Azure Blob Storage, and MinIO) for storing InfluxDB Parquet files. Refer to your object storage provider’s documentation for information about setting up an object store:

* This list does not represent all S3-compatible object stores that work with InfluxDB Clustered. Other S3-compatible object stores should work as well.

Object storage recommendations

We strongly recommend the following:

  • Enable object versioning

    Enable object versioning in your object store. Refer to your object storage provider’s documentation for information about enabling object versioning.

  • Run the object store in a separate namespace or outside of Kubernetes

    Run the Object store in a separate namespace from InfluxDB or external to Kubernetes entirely. Doing so makes management of the InfluxDB cluster easier and helps to prevent accidental data loss. While deploying everything in the same namespace is possible, we do not recommend it for production environments.

Configure object storage permissions

Ensure the identity you use to connect to your S3-compatible object store has the correct permissions to allow InfluxDB to perform all the actions it needs to.

View example AWS S3 access policy

View the requirements for Google Cloud Storage

View the requirements for Azure Blob Storage

To configure permissions with MinIO, use the example AWS access policy.

Set up your PostgreSQL-compatible database

The InfluxDB Catalog that stores metadata related to your time series data requires a PostgreSQL or PostgreSQL-compatible database (AWS Aurora, hosted PostgreSQL, etc.). The process for installing and setting up your PostgreSQL-compatible database depends on the database and database provider you use. Refer to your database’s or provider’s documentation for setting up your PostgreSQL-compatible database.

PostgreSQL-compatible database requirements

  • PostgreSQL versions 13.8–14.6.
  • Minimum of 4 GB of memory or equivalent provider-specific units.
  • To avoid conflicts and prevent issues caused by shared usage with other applications, ensure that your PostgreSQL-compatible instance is dedicated exclusively to InfluxDB.

We strongly recommended running the PostgreSQL-compatible database in a separate namespace from InfluxDB or external to Kubernetes entirely. Doing so makes management of the InfluxDB cluster easier and helps to prevent accidental data loss.

While deploying everything in the same namespace is possible, we do not recommend it for production environments.

Set up local or attached storage

The InfluxDB Ingester needs local or attached storage to store the Write-Ahead Log (WAL). The read and write speed of the attached storage affects the write performance of the Ingester, so the faster the storage device, the better your write performance will be. The recommended minimum size of the local storage is 2 gibibytes (2Gi).

Installation and setup of local or attached storage depends on your underlying hardware or cloud provider. Refer to your provider’s documentation for information about installing and configuring local storage.


Was this page helpful?

Thank you for your feedback!


The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Read more

InfluxDB v3 enhancements and InfluxDB Clustered is now generally available

New capabilities, including faster query performance and management tooling advance the InfluxDB v3 product line. InfluxDB Clustered is now generally available.

InfluxDB v3 performance and features

The InfluxDB v3 product line has seen significant enhancements in query performance and has made new management tooling available. These enhancements include an operational dashboard to monitor the health of your InfluxDB cluster, single sign-on (SSO) support in InfluxDB Cloud Dedicated, and new management APIs for tokens and databases.

Learn about the new v3 enhancements


InfluxDB Clustered general availability

InfluxDB Clustered is now generally available and gives you the power of InfluxDB v3 in your self-managed stack.

Talk to us about InfluxDB Clustered