Documentation

InfluxDB Clustered

Use the InfluxDB AppInstance resource configuration

Manage your InfluxDB Clustered deployments using Kubernetes and apply configuration settings using a YAML configuration file.

  • influxdb-docker-config.json: an authenticated Docker configuration file. The InfluxDB Clustered software is in a secure container registry. This file grants access to the collection of container images required to install InfluxDB Clustered.

  • A tarball that contains the following files:

    • app-instance-schema.json: Defines the schema for example-customer.yml to be used with Visual Studio Code (VS Code).

    • example-customer.yml: Configuration for your InfluxDB cluster that includes information about prerequisites.

      This documentation refers to a myinfluxdb.yml file that you copy from example-customer.yml and edit for your InfluxDB cluster.

Configuration data

When ready to install InfluxDB, have the following information available:

  • InfluxDB cluster hostname: the hostname Kubernetes uses to expose InfluxDB API endpoints
  • PostgreSQL-style data source name (DSN): used to access your PostgreSQL-compatible database that stores the InfluxDB Catalog.
  • Object store credentials (AWS S3 or S3-compatible)
    • Endpoint URL
    • Access key
    • Bucket name
    • Region (required for S3, may not be required for other object stores)
  • Local storage information (for ingester pods)
    • Storage class
    • Storage size
  • OAuth2 provider credentials
    • Client ID
    • JWKS endpoint
    • Device authorization endpoint
    • Token endpoint

InfluxDB is deployed to a Kubernetes namespace which, throughout the following installation procedure, is referred to as the target namespace. For simplicity, we assume this namespace is influxdb, however you may use any name you like.

The InfluxDB installation, update, and upgrade processes are driven by editing and applying a Kubernetes custom resource (CRD) called AppInstance. The AppInstance CRD is defined in a YAML file (use example-customer.yml as a template).

The AppInstance resource contains key information, such as:

  • Name of the target namespace
  • Version of the InfluxDB package
  • Reference to the InfluxDB container registry pull secrets
  • Hostname where the InfluxDB API is exposed
  • Parameters to connect to external prerequisites

Configure your cluster

  1. Create a cluster configuration file
  2. Create a namespace for InfluxDB
  3. Install kubecfg kubit operator
  4. Configure access to the InfluxDB container registry
  5. Set up cluster ingress
  6. Modify the configuration file to point to prerequisites
  7. Provide a custom certificate authority bundle (Optional)

Create a cluster configuration file

Copy the provided example-customer.yml file to create a new configuration file specific to your InfluxDB cluster. For example, myinfluxdb.yml.

cp example-customer.yml myinfluxdb.yml

Use VS Code to edit your configuration file

We recommend using Visual Studio Code (VS Code) to edit your myinfluxdb.yml configuration file due to its JSON Schema support, autocompletion, and validation features that ensure the best experience when editing your InfluxDB configuration. InfluxData provides an app-instance-schema.json JSON schema file that VS Code can use to validate your configuration settings.

Create a namespace for InfluxDB

Create a namespace for InfluxDB–for example, enter the following kubectl command in your terminal:

kubectl create namespace influxdb

If you use a namespace name other than influxdb, update the .metadata.namespace field in your myinfluxdb.yml to use your custom namespace name.

Install kubecfg kubit operator

The kubecfg kubit operator (maintained by InfluxData) simplifies the installation and management of the InfluxDB Clustered package. It manages the application of the jsonnet templates used to install, manage, and update an InfluxDB cluster.

Use kubectl to install the kubecfg kubit operator.

kubectl apply -k 'https://github.com/kubecfg/kubit//kustomize/global?ref=v0.0.19'

Configure access to the InfluxDB container registry

The provided influxdb-docker-config.json grants access to a collection of container images required to run InfluxDB Clustered. Your Kubernetes Cluster needs access to the container registry to pull down and install InfluxDB.

When pulling InfluxDB Clustered images, there are two main scenarios:

  • You have a Kubernetes cluster that can pull from the InfluxData container registry.
  • You run in an environment with no network interfaces (“air-gapped”) and you can only access a private container registry.

In both scenarios, you need a valid pull secret.

Public registry (non-air-gapped) Private registry (air-gapped)

Public registry (non-air-gapped)

To pull from the InfluxData registry, you need to create a Kubernetes secret in the target namespace.

kubectl create secret docker-registry gar-docker-secret \
  --from-file=.dockerconfigjson=influxdb-docker-config.json \
  --namespace influxdb

If successful, the output is the following:

secret/gar-docker-secret created

By default, this secret is named gar-docker-secret. If you change the name of this secret, you must also change the value of the imagePullSecret field in the AppInstance custom resource to match.

Private registry (air-gapped)

If your Kubernetes cluster can’t use a public network to download container images from our container registry, do the following:

  1. Copy the images from the InfluxDB registry to your own private registry.
  2. Configure your AppInstance resource with a reference to your private registry name.
  3. Provide credentials to your private registry.
Copy the images

We recommend using crane to copy images into your private registry.

  1. Install crane for your system.
  2. Use the following command to create a container registry secret file and retrieve the necessary secrets:
mkdir /tmp/influxdbsecret
cp influxdb-docker-config.json /tmp/influxdbsecret/config.json
DOCKER_CONFIG=/tmp/influxdbsecret \
  crane manifest \
  us-docker.pkg.dev/influxdb2-artifacts/clustered/influxdb:
PACKAGE_VERSION

Replace PACKAGE_VERSION with your InfluxDB Clustered package version.

If your Docker configuration is valid and you’re able to connect to the container registry, the command succeeds and the output is the JSON manifest for the Docker image, similar to the following:

View JSON manifest

If there’s a problem with the Docker configuration, crane won’t retrieve the manifest and the output is similar to the following error:

Error: fetching manifest us-docker.pkg.dev/influxdb2-artifacts/clustered/influxdb:<package-version>: GET https://us-docker.pkg.dev/v2/token?scope=repository%3Ainfluxdb2-artifacts%2Fclustered%2Finfluxdb%3Apull&service=: DENIED: Permission "artifactregistry.repositories.downloadArtifacts" denied on resource "projects/influxdb2-artifacts/locations/us/repositories/clustered" (or it may not exist)

The list of images that you need to copy is included in the package metadata. You can obtain it with any standard OCI image inspection tool–for example:

DOCKER_CONFIG=/tmp/influxdbsecret \
crane config \
  us-docker.pkg.dev/influxdb2-artifacts/clustered/influxdb:
PACKAGE_VERSION
 \
  | jq -r '.metadata["oci.image.list"].images[]' \
  > /tmp/images.txt

The output is a list of image names, similar to the following:

us-docker.pkg.dev/influxdb2-artifacts/idpe/idpe-cd-ioxauth@sha256:5f015a7f28a816df706b66d59cb9d6f087d24614f485610619f0e3a808a73864
us-docker.pkg.dev/influxdb2-artifacts/iox/iox@sha256:b59d80add235f29b806badf7410239a3176bc77cf2dc335a1b07ab68615b870c
...

Use crane to copy the images to your private registry:

</tmp/images.txt xargs -I% crane cp % 
REGISTRY_HOSTNAME
/%

Replace REGISTRY_HOSTNAME with the hostname of your private registry–for example:

myregistry.mydomain.io
Configure your AppInstance

Set the .spec.package.spec.images.registryOverride field in myinfluxdb.yml to the location of your private registry–for example:

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      images:
        registryOverride: 
REGISTRY_HOSTNAME
Provide credentials to your private registry

If your private container registry requires pull secrets to access images, you can create the required kubernetes secrets, and then configure them in your AppInstance resource.

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  imagePullSecrets:
    - name: 
PULL_SECRET_NAME

Set up cluster ingress

InfluxDB Clustered components use gRPC/HTTP2 protocols. If using an external load balancer, you may need to explicitly enable these protocols on your load balancers.

Kubernetes ingress routes HTTP/S requests to services within the cluster and requires deploying an ingress controller.

You can provide your own ingress or you can install Nginx Ingress Controller to use the InfluxDB-defined ingress.

If using the InfluxDB-defined ingress, add a valid TLS Certificate to the cluster as a secret. Provide the paths to the TLS certificate file and key file:

kubectl create secret tls ingress-tls \
  --namespace influxdb \
  --cert 
TLS_CERT_PATH
 \
  --key 
TLS_KEY_PATH

Replace the following:

  • TLS_CERT_PATH: Path to the certificate file on your local machine.
  • TLS_KEY_PATH: Path to the certificate secret key file on your local machine.

Provide the TLS certificate secret to the InfluxDB configuration in the Configure ingress step.

Modify the configuration file to point to prerequisites

Update your myinfluxdb.yml configuration file with credentials necessary to connect your cluster to your prerequisites.

Configure ingress

To configure ingress, provide values for the following fields in your myinfluxdb.yml configuration file:

  • spec.package.spec.ingress.hosts: Cluster hostnames

    Provide the hostnames that Kubernetes should use to expose the InfluxDB API endpoints. For example: cluster-host.com.

    You can provide multiple hostnames. The ingress layer accepts incoming requests for all listed hostnames. This can be useful if you want to have distinct paths for your internal and external traffic.

    You are responsible for configuring and managing DNS. Options include:

    • Manually managing DNS records
    • Using external-dns to synchronize exposed Kubernetes services and ingresses with DNS providers.

  • spec.package.spec.ingress.tlsSecretName: TLS certificate secret name

    Provide the name of the secret that contains your TLS certificate and key. The examples in this guide use the name ingress-tls.

    The tlsSecretName field is optional. You may want to use it if you already have a TLS certificate for your DNS name.

    Use cert-manager and Let’s Encrypt to manage TLS certificates

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
# ...
ingress:
  hosts:
    - cluster-host.com
  tlsSecretName: ingress-tls

Configure the object store

To connect your InfluxDB cluster to your object store, provide values for the following fields in your myinfluxdb.yml configuration file:

  • spec.package.spec.objectStore
    • .endpoint: Object storage endpoint URL
    • .allowHttp: Set to true to allow unencrypted HTTP connections
    • .accessKey.value: Object storage access key
    • .secretKey.value: Object storage secret key
    • .bucket: Object storage bucket name
    • .region: Object storage region
apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      objectStore:
        # URL for S3 Compatible object store
        endpoint: 
S3_URL


        # Set to true to allow communication over HTTP (instead of HTTPS)
        allowHttp: 'false'

        # S3 Access Key
        # This can also be provided as a valueFrom: secretKeyRef:
        accessKey:
          value: 
S3_ACCESS_KEY


        # S3 Secret Key
        # This can also be provided as a valueFrom: secretKeyRef:
        secretKey:
          value: 
S3_SECRET_KEY


        # Bucket that the Parquet files will be stored in
        bucket: 
S3_BUCKET_NAME


        # This value is required for AWS S3, it may or may not be required for other providers.
        region: 
S3_REGION

Replace the following:

  • S3_URL: Object storage endpoint URL
  • S3_ACCESS_KEY: Object storage access key
  • S3_SECRET_KEY: Object storage secret key
  • S3_BUCKET_NAME: Object storage bucket name
  • S3_REGION: Object storage region

Configure the catalog database

The InfluxDB catalog is a PostgreSQL-compatible relational database that stores metadata about your time series data. To connect your InfluxDB cluster to your PostgreSQL-compatible database, provide values for the following fields in your myinfluxdb.yml configuration file:

We recommend storing sensitive credentials, such as your PostgreSQL-compatible DSN, as secrets in your Kubernetes cluster.

Percent-encode special symbols in PostgreSQL DSNs

Special symbols in PostgreSQL DSNs should be percent-encoded to ensure they are parsed correctly by InfluxDB Clustered. This is important to consider when using DSNs containing auto-generated passwords which may include special symbols to make passwords more secure.

For example, the following DSN used in InfluxDB Clustered:

postgresql://postgres:meow#meow@my-fancy.cloud-database.party:5432/postgres

Would result in an error similar to:

Catalog DSN error: A catalog error occurred: unhandled external error: error with configuration: invalid port number

To fix this, percent-encode special symbols in the connection string:

postgresql://postgres:meow%23meow@my-fancy.cloud-database.party:5432/postgres

For more information, see the PostgreSQL Connection URI docs.

  • spec.package.spec.catalog.dsn.valueFrom.secretKeyRef
    • .name: Secret name
    • .key: Key in the secret that contains the DSN
apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      catalog:
        # A postgresql style DSN that points to a postgresql compatible database.
        # postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]
        dsn:
          valueFrom:
            secretKeyRef:
              name: 
SECRET_NAME

              key: 
SECRET_KEY

Replace the following:

  • SECRET_NAME: Name of the secret containing your PostgreSQL-compatible DSN
  • SECRET_KEY: Key in the secret that references your PostgreSQL-compatible DSN
PostgreSQL instances without TLS or SSL

If your PostgreSQL-compatible instance runs without TLS or SSL, you must include the sslmode=disable parameter in the DSN. For example:

postgres://username:passw0rd@mydomain:5432/influxdb?sslmode=disable

Configure local storage for ingesters

InfluxDB ingesters require local storage to store the Write Ahead Log (WAL) for incoming data. To connect your InfluxDB cluster to local storage, provide values for the following fields in your myinfluxdb.yml configuration file:

  • spec.package.spec.ingesterStorage
    • .storageClassName: Kubernetes storage class. This differs based on the Kubernetes environment and desired storage characteristics.
    • storage: Storage size. We recommend a minimum of 2 gibibytes (2Gi).
apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      ingesterStorage:
        storageClassName: 
STORAGE_CLASS

        storage: 
STORAGE_SIZE

Replace the following:

Configure your OAuth2 provider

InfluxDB Clustered uses OAuth2 to authenticate administrative access to your cluster. To connect your InfluxDB cluster to your OAuth2 provide, provide values for the following fields in your myinfluxdb.yml configuration file:

  • spec.package.spec.admin
    • identityProvider: Identity provider name. If using Microsoft Entra ID (formerly Azure Active Directory), set the name to azure.
    • jwksEndpoint: JWKS endpoint provide by your identity provider.
    • users: List of OAuth2 users to grant administrative access to your InfluxDB cluster. IDs are provided by your identity provider.

Below are examples for Keycloak, Auth0, and Microsoft Entra ID, but other OAuth2 providers should work as well:

Keycloak Auth0 Microsoft Entra ID

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      admin:
        identityProvider: keycloak
        jwksEndpoint: |-
          https://
KEYCLOAK_HOST
/auth/realms/
KEYCLOAK_REALM
/protocol/openid-connect/certs          
        users:
          # All fields are required but `firstName`, `lastName`, and `email` can be
          # arbitrary values. However, `id` must match the user ID provided by Keycloak.
          - id: 
KEYCLOAK_USER_ID

            firstName: Marty
            lastName: McFly
            email: mcfly@influxdata.com

Replace the following:

  • KEYCLOAK_HOST: Host and port of your Keycloak server
  • KEYCLOAK_REALM: Keycloak realm
  • KEYCLOAK_USER_ID: Keycloak user ID to grant InfluxDB administrative access to
apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      admin:
        identityProvider: auth0
        jwksEndpoint: |-
          https://
AUTH0_HOST
/.well-known/openid-configuration          
        users:
          - 
AUTH0_USER_ID

Replace the following:

  • AUTH0_HOST: Host and port of your Auth0 server
  • AUTH0_USER_ID: Auth0 user ID to grant InfluxDB administrative access to
apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      admin:
        identityProvider: azure
        jwksEndpoint: |-
          https://login.microsoftonline.com/
AZURE_TENANT_ID
/discovery/v2.0/keys          
        users:
          - 
AZURE_USER_ID

Replace the following:

Add users

Finally, to give users access to use influxctl, add the list of users to the spec.package.spec.admin.users field. See Manage users for more details.

Configure the size of your cluster

Default scale settings
  • 3 ingesters: Ensures redundancy on the write path.
  • 1 compactor: While you can have multiple compactors, it is more efficient to scale the compactor vertically (assign more CPU and memory) rather than horizontally (increase the number of compactors).
  • 1 querier: The optimal number of queriers depends on the number of concurrent queries you are likely to have and how long they take to execute.

The default values provide a good starting point for testing. Once you have your cluster up and running and are looking for scaling recommendations for your anticipated workload, please contact the InfluxData Support team.

Customize scale settings

To use custom scale settings for your InfluxDB cluster, edit values for the following fields in your myinfluxdb.yml. If omitted, your cluster uses the default scale settings.

  • spec.package.spec.resources
    • ingester.requests
      • cpu: CPU resource units to assign to ingesters
      • memory: Memory resource units to assign to ingesters
      • replicas: Number of ingester replicas to provision
    • compactor.requests
      • cpu: CPU resource units to assign to compactors
      • memory: Memory resource units to assign to compactors
      • replicas: Number of compactor replicas to provision
    • querier.requests
      • cpu: CPU resource units to assign to queriers
      • memory: Memory resource units to assign to queriers
      • replicas: Number of querier replicas to provision
    • router.requests
      • cpu: CPU resource units to assign to routers
      • memory: Memory resource units to assign to routers
      • replicas: Number of router replicas to provision
apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      # The following settings tune the various pods for their cpu/memory/replicas
      # based on workload needs. Only uncomment the specific resources you want
      # to change. Anything left commented will use the package default.
      resources:
        # The ingester handles data being written
        ingester:
          requests:
            cpu: 
INGESTER_CPU

            memory: 
INGESTER_MEMORY

            replicas: 
INGESTER_REPLICAS
 # Default is 3

        # The compactor reorganizes old data to improve query and storage efficiency.
        compactor:
          requests:
            cpu: 
COMPACTOR_CPU

            memory: 
COMPACTOR_MEMORY

            replicas: 
COMPACTOR_REPLICAS
 # Default is 1

        # The querier handles querying data.
        querier:
          requests:
            cpu: 
QUERIER_CPU

            memory: 
QUERIER_MEMORY

            replicas: 
QUERIER_REPLICAS
 # Default is 1

        # The router performs some api routing.
        router:
          requests:
            cpu: 
ROUTER_CPU

            memory: 
ROUTER_MEMORY

            replicas: 
ROUTER_REPLICAS
 # Default is 1

Provide a custom certificate authority bundle

InfluxDB attempts to make TLS connections to the services it depends on–notably, the Catalog and the Object store. InfluxDB validates certificates for all connections.

If you host dependent services yourself and you use a private or otherwise not well-known certificate authority to issue certificates to them, InfluxDB won’t recognize the issuer and can’t validate the certificates. To allow InfluxDB to validate the certificates from your custom CA, configure the AppInstance resource to use a PEM certificate bundle that contains your custom certificate authority chain.

  1. Use kubectl to create a config map that contains your PEM-formatted certificate bundle file. Your certificate authority administrator should provide you with a PEM-formatted bundle file.

    This PEM bundle file establishes a chain of trust for the external services that InfluxDB depends on. It’s not the certificate that InfluxDB uses to host its own TLS endpoints.

    In the example, replace /path/to/private_ca.pem with the path to your PEM-formatted certificate bundle file:

    kubectl --namespace influxdb create configmap custom-ca --from-file=certs.pem=/path/to/private_ca.pem

    Bundle multiple certificates

    You can append multiple certificates into the same bundle. This approach helps when you need to include intermediate certificates or explicitly include leaf certificates.

    Include certificates in the bundle in the following order:

    1. Leaf certificates
    2. Intermediate certificates required by leaf certificates
    3. Root certificate

  2. In myinfluxdb.yml, update the .spec.package.spec.egress field to refer to the config map that you generated in the preceding step–for example:

    spec:
  package:
    spec:
      egress:
        customCertificates:
          valueFrom:
            configMapKeyRef:
              key: ca.pem
              name: custom-ca

Was this page helpful?

Thank you for your feedback!

© 2024 InfluxData, Inc.

What is your InfluxDB cluster URL?

Enter your InfluxDB cluster URL and we’ll update code examples for you.

Enter cluster URL

Thank you for your feedback!

Let us know what we can do better:

Thank you!

No thanks

The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Read more

InfluxDB v3 enhancements and InfluxDB Clustered is now generally available

New capabilities, including faster query performance and management tooling advance the InfluxDB v3 product line. InfluxDB Clustered is now generally available.

InfluxDB v3 performance and features

The InfluxDB v3 product line has seen significant enhancements in query performance and has made new management tooling available. These enhancements include an operational dashboard to monitor the health of your InfluxDB cluster, single sign-on (SSO) support in InfluxDB Cloud Dedicated, and new management APIs for tokens and databases.

Learn about the new v3 enhancements

InfluxDB Clustered general availability

InfluxDB Clustered is now generally available and gives you the power of InfluxDB v3 in your self-managed stack.

Talk to us about InfluxDB Clustered