Use the InfluxDB AppInstance resource configuration

Configure your InfluxDB cluster by editing configuration options in the AppInstance resource provided by InfluxData. Resource configuration for your cluster includes the following:

• influxdb-docker-config.json: an authenticated Docker configuration file. The InfluxDB Clustered software is in a secure container registry. This file grants access to the collection of container images required to install InfluxDB Clustered.

• A tarball that contains the following files:

  • app-instance-schema.json: Defines the schema that you can use to validate example-customer.yml and your cluster configuration in tools like Visual Studio Code (VS Code).

  • example-customer.yml: Configuration for your InfluxDB cluster that includes information about prerequisites.

    The following sections refer to a myinfluxdb.yml file that you copy from example-customer.yml and edit for your InfluxDB cluster.

Configuration data

When ready to configure your InfluxDB cluster, have the following information available:

• InfluxDB cluster hostname: the hostname Kubernetes uses to expose InfluxDB API endpoints
• PostgreSQL-style data source name (DSN): used to access your PostgreSQL-compatible database that stores the InfluxDB Catalog.
• Object store credentials (AWS S3 or S3-compatible)
  • Endpoint URL
  • Access key
  • Bucket name
  • Region (required for S3, might not be required for other object stores)
• Local or attached storage information (for ingester pods)
  • Storage class
  • Storage size

You deploy InfluxDB Clustered to a Kubernetes namespace, referred to as the target namespace in the following procedures. For simplicity, we assume this namespace is influxdb–you can use any name you like.

To manage InfluxDB Clustered installation, updates, and upgrades, you edit
and apply a Kubernetes custom resource (CRD)
(AppInstance), which you define in a YAML file that conforms to the app-instance-schema.json schema.

InfluxDB Clustered includes example-customer.yml as a configuration template.

The AppInstance resource contains key information, such as:

• Name of the target namespace
• Version of the InfluxDB package
• Reference to the InfluxDB container registry pull secrets
• Hostname of your cluster’s InfluxDB API
• Parameters to connect to external prerequisites

Configure your cluster

1. Create a cluster configuration file
2. Configure access to the InfluxDB container registry
3. Modify the configuration file to point to prerequisites

Create a cluster configuration file

Copy the provided example-customer.yml file to create a new configuration file specific to your InfluxDB cluster. For example, myinfluxdb.yml.

cp example-customer.yml myinfluxdb.yml

Configure access to the InfluxDB container registry

The provided influxdb-docker-config.json grants access to a collection of container images required to run InfluxDB Clustered. Your Kubernetes Cluster needs access to the container registry to pull and install InfluxDB.

When pulling the InfluxDB Clustered image, you’re likely in one of two scenarios:

• Your Kubernetes cluster can pull from the InfluxData container registry.
• Your cluster runs in an environment without network interfaces (“air-gapped”) and can only access a private container registry.

In both scenarios, you need a valid pull secret.

kubectl create secret docker-registry gar-docker-secret \
  --from-file=.dockerconfigjson=influxdb-docker-config.json \
  --namespace influxdb

secret/gar-docker-secret created

mkdir /tmp/influxdbsecret
cp influxdb-docker-config.json /tmp/influxdbsecret/config.json
DOCKER_CONFIG=/tmp/influxdbsecret \
  crane manifest \
  us-docker.pkg.dev/influxdb2-artifacts/clustered/influxdb:
PACKAGE_VERSION

{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.kubecfg.bundle.config.v1+json",
    "digest": "sha256:6900d2f248e678176c68f3768e7e48958bb96a59232070ff31b3b018cf299aa7",
    "size": 8598
  },
  "layers": [
    {
      "mediaType": "application/vnd.kubecfg.bundle.tar+gzip",
      "digest": "sha256:7c1d62e76287035a9b22b2c155f328fae9beff2c6aa7a09a2dd2697539f41d98",
      "size": 404059
    }
  ],
  "annotations": {
    "org.opencontainers.image.created": "1970-01-01T00:00:00Z",
    "org.opencontainers.image.revision": "unknown",
    "org.opencontainers.image.source": "kubecfg pack"
  }
}

Error: fetching manifest us-docker.pkg.dev/influxdb2-artifacts/clustered/influxdb:<package-version>: GET https://us-docker.pkg.dev/v2/token?scope=repository%3Ainfluxdb2-artifacts%2Fclustered%2Finfluxdb%3Apull&service=: DENIED: Permission "artifactregistry.repositories.downloadArtifacts" denied on resource "projects/influxdb2-artifacts/locations/us/repositories/clustered" (or it may not exist)

DOCKER_CONFIG=/tmp/influxdbsecret \
crane config \
  us-docker.pkg.dev/influxdb2-artifacts/clustered/influxdb:
PACKAGE_VERSION
 \
  | jq -r '.metadata["oci.image.list"].images[]' \
  > /tmp/images.txt

us-docker.pkg.dev/influxdb2-artifacts/idpe/idpe-cd-ioxauth@sha256:5f015a7f28a816df706b66d59cb9d6f087d24614f485610619f0e3a808a73864
us-docker.pkg.dev/influxdb2-artifacts/iox/iox@sha256:b59d80add235f29b806badf7410239a3176bc77cf2dc335a1b07ab68615b870c
...

</tmp/images.txt xargs -I% crane cp % 
REGISTRY_HOSTNAME
/%

myregistry.mydomain.io

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      images:
        registryOverride: 
REGISTRY_HOSTNAME

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  imagePullSecrets:
    - name: 
PULL_SECRET_NAME

Modify the configuration file to point to prerequisites

Update your myinfluxdb.yml configuration file with credentials necessary to connect your cluster to your prerequisites.

• Configure ingress
• Configure the object store
• Configure the catalog database
• Configure local storage for ingesters

Configure ingress

To configure ingress, provide values for the following fields in your myinfluxdb.yml configuration file:

• spec.package.spec.ingress.hosts: Cluster hostnames

  Provide the hostnames that Kubernetes should use to expose the InfluxDB API endpoints. For example: cluster-host.com.

  You can provide multiple hostnames. The ingress layer accepts incoming requests for all listed hostnames. This can be useful if you want to have distinct paths for your internal and external traffic.

  You are responsible for configuring and managing DNS. Options include:

  • Manually managing DNS records
  • Using external-dns to synchronize exposed Kubernetes services and ingresses with DNS providers.

• spec.package.spec.ingress.tlsSecretName: TLS certificate secret name

  (Optional): Provide the name of the secret that contains your TLS certificate and key. The examples in this guide use the name ingress-tls.

  Writing to and querying data from InfluxDB does not require TLS. For simplicity, you can wait to enable TLS before moving into production. For more information, see Phase 4 of the InfluxDB Clustered installation process, Secure your cluster.

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
# ...
ingress:
  hosts:
    - cluster-host.com
  tlsSecretName: ingress-tls

Configure the object store

To connect your InfluxDB cluster to your object store, provide the required credentials in your myinfluxdb.yml. The credentials required depend on your object storage provider.

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      objectStore:
        s3:
          # URL for S3 Compatible object store
          endpoint: 
S3_URL

          # Set to true to allow communication over HTTP (instead of HTTPS)
          allowHttp: 'true'

          # S3 Access Key
          # This can also be provided as a valueFrom: secretKeyRef:
          accessKey:
            value: 
S3_ACCESS_KEY

          # S3 Secret Key
          # This can also be provided as a valueFrom: secretKeyRef:
          secretKey:
            value: 
S3_SECRET_KEY

          # Bucket that the Parquet files will be stored in
          bucket: 
S3_BUCKET_NAME

          # This value is required for AWS S3, it may or may not be required for other providers.
          region: 
S3_REGION

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      objectStore:
          # Bucket that the Parquet files will be stored in
        bucket: 
AZURE_BUCKET_NAME
        
        azure:
          # Azure Blob Storage Access Key
          # This can also be provided as a valueFrom:
          accessKey:
            value: 
AZURE_ACCESS_KEY

          # Azure Blob Storage Account
          # This can also be provided as a valueFrom: secretKeyRef:
          account:
            value: 
AZURE_STORAGE_ACCOUNT

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      objectStore:
          # Bucket that the Parquet files will be stored in
        bucket: 
GOOGLE_BUCKET_NAME
        
        google:
          # This section is not needed if you are using GKE Workload Identity.
          # It is only required to use explicit service account secrets (JSON files)
          serviceAccountSecret:
            # Kubernetes Secret name containing the credentials for a Google IAM
            # Service Account.
            name: 
GOOGLE_IAM_SECRET
            # The key within the Secret containing the credentials.
            key: 
GOOGLE_CREDENTIALS_KEY

Configure the catalog database

The InfluxDB catalog is a PostgreSQL-compatible relational database that stores metadata about your time series data. To connect your InfluxDB cluster to your PostgreSQL-compatible database, provide values for the following fields in your myinfluxdb.yml configuration file:

• spec.package.spec.catalog.dsn.valueFrom.secretKeyRef
  • .name: Secret name
  • .key: Key in the secret that contains the DSN

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      catalog:
        # A postgresql style DSN that points to a postgresql compatible database.
        # postgres://[user[:password]@][netloc][:port][/dbname][?param1=value1&...]
        dsn:
          valueFrom:
            secretKeyRef:
              name: 
SECRET_NAME
              key: 
SECRET_KEY

Replace the following:

• SECRET_NAME: Name of the secret containing your PostgreSQL-compatible DSN
• SECRET_KEY: Key in the secret that references your PostgreSQL-compatible DSN

Catalog DSN error: A catalog error occurred: unhandled external error: error with configuration: invalid port number

postgresql://postgres:meow#meow@my-fancy.cloud-database.party:5432/postgres

postgresql://postgres:meow%23meow@my-fancy.cloud-database.party:5432/postgres

postgres://username:passw0rd@mydomain:5432/influxdb?sslmode=disable

Configure local storage for ingesters

InfluxDB ingesters require local storage to store the Write Ahead Log (WAL) for incoming data. To connect your InfluxDB cluster to local storage, provide values for the following fields in your myinfluxdb.yml configuration file:

• spec.package.spec.ingesterStorage
  • .storageClassName: Kubernetes storage class. This differs based on the Kubernetes environment and desired storage characteristics.
  • storage: Storage size. We recommend a minimum of 2 gibibytes (2Gi).

apiVersion: kubecfg.dev/v1alpha1
kind: AppInstance
# ...
spec:
  package:
    spec:
      ingesterStorage:
        storageClassName: 
STORAGE_CLASS
        storage: 
STORAGE_SIZE

Replace the following:

• STORAGE_CLASS: Kubernetes storage class
• STORAGE_SIZE: Storage size (example: 2Gi)