Set up prerequisites
InfluxDB Clustered requires the following prerequisite external dependencies:
- kubectl command line tool
- Kubernetes cluster
- kubecfg kubit operator
- Kubernetes ingress controller
- Object storage: AWS S3 or S3-compatible storage (including Google Cloud Storage or Azure Blob Storage) to store the InfluxDB Parquet files.
- PostgreSQL-compatible database (AWS Aurora, hosted PostgreSQL, etc.): Stores the InfluxDB Catalog.
- Local or attached storage: Stores the Write-Ahead Log (WAL) for InfluxDB Ingesters.
The following walks through preparing these prerequisites.
- Install kubectl
- Set up your Kubernetes cluster
- Install the kubecfg kubit operator
- Set up a Kubernetes ingress controller
- Set up your object store
- Set up your PostgreSQL-compatible database
- Set up local or attached storage
Install kubectl
Kubernetes provides the kubectl
command line tool for communicating with a
Kubernetes cluster’s control plane. kubectl
is used to manage your InfluxDB
cluster.
Follow instructions to install kubectl
on your local machine:
InfluxDB Clustered Kubernetes deployments require kubectl
1.27 or higher.
Set up your Kubernetes cluster
Deploy a Kubernetes cluster. The deployment process depends on your Kubernetes environment or Kubernetes cloud provider. Refer to the Kubernetes documentation or your cloud provider’s documentation for information about deploying a Kubernetes cluster.
Ensure
kubectl
can connect to your Kubernetes cluster. Your Manage kubeconfig file defines cluster connection credentials.Create two namespaces–
influxdb
andkubit
. Usekubectl create namespace
to create the namespaces:kubectl create namespace influxdb && \ kubectl create namespace kubit
Install an ingress controller in the cluster and a mechanism to obtain a valid TLS certificate (for example: cert-manager or provide the certificate PEM manually out of band). To use the InfluxDB-specific ingress controller, install Ingress NGINX.
Ensure your Kubernetes cluster can access the InfluxDB container registry, or, if running in an air-gapped environment, a local container registry to which you can copy the InfluxDB images.
Cluster sizing recommendation
As a starting point for a production workload, InfluxData recommends the following sizing for InfluxDB Clustered components:
- Catalog (PostgreSQL-compatible database) (x1):
- Ingesters and Routers (x3):
- EC2 m6i.2xlarge (8 CPU, 32 GB RAM)
- Local storage: mimumum of 2 GB (high-speed SSD)
- Queriers (x3):
- EC2 m6i.2xlarge (8 CPU, 32 GB RAM)
- Compactors (x1):
- EC2 m6i.2xlarge (8 CPU, 32 GB RAM)
- Kubernetes Control Plane (x1):
- EC2 t3.large (2 CPU, 8 GB RAM)
- Catalog (PostgreSQL-compatible database) (x1):
- Ingesters and Routers (x3):
- GCE c2-standard-8 (8 CPU, 32 GB RAM)
- Local storage: mimumum of 2 GB (high-speed SSD)
- Queriers (x3):
- GCE c2-standard-8 (8 CPU, 32 GB RAM)
- Compactors (x1):
- GCE c2-standard-8 (8 CPU, 32 GB RAM)
- Kubernetes Control Plane (x1):
- GCE c2d-standard-2 (2 CPU, 8 GB RAM)
- Catalog (PostgreSQL-compatible database) (x1):
- Ingesters and Routers (x3):
- Standard_D8s_v3 (8 CPU, 32 GB RAM)
- Local storage: mimumum of 2 GB (high-speed SSD)
- Queriers (x3):
- Standard_D8s_v3 (8 CPU, 32 GB RAM)
- Compactors (x1):
- Standard_D8s_v3 (8 CPU, 32 GB RAM)
- Kubernetes Control Plane (x1):
- Standard_B2ms (2 CPU, 8 GB RAM)
- Catalog (PostgreSQL-compatible database) (x1):
- CPU: 4-8 cores
- RAM: 16-32 GB
- Ingesters and Routers (x3):
- CPU: 8 cores
- RAM: 32 GB
- Local storage: 2 GB (high-speed SSD)
- Queriers (x3):
- CPU: 8 cores
- RAM: 32 GB
- Compactors (x1):
- CPU: 8 cores
- RAM: 32 GB
- Kubernetes Control Plane (x1):
- CPU: 2 cores
- RAM: 8 GB
Your sizing may need to be different based on your environment, cloud provider, and workload, but this is a reasonable starting size for your initial testing.
Install the kubecfg kubit operator
The kubecfg kubit
operator (maintained by InfluxData)
simplifies the installation and management of the InfluxDB Clustered package.
It manages the application of the jsonnet templates used to install, manage, and
update an InfluxDB cluster.
The InfluxDB Clustered Helm chart includes the kubit operator
If using the InfluxDB Clustered Helm chart to deploy your InfluxDB cluster, you do not need to install the kubit operator separately. The Helm chart installs the kubit operator.
Use kubectl
to install the kubecfg kubit
operator v0.0.18 or later.
kubectl apply -k 'https://github.com/kubecfg/kubit//kustomize/global?ref=v0.0.19'
Set up a Kubernetes ingress controller
Kubernetes ingress routes HTTP/S requests to services within the cluster and requires deploying an ingress controller. You can provide your own ingress or you can install Nginx Ingress Controller to use the InfluxDB-defined ingress.
InfluxDB Clustered components use gRPC/HTTP2 protocols. If using an external load balancer, you may need to explicitly enable these protocols on your load balancers.
Set up your object store
InfluxDB Clustered supports AWS S3 or S3-compatible storage (including Google Cloud Storage, Azure Blob Storage, and MinIO) for storing InfluxDB Parquet files. Refer to your object storage provider’s documentation for information about setting up an object store:
- Create an AWS S3 bucket
- Create a Google Cloud Storage bucket
- Create an Azure Blog Storage container
- Create a MinIO bucket
Object storage recommendations
We strongly recommend the following:
Enable object versioning
Enable object versioning in your object store. Refer to your object storage provider’s documentation for information about enabling object versioning.
Run the object store in a separate namespace or outside of Kubernetes
Run the Object store in a separate namespace from InfluxDB or external to Kubernetes entirely. Doing so makes management of the InfluxDB cluster easier and helps to prevent accidental data loss. While deploying everything in the same namespace is possible, we do not recommend it for production environments.
Configure object storage permissions
Ensure the identity you use to connect to your S3-compatible object store has the correct permissions to allow InfluxDB to perform all the actions it needs to.
To configure permissions with MinIO, use the example AWS access policy.
Set up your PostgreSQL-compatible database
The InfluxDB Catalog that stores metadata related to your time series data requires a PostgreSQL or PostgreSQL-compatible database (AWS Aurora, hosted PostgreSQL, etc.). The process for installing and setting up your PostgreSQL-compatible database depends on the database and database provider you use. Refer to your database’s or provider’s documentation for setting up your PostgreSQL-compatible database.
PostgreSQL-compatible database requirements
- PostgreSQL versions 13.8–14.6.
- Minimum of 4 GB of memory or equivalent provider-specific units.
- To avoid conflicts and prevent issues caused by shared usage with other applications, ensure that your PostgreSQL-compatible instance is dedicated exclusively to InfluxDB.
We strongly recommended running the PostgreSQL-compatible database in a separate namespace from InfluxDB or external to Kubernetes entirely. Doing so makes management of the InfluxDB cluster easier and helps to prevent accidental data loss.
While deploying everything in the same namespace is possible, we do not recommend it for production environments.
Set up local or attached storage
The InfluxDB Ingester
needs local or attached storage to store the Write-Ahead Log (WAL).
The read and write speed of the attached storage affects the write performance
of the Ingester, so the faster the storage device, the better your write
performance will be.
The recommended minimum size of the local storage is 2 gibibytes (2Gi
).
Installation and setup of local or attached storage depends on your underlying hardware or cloud provider. Refer to your provider’s documentation for information about installing and configuring local storage.
Was this page helpful?
Thank you for your feedback!
Support and feedback
Thank you for being part of our community! We welcome and encourage your feedback and bug reports for InfluxDB and this documentation. To find support, use the following resources:
Customers with an annual or support contract can contact InfluxData Support.