Use Amazon S3 for object storage

Use Amazon S3 as the object store for your InfluxDB 3 Core instance.

Amazon S3 is the reference object storage implementation InfluxDB 3 Core is built against and tested with.

• Object store requirements
• Create an S3 bucket
• Create IAM credentials
• Configure InfluxDB to connect to S3
• Confirm the object store is working

Object store requirements

InfluxDB 3 Core depends on strict object store consistency semantics— strong read-after-write and list-after-write consistency, and conditional PUT (PUT-if-not-exists) support. Amazon S3 provides these semantics. See Object store requirements for the full list of semantics InfluxDB 3 Core depends on and how to verify your object store meets them.

Create an S3 bucket

1. Sign in to the Amazon S3 console.
2. Click Create bucket.
3. Enter a bucket name–for example, influxdb3–and choose an AWS Region.
4. Keep the default Block Public Access and versioning settings unless your organization requires otherwise, and click Create bucket.

Alternatively, use the AWS CLI to create a bucket:

aws s3 mb s3://influxdb3 --region 
AWS_REGION

Create IAM credentials

Create an IAM user or role with permission to read from and write to your bucket, and generate an access key for it:

1. In the IAM console, create a new user (or use an existing one) and attach a policy that grants access to your bucket–for example:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": ["s3:GetBucketLocation", "s3:ListBucket"],
         "Resource": ["arn:aws:s3:::influxdb3"]
       },
       {
         "Effect": "Allow",
         "Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"],
         "Resource": ["arn:aws:s3:::influxdb3/*"]
       }
     ]
   }

2. Generate an access key for the user under Security credentials. Save the access key ID and secret access key–you use them to configure InfluxDB 3 Core.

Configure InfluxDB to connect to S3

To use your S3 bucket as the object store for your InfluxDB 3 Core instance, provide the following options or environment variables with the influxdb3 serve command:

influxdb3 serve \
  
--node-id 
INFLUXDB_NODE_ID
 \
  --object-store s3 \
  --bucket influxdb3 \
  --aws-access-key-id 
AWS_ACCESS_KEY_ID
 \
  --aws-secret-access-key 
AWS_SECRET_ACCESS_KEY
 \
  --aws-default-region 
AWS_REGION

export INFLUXDB3_NODE_ID=
INFLUXDB_NODE_ID
export INFLUXDB3_OBJECT_STORE=s3
export INFLUXDB3_BUCKET=influxdb3
export 
AWS_ACCESS_KEY_ID=
AWS_ACCESS_KEY_ID
export 
AWS_SECRET_ACCESS_KEY=
AWS_SECRET_ACCESS_KEY
export AWS_DEFAULT_REGION=
AWS_REGION

influxdb3 serve

Confirm the object store is working

When InfluxDB 3 Core starts, it seeds your S3 bucket with the necessary directory structure and begins storing data there. Confirm the object store is functioning properly:

1. View the influxdb3 serve log output to confirm that the server is running correctly.

2. Inspect the contents of your bucket to confirm the necessary directory structure is created–for example, using the AWS CLI:

   aws s3 ls s3://influxdb3