Documentation

Replicate data from a remote instance to InfluxDB Cloud

This page documents an earlier version of InfluxDB. InfluxDB v2.6 is the latest stable version. View this page in the v2.6 documentation.

Use InfluxDB replication streams (InfluxDB Edge Data Replication) to replicate the incoming data of select buckets to one or more buckets on a remote InfluxDB instance.

Configure a replication stream

  1. Download and install the influx CLI.

  2. In your local InfluxDB OSS instance, use the influx remote create command to create a remote connection to replicate data to.

    Provide the following:

    • Remote connection name
    • Remote InfluxDB instance URL
    • Remote InfluxDB API token (API token must have write access to the target bucket)
    • Remote InfluxDB organization ID
    influx remote create \
      --name example-remote-name \
      --remote-url https://cloud2.influxdata.com \
      --remote-api-token mYsuP3r5Ecr37t0k3n \
      --remote-org-id 00xoXXoxXX00
    

    If you already have remote InfluxDB connections configured, you can use an existing connection. To view existing connections, run influx remote list.

  3. In your local InfluxDB OSS instance, use the influx replication create command to create a replication stream. Provide the following:

    • Replication stream name
    • Remote connection ID
    • Local bucket ID to replicate writes from
    • Remote bucket ID to replicate writes to
    influx replication create \
      --name example-replication-stream-name \
      --remote-id 00xoXXXo0X0x \
      --local-bucket-id Xxxo00Xx000o \
      --remote-bucket-id 0xXXx00oooXx
    

Once a replication stream is created, InfluxDB OSS will replicate all writes to the specified bucket to the remote InfluxDB bucket. Use the influx replication list command to view information such as the current queue size, max queue size, and latest status code.

Important things to note

  • Only write operations are replicated. Other data operations (like deletes or restores) are not replicated.
  • In InfluxDB OSS, large write request bodies are written entirely. When replicated, write requests are sent to the remote bucket in batches. The maximum batch size is 500 kB (typically between 250 to 500 lines of line protocol). This may result in scenarios where some batches succeed and others fail.

Replicate downsampled or processed data

In some cases, you may not want to write raw, high-precision data to a remote InfluxDB instance. To replicate only downsampled or processed data:

  1. Create a bucket in your InfluxDB OSS instance to store downsampled or processed data in.

  2. Create an InfluxDB task that downsamples or processes data and stores it in the new bucket. For example:

    import "influxdata/influxdb/tasks"
    import "types"
    
    // omit this line if adding task via the UI
    option task = {name: "Downsample raw data", every: 10m}
    
    data = () => from(bucket: "example-bucket")
        |> range(start: tasks.lastSuccess(orTime: -task.every))
    
    numeric = data()
        |> filter(fn: (r) => types.isType(v: r._value, type: "float") or types.isType(v: r._value, type: "int") or types.isType(v: r._value, type: "uint"))
        |> aggregateWindow(every: task.every, fn: mean)
    
    nonNumeric = data()
        |> filter(fn: (r) => types.isType(v: r._value, type: "string") or types.isType(v: r._value, type: "bool"))
        |> aggregateWindow(every: task.every, fn: last)
    
    union(tables: [numeric, nonNumeric])
        |> to(bucket: "example-downsampled-bucket")
    
  3. Create a replication stream to replicate data from the downsampled bucket to the remote InfluxDB instance.


Was this page helpful?

Thank you for your feedback!


Set your InfluxDB URL

Linux Package Signing Key Rotation

All signed InfluxData Linux packages have been resigned with an updated key. If using Linux, you may need to update your package configuration to continue to download and verify InfluxData software packages.

For more information, see the Linux Package Signing Key Rotation blog post.

InfluxDB Cloud backed by InfluxDB IOx

All InfluxDB Cloud organizations created on or after January 31, 2023 are backed by the new InfluxDB IOx storage engine. Check the right column of your InfluxDB Cloud organization homepage to see which InfluxDB storage engine you’re using.

If powered by IOx, this is the correct documentation.

If powered by TSM, see the TSM-based InfluxDB Cloud documentation.

InfluxDB Cloud backed by InfluxDB TSM

All InfluxDB Cloud organizations created on or after January 31, 2023 are backed by the new InfluxDB IOx storage engine which enables nearly unlimited series cardinality and SQL query support. Check the right column of your InfluxDB Cloud organization homepage to see which InfluxDB storage engine you’re using.

If powered by TSM, this is the correct documentation.

If powered by IOx, see the IOx-based InfluxDB Cloud documentation.

State of the InfluxDB Cloud (IOx) documentation

The new documentation for InfluxDB Cloud backed by InfluxDB IOx is a work in progress. We are adding new information and content almost daily. Thank you for your patience!

If there is specific information you’re looking for, please submit a documentation issue.