Documentation

Replicate data from InfluxDB OSS to InfluxDB Cloud

Use InfluxDB replication streams to replicate all data written to an InfluxDB OSS bucket to an InfluxDB Cloud bucket.

Configure a replication stream

  1. Download and install the influx CLI.

  2. In your InfluxDB OSS instance, use the influx remote create command to create a remote connection to replicate data to. Provide the following:

    • Remote connection name
    • InfluxDB Cloud region URL
    • InfluxDB Cloud API token (API token must have write access to the target bucket)
    • InfluxDB Cloud organization ID
    influx remote create \
      --name example-remote-name \
      --remote-url cloud2.influxdata.com \
      --remote-api-token mYsuP3r5Ecr37t0k3n \
      --remote-org-id 00xoXXoxXX00
    

    If you already have remote InfluxDB connections configured, you can use an existing connection. To view existing connections, run influx remote list.

  3. In your InfluxDB OSS instance, use the influx replication create command to create a replication stream. Provide the following:

    • Replication stream name
    • Remote connection ID
    • InfluxDB OSS bucket ID to replicate writes from
    • InfluxDB Cloud bucket ID to replicate writes to
    influx replication create \
      --name example-replication-stream-name \
      --remote-id 00xoXXXo0X0x \
      --local-bucket Xxxo00Xx000o \
      --remote-bucket 0xXXx00oooXx
    

Once a replication stream is created, InfluxDB will replicate all writes to the specified bucket to the InfluxDB Cloud bucket. Use the influx replication list command to view information such as the current queue size, max queue size, and latest status code.

Important things to note

  • Only write operations are replicated. Other data operations (like deletes or restores) are not replicated.
  • In InfluxDB OSS, large write request bodies are written entirely. When replicated, write requests are sent to the remote bucket in batches. The maximum batch size is 500 kB (typically between 250 to 500 lines of line protocol). This may result in scenarios where some batches succeed and others fail.

Replicate downsampled or processed data

In some cases, you may not want to write raw, high-precision data to a remote InfluxDB Cloud instance. To replicate only downsampled or processed data:

  1. Create a bucket in your InfluxDB OSS instance to store downsampled or processed data in.

  2. Create an InfluxDB task that downsamples or processes data and stores it in the new bucket. For example:

    import "influxdata/influxdb/tasks"
    import "types"
    
    option task = {name: "Downsample raw data", every: 10m}
    
    data = () => from(bucket: "example-bucket")
        |> range(start: tasks.lastSuccess(orTime: -task.every))
    
    numeric = data()
        |> filter(fn: (r) => types.isType(v: r._value, type: "float") or types.isType(v: r._value, type: "int") or types.isType(v: r._value, type: "uint"))
        |> aggregateWindow(every: -task.every, fn: mean)
    
    nonNumeric = data()
        |> filter(fn: (r) => types.isType(v: r._value, type: "string") or types.isType(v: r._value, type: "bool"))
        |> aggregateWindow(every: -task.every, fn: last)
    
    union(tables: [numeric, nonNumeric])
        |> to(bucket: "example-downsampled-bucket"])
    
  3. Create a replication stream to replicate data from the downsampled bucket to the remote InfluxDB Cloud instance.


Was this page helpful?

Thank you for your feedback!


Select your region

Upgrade to InfluxDB Cloud or InfluxDB 2.0!

InfluxDB Cloud and InfluxDB OSS 2.0 ready for production.