Migrate InfluxDB OSS instances to InfluxDB Enterprise clusters

Migrate a running instance of InfluxDB open source (OSS) to an InfluxDB Enterprise cluster.

Prerequisites

• An InfluxDB OSS instance running InfluxDB 1.7.10 or later.

• An InfluxDB Enterprise cluster running InfluxDB Enterprise 1.7.10 or later

• Your OSS and Enterprise version is the same, for example, InfluxDB 1.8 and InfluxDB Enterprise 1.8.

• Network accessibility between the OSS instances and all data and meta nodes.

  Migrating does the following:

  • Deletes data in existing InfluxDB Enterprise data nodes (not applicable if you’re migrating to a new cluster)
  • Transfers all users from the OSS instance to the InfluxDB Enterprise cluster

Migrate to InfluxDB Enterprise

Complete the following tasks:

1. Upgrade InfluxDB to the latest version
2. Set up InfluxDB Enterprise meta nodes
3. Set up InfluxDB Enterprise data nodes
4. Do one of the following:
   • Migrate a data set with zero downtime. We recommend using this method to create a portable backup first. This method lets you move data between OSS and Enterprise as you’re testing the migration.
   • Migrate a data set with downtime. Note, with this method, you cannot move data from Enterprise back to OSS. This method is useful if you’re not able to run a portable backup. Some reasons you may not be able to create a portable backup:
   • Data set exceeds a certain size
   • Hardware requirements aren’t available
   • Time constraints (large data sets increase the time needed to back up data)

Upgrade InfluxDB to the latest version

Upgrade InfluxDB OSS and InfluxDB Enterprise to the latest stable version. Make sure the OSS and Enterprise version is the same.

• Upgrade InfluxDB OSS
• Upgrade InfluxDB Enterprise

Set up InfluxDB Enterprise meta nodes

Set up all meta nodes in your InfluxDB Enterprise cluster. For information about installing and setting up meta nodes, see Install meta nodes.

Set up InfluxDB Enterprise data nodes

If you don’t have any existing data nodes in your InfluxDB Enterprise cluster, skip this step.

For each existing data node:

1. Remove the data node from the InfluxDB Enterprise cluster

   From a meta node in your InfluxDB Enterprise cluster, run:

   influxd-ctl remove-data <data_node_hostname>:8088
   

2. Delete existing data

   On each data node dropped from the cluster, run:

   sudo rm -rf /var/lib/influxdb/{meta,data,hh}
   

3. Recreate data directories

   On each data node dropped from the cluster, run:

   sudo mkdir /var/lib/influxdb/{data,hh,meta}
   

4. Ensure file permissions are correct

   On each data node dropped from the cluster, run:

   sudo chown -R influxdb:influxdb /var/lib/influxdb
   

5. Update the /etc/hosts file

   On each data node, add the IP and hostname of the OSS instance to the /etc/hosts file to allow the data node to communicate with the OSS instance.

Migrate a data set with zero downtime

1. Take a portable backup from OSS:

    influxd backup -portable -host <IP address>:8088 /tmp/mysnapshot
   

   For more information, see -backup

2. Restore the backup on the cluster by running the following:

   influxd restore -portable  [ -host <host:port> ] <path-to-backup-files>
   

   For more information, see -restore

3. Dual write to both OSS and Enterprise. See Write data with the InfluxDB API. This keeps the OSS and cluster active for testing and acceptance work.

4. Export data from OSS from the time the backup was taken to the time the dual write started. For example, if you take the backup on 2020-07-19T00:00:00.000Z, and started writing data to Enterprise at 2020-07-19T23:59:59.999Z, you could run the following command:

   influx_inspect export -compress -start 2020-07-19T00:00:00.000Z -end 2020-07-19T23:59:59.999Z`
   

   For more information, see -export.

5. Import data into Enterprise.

6. Verify data is successfully migrated. To review your data, see how to:

   • Query data with the InfluxDB API
   • View data in Chronograf

7. Stop writes and remove OSS.

Migrate a data set with downtime

1. Stop writes and remove OSS
2. Back up OSS configuration
3. Add the upgraded OSS instance to the InfluxDB Enterprise cluster
4. Add existing data nodes back to the cluster
5. Rebalance the cluster

Stop writes and remove OSS

1. Stop all writes to the InfluxDB OSS instance

2. Stop the influxdb service on the InfluxDB OSS instance

   sysvinit systemd

   sudo service influxdb stop
   

   sudo systemctl stop influxdb
   

   Double check that the service is stopped. The following command should return nothing:

   ps ax | grep influxd
   

3. Remove the InfluxDB OSS package

   Debian & Ubuntu RHEL & CentOS

   sudo apt-get remove influxdb
   

   sudo yum remove influxdb
   

Back up your InfluxDB OSS configuration file

1. Back up your InfluxDB OSS configuration file

   If you have custom configuration settings for InfluxDB OSS, back up and save your configuration file. Without a backup, you’ll lose custom configuration settings when updating the InfluxDB binary.

2. Update the InfluxDB binary

   Updating the InfluxDB binary overwrites the existing configuration file. To keep custom settings, back up your configuration file.

   Debian & Ubuntu RHEL & CentOS

   wget https://dl.influxdata.com/enterprise/releases/influxdb-data_1.8.10-c1.8.10_amd64.deb
   sudo dpkg -i influxdb-data_1.8.10-c1.8.10_amd64.deb
   

   wget https://dl.influxdata.com/enterprise/releases/influxdb-data-1.8.10-c1.8.10.x86_64.rpm
   sudo yum localinstall influxdb-data-1.8.10-c1.8.10.x86_64.rpm
   

3. Update the configuration file

   In /etc/influxdb/influxdb.conf, set:

   • hostname to the full hostname of the data node

   • license-key in the [enterprise] section to the license key you received on InfluxPortal OR license-path in the [enterprise] section to the local path to the JSON license file you received from InfluxData.

     The license-key and license-path settings are mutually exclusive and one must remain set to an empty string.

   # Hostname advertised by this host for remote addresses.
   # This must be accessible to all nodes in the cluster.
   hostname="<data-node-hostname>"
   
   [enterprise]
     # license-key and license-path are mutually exclusive,
     # use only one and leave the other blank
     license-key = "<your_license_key>"
     license-path = "/path/to/readable/JSON.license.file"
   

   Transfer any custom settings from the backup of your OSS configuration file to the new Enterprise configuration file.

4. Update the /etc/hosts file

   Add all meta and data nodes to the /etc/hosts file to allow the OSS instance to communicate with other nodes in the InfluxDB Enterprise cluster.

5. Start the data node

   sysvinit systemd

   sudo service influxdb start
   

   sudo systemctl start influxdb
   

Add the new data node to the cluster

After you upgrade your OSS instance to InfluxDB Enterprise, add the node to your Enterprise cluster.

• From a meta node in the cluster, run:

  influxd-ctl add-data <new-data-node-hostname>:8088
  

  The output should look like:

  Added data node y at new-data-node-hostname:8088
  

Add existing data nodes back to the cluster

If you removed any existing data nodes from your InfluxDB Enterprise cluster, add them back to the cluster.

1. From a meta node in the InfluxDB Enterprise cluster, run the following for each data node:

   influxd-ctl add-data <the-hostname>:8088
   

   It should output:

   Added data node y at the-hostname:8088
   

2. Verify that all nodes are now members of the cluster as expected:

   influxd-ctl show
   

Once added to the cluster, InfluxDB synchronizes data stored on the upgraded OSS node with other data nodes in the cluster. It may take a few minutes before the existing data is available.

Rebalance the cluster

1. Use the ALTER RETENTION POLICY statement to increase the replication factor on all existing retention polices to the number of data nodes in your cluster.
2. Rebalance your cluster manually to meet the desired replication factor for existing shards.
3. If you were using Chronograf, add your Enterprise instance as a new data source.