Optimize writes to InfluxDB
Use these tips to optimize performance and system overhead when writing data to InfluxDB.
- Batch writes
- Sort tags by key
- Use the coarsest time precision possible
- Use gzip compression
- Synchronize hosts with NTP
- Write multiple data points in one request
The following tools write to InfluxDB and employ most write optimizations by default:
Write data in batches to minimize network overhead when writing data to InfluxDB.
The optimal batch size is 10,000 lines of line protocol or 10 MBs, whichever threshold is met first.
Sort tags by key
Before writing data points to InfluxDB, sort tags by key in lexicographic order.
Verify sort results match results from the Go
# Line protocol example with unsorted tags measurement,tagC=therefore,tagE=am,tagA=i,tagD=i,tagB=think fieldKey=fieldValue 1562020262 # Optimized line protocol example with tags sorted by key measurement,tagA=i,tagB=think,tagC=therefore,tagD=i,tagE=am fieldKey=fieldValue 1562020262
Use the coarsest time precision possible
By default, InfluxDB writes data in nanosecond precision. However if your data isn’t collected in nanoseconds, there is no need to write at that precision. For better performance, use the coarsest precision possible for timestamps.
Specify timestamp precision when writing to InfluxDB.
Use gzip compression
Use gzip compression to speed up writes to InfluxDB. Benchmarks have shown up to a 5x speed improvement when data is compressed.
Enable gzip compression in Telegraf
influxdb_v2 output plugin configuration in your
telegraf.conf, set the
content_encoding option to
[[outputs.influxdb_v2]] urls = ["https://cloud2.influxdata.com"] # ... content_encoding = "gzip"
Enable gzip compression in InfluxDB client libraries
Each InfluxDB client library provides options for compressing write requests or enforces compression by default. The method for enabling compression is different for each library. For specific instructions, see the InfluxDB client libraries documentation.
Use gzip compression with the InfluxDB API
When using the InfluxDB API
/api/v2/write endpoint to write data,
compress the data with
gzip and set the
Content-Encoding header to
echo "mem,host=host1 used_percent=23.43234543 1641024000 mem,host=host2 used_percent=26.81522361 1641027600 mem,host=host1 used_percent=22.52984738 1641031200 mem,host=host2 used_percent=27.18294630 1641034800" | gzip > system.gzip \ curl --request POST "https://cloud2.influxdata.com/api/v2/write?org=ORG_NAME&bucket=BUCKET_NAME&precision=s" \ --header "Authorization: Token API_TOKEN" \ --header "Content-Type: text/plain; charset=utf-8" \ --header "Content-Encoding: gzip" \ --data-binary @system.gzip
Synchronize hosts with NTP
Use the Network Time Protocol (NTP) to synchronize time between hosts. If a timestamp isn’t included in line protocol, InfluxDB uses its host’s local time (in UTC) to assign timestamps to each point. If a host’s clocks isn’t synchronized with NTP, timestamps may be inaccurate.
Write multiple data points in one request
To write multiple lines in one request, each line of line protocol must be delimited by a new line (
Was this page helpful?
Thank you for your feedback!
Support and feedback
Thank you for being part of our community! We welcome and encourage your feedback and bug reports for InfluxDB and this documentation. To find support, use the following resources:
Customers with an annual or support contract can contact InfluxData Support.