# InfluxDB OSS v1 — full Markdown corpus > Generated 2026-10-31T04:19:00.069Z from https://docs.influxdata.com/influxdb/v1/ > See https://docs.influxdata.com/llms.txt for the cross-product table of contents. > Publisher: InfluxData (https://www.influxdata.com) > sameAs: https://github.com/influxdata, https://hub.docker.com/u/influxdata, https://www.crunchbase.com/organization/influxdb, https://www.linkedin.com/company/influxdb, https://twitter.com/influxdb, https://www.youtube.com/@InfluxData, https://www.facebook.com/influxdb/ --- # InfluxDB OSS v1 documentation Source: https://docs.influxdata.com/influxdb/v1/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxDB is a [time series database](https://www.influxdata.com/time-series-database/) designed to handle high write and query loads. InfluxDB OSS v1 is purpose-built to handle any use case involving large amounts of timestamped data and is an integral component of the[TICK stack](https://influxdata.com/time-series-platform/). Common use cases include: * Infrastructure and DevOps monitoring * Application metrics and performance monitoring * IoT sensor data collection * Real-time analytics * Events handling > [!Note] > #### InfluxDB Cloud 1 users > > **InfluxDB Cloud 1** (InfluxCloud 1.x) is based on InfluxDB > Enterprise v1. Use the[Enterprise v1 documentation](/enterprise_influxdb/v1/) instead. > > * **InfluxDB Cloud 1 Support**:[help.influxcloud.net](https://help.influxcloud.net) > * **Migrate**: Consider[migrating to InfluxDB 3](/platform/#migration-to-influxdb-3)(Enterprise, Cloud Serverless, or Cloud Dedicated) with v1 API > compatibility ## Key features InfluxDB v1.13 supports the following features for working with time series data. * Custom high performance datastore written specifically for time series data. The TSM engine allows for high ingest speed and data compression * Written entirely in Go. It compiles into a single binary with no external dependencies. * Simple, high performing write and query HTTP APIs. * Plugins support for other data ingestion protocols such as Graphite, collectd, and OpenTSDB. * Expressive SQL-like query language tailored to easily query aggregated data. * Tags allow series to be indexed for fast and efficient queries. * Retention policies efficiently auto-expire stale data. * Continuous queries automatically compute aggregate data to make frequent queries more efficient. InfluxDB OSS v1 runs on a single node. If you require high availability to eliminate a single point of failure, consider [InfluxDB 3 Enterprise](/influxdb3/enterprise/), InfluxDB’s next generation that supports multi-node clustering, allows infinite series cardinality without impact on overall database performance, and brings native SQL support and improved InfluxQL performance. --- # About InfluxDB OSS Source: https://docs.influxdata.com/influxdb/v1/about_the_project/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## [Contribute to InfluxDB OSS](/influxdb/v1/about_the_project/contributing/) Contribute to the InfluxDB OSS project. ## [InfluxData Contributor License Agreement (CLA)](/influxdb/v1/about_the_project/cla/) Before contributing to the InfluxDB OSS project, you must complete and sign the [InfluxData Contributor License Agreement (CLA)](https://www.influxdata.com/legal/cla/), available on the InfluxData website. ## [InfluxDB v1 release notes](/influxdb/v1/about_the_project/release-notes/) Important features, fixes, and updates in each version of InfluxDB OSS v1. ## [Open source license for InfluxDB](/influxdb/v1/about_the_project/licenses/) The [open source license for InfluxDB](https://github.com/influxdata/influxdb/blob/master/LICENSE)is available in the GitHub repository. ## [Third party software](/influxdb/v1/about_the_project/third-party/) InfluxData products contain third party software, which means the copyrighted, patented, or otherwise legally protected software of third parties that is incorporated in InfluxData products. Third party suppliers make no representation nor warranty with respect to such third party software or any portion thereof. Third party suppliers assume no liability for any claim that might arise with respect to such third party software, nor for a customer’s use of or inability to use the third party software. --- # InfluxData Contributor License Agreement (CLA) Source: https://docs.influxdata.com/influxdb/v1/about_the_project/cla/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Before contributing to the InfluxDB OSS project, you must complete and sign the [InfluxData Contributor License Agreement (CLA)](https://www.influxdata.com/legal/cla/), available on the InfluxData website. --- # Contribute to InfluxDB OSS Source: https://docs.influxdata.com/influxdb/v1/about_the_project/contributing/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. To learn how you can contribute to the InfluxDB OSS project, see[Contributing to InfluxDB](https://github.com/influxdata/influxdb/tree/1.8/CONTRIBUTING.md)in the GitHub repository. --- # Open source license for InfluxDB Source: https://docs.influxdata.com/influxdb/v1/about_the_project/licenses/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The [open source license for InfluxDB](https://github.com/influxdata/influxdb/blob/master/LICENSE)is available in the GitHub repository. --- # InfluxDB v1 release notes Source: https://docs.influxdata.com/influxdb/v1/about_the_project/release-notes/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## v1.13.2 ### Bug fixes * Resolve unimplemented interface member ([#27715](https://github.com/influxdata/influxdb/pull/27716)) ### Maintenance * Update go version to 1.26.10 ([#27730](https://github.com/influxdata/influxdb/pull/27730)) ## v1.13.1 ### Features * **mTLS support**: Add mutual TLS (mTLS) authentication for the HTTP, OpenTSDB, and subscriber services, including certificate reload on `SIGHUP` and separate client and server certificate configuration. ([#27534](https://github.com/influxdata/influxdb/pull/27534),[#27543](https://github.com/influxdata/influxdb/pull/27543)) * **Adaptive TSI cache sizing**: The[`series-id-set-cache-size`](/influxdb/v1/administration/config/#series-id-set-cache-size)cache can now grow and shrink itself between a configurable floor and ceiling based on the measured query hit rate, instead of staying at one fixed size. Configure it with the new[`series-id-set-cache-max-size`](/influxdb/v1/administration/config/#series-id-set-cache-max-size),[`series-id-set-cache-target-hit-rate`](/influxdb/v1/administration/config/#series-id-set-cache-target-hit-rate), and[`series-id-set-cache-shrink-conservatism`](/influxdb/v1/administration/config/#series-id-set-cache-shrink-conservatism)settings. Monitor it through the new `tsi1_cache` `SHOW STATS` measurement. ([#27480](https://github.com/influxdata/influxdb/pull/27480),[#27552](https://github.com/influxdata/influxdb/pull/27552)) * **`hardening-enabled`**: Add a `hardening-enabled` configuration option that restricts Flux HTTP requests to public addresses, mitigating server-side request forgery (SSRF). Disabled by default to preserve existing behavior. ([#27488](https://github.com/influxdata/influxdb/pull/27488)) * **Backup compression**: Add a `-gzipCompressionLevel` flag to `influxd backup`for trading compression ratio against speed. ([#27293](https://github.com/influxdata/influxdb/pull/27293)) * **Broader config value formats**: Size and duration configuration settings now accept a wider range of human-readable input formats, while remaining compatible with all previously accepted values. ([#27376](https://github.com/influxdata/influxdb/pull/27376)) * **[`SHOW MEASUREMENTS`](/influxdb/v1/query_language/explore-schema/#show-measurements)**: Support partial results when some shards are unavailable. ([#27443](https://github.com/influxdata/influxdb/pull/27443)) * **Operational visibility**: Failed and slow queries, per-user query and[write byte statistics](/influxdb/v1/administration/config/#user-write-bytes-enabled)(opt-in), the remote host and user in query logs, compaction planning statistics, continuous query diagnostics, and TSM file-store merge metrics are now available through `SHOW STATS`, `/debug/vars`, and`EXPLAIN ANALYZE`. ([#27536](https://github.com/influxdata/influxdb/pull/27536),[#27547](https://github.com/influxdata/influxdb/pull/27547),[#27188](https://github.com/influxdata/influxdb/pull/27188),[#27528](https://github.com/influxdata/influxdb/pull/27528),[#26981](https://github.com/influxdata/influxdb/pull/26981),[#27523](https://github.com/influxdata/influxdb/pull/27523),[#26874](https://github.com/influxdata/influxdb/pull/26874),[#26615](https://github.com/influxdata/influxdb/pull/26615),[#26624](https://github.com/influxdata/influxdb/pull/26624)) ### Bug Fixes * Fixed a potential deadlock and reduced lock contention between compactions and hot query reads in the TSM file store by splitting its single lock into separate fast and slow paths. ([#27501](https://github.com/influxdata/influxdb/pull/27501)) * Corrected several TSI compaction-planning and lock-contention issues. ([#27146](https://github.com/influxdata/influxdb/pull/27146),[#27123](https://github.com/influxdata/influxdb/pull/27123),[#26649](https://github.com/influxdata/influxdb/pull/26649),[#26647](https://github.com/influxdata/influxdb/pull/26647),[#26432](https://github.com/influxdata/influxdb/pull/26432)) * Fixed authorizer leakage in `SHOW` queries. ([#27196](https://github.com/influxdata/influxdb/pull/27196)) Other updates include numerous compaction planning, locking, and stability improvements. > [!Important] > #### We strongly recommend upgrading to v1.12.4 or later > > If you’re using any previous InfluxDB v1.x version, we strongly > recommend [upgrading to 1.12.4 or later](/influxdb/v1/administration/upgrading/). ## v1.12.4 ### Bug Fixes * Fixed `fatal error: concurrent map iteration and map write` panic in the TSI index that could crash InfluxDB during concurrent read and write operations. This was a regression introduced in v1.12.3 by an unnecessary backport from the 2.x branch that released a read lock before the underlying map was fully iterated. The fix restores the original locking behavior.[#27344](https://github.com/influxdata/influxdb/pull/27344),[#27343](https://github.com/influxdata/influxdb/issues/27343) > [!Important] > #### We strongly recommend upgrading to v1.12.4 or later > > If you’re using any previous InfluxDB v1.x version, we strongly > recommend [upgrading to 1.12.4 or later](/influxdb/v1/administration/upgrading/). ## v1.12.3 ### Features * Add [`https-insecure-certificate` configuration option](/influxdb/v1/administration/config/#https-insecure-certificate)to skip file permission checking for TLS certificate and private key files. * Add [`advanced-expiration` TLS configuration option](/influxdb/v1/administration/config/#advanced-expiration)to configure how far in advance to log warnings about TLS certificate expiration. * Add TLS certificate reloading on `SIGHUP`. * Add [`config`](/influxdb/v1/api/debug/) and [`cq` (continuous query) statistics](/influxdb/v1/api/debug/) to the `/debug/vars` endpoint. * Improve dropped point logging. * [Show user when displaying or logging queries](/influxdb/v1/troubleshooting/query_management/#list-currently-running-queries-with-show-queries). * Add [`time_format` parameter](/influxdb/v1/api/query/) for the HTTP API. * Use dynamic logging levels (`zap.AtomicLevel`). * [Report user query bytes](/influxdb/v1/administration/config/#user-query-bytes-enabled). ### Bug fixes * Fix `FUTURE LIMIT` and `PAST LIMIT`[clause order](/influxdb/v1/query_language/manage-database/#future-limit)in retention policy statements. * Add locking in `ClearBadShardList`. * Stop noisy logging about phantom shards that do not belong to a node. * Resolve `RLock()` leakage in `Store.DeleteSeries()`. * Fix condition check for optimization of array cursor (tsm1). * Run `init.sh` `buildtsi` as `influxdb` user. * Reduce unnecessary purger operations and logging. * Sort files for adjacency testing. * Fix operator in host detection (systemd). * Use correct path in open WAL error message. * Handle nested low-level files in compaction. * Correct error logic for writing empty index files. * Reduce lock contention and races in purger. * Fix bug with authorizer leakage in `SHOW QUERIES`. * Rename compact throughput logging keys. * Fix `https-insecure-certificate` not handled properly in httpd. * Prevent level regression when compacting mixed-level TSM files. ### Other * Update Go to 1.24.13. ## v1.12.2 ### Features * Add additional log output when using[`influx_inspect buildtsi`](/influxdb/v1/tools/influx_inspect/#buildtsi) to rebuild the TSI index. * Use [`influx_inspect export`](/influxdb/v1/tools/influx_inspect/#export) with[`-tsmfile` option](/influxdb/v1/tools/influx_inspect/#--tsmfile-tsm_file-) to export a single TSM file. * Add `fluxQueryRespBytes` metric to the `/debug/vars` metrics endpoint. * Add [`aggressive-points-per-block` configuration option](/influxdb/v1/administration/config/#aggressive-points-per-block)to prevent TSM files from not getting fully compacted. * Improve error handling. * InfluxQL updates: * Delete series by retention policy. * Allow retention policies to discard writes that fall within their range, but outside of [`FUTURE LIMIT`](/influxdb/v1/query_language/manage-database/#future-limit)and [`PAST LIMIT`](/influxdb/v1/query_language/manage-database/#past-limit). ### Bug fixes * Log rejected writes to subscriptions. * Update `xxhash` and avoid `stringtoslicebyte` in the cache. * Prevent a panic when a shard group has no shards. * Fix file handle leaks in `Compactor.write`. * Ensure fields in memory match the fields on disk. * Ensure temporary files are removed after failed compactions. * Do not panic on invalid multiple subqueries. ### Other * Update Go to 1.23.5. * Upgrade Flux to v0.196.1. * Upgrade InfluxQL to v1.4.1. * Various other dependency updates. ## v1.11.8 ### Bug Fixes * Strip double quotes from measurement names in the [`/api/v2/delete`compatibility API](/influxdb/v1/api/delete-v2-compatible/) before string comparisons (e.g. to allow special characters in measurement names). ## v1.11.7 This release represents the first public release of InfluxDB OSS v1 since 2021 and includes many enhancements and bug fixes added to InfluxDB Enterprise and then back-ported to InfluxDB OSS v1. Many of these enhancements improve compatibility between InfluxDB v1 and InfluxDB 3 and help to ease the migration of InfluxDB v1 workloads to InfluxDB 3. > [!Warning] > #### Before upgrading to InfluxDB 1.11 > > The last public release of InfluxDB v1 was v1.8.10. Upgrading from v1.8.10 to > v1.11.7 is a large jump and should be done with care. Consider doing > one or more of the following before upgrading: > > * [Back up your data](/influxdb/v1/administration/backup_and_restore/) > * Create a clone of your current InfluxDB using InfluxDB 1.11 with identical > configuration options. Dual-write to your current InfluxDB > instance and your new 1.11 instance. Test writing and querying data with > InfluxDB 1.11. > > #### No 32-bit builds > > InfluxData no longer provides builds of InfluxDB v1 for 32-bit architectures. > All official build packages are for 64-bit architectures. ### Features * **InfluxQL improvements:** * Upgrade to InfluxQL 1.2.0. * Support database and retention policy wildcards in `SHOW MEASUREMENTS`–for example:`SHOW MEASUREMENTS ON *.*` * Optimize `SHOW FIELD KEY CARDINALITY`. * `SHOW TAG VALUES` returns results from one specific retention policy. * Support `WITH KEY` clause in with `SHOW TAG KEYS`. * Support Hyper Log operators: `count_hll`, `sum_hll`, `merge_hll`. * Use `count_hll` for `SHOW SERIES CARDINALITY` queries. * **Logging improvements:** * Log slow queries even without query logging enabled. * Always log the log level. * Add version number to `/debug/vars` output. * Include `crypto` diagnostics in `/debug/vars` output. * Add the ability to log queries killed by `query-timeout`. * Add logging to compaction. * **New InfluxDB inspection tools:** * Add [`influx_inspect report-db` command](/influxdb/v1/tools/influx_inspect/#report-db)to estimate the cardinality of a 1.x database when migrated to InfluxDB Cloud (TSM). * Add the [`influx_inspect check-schema`](/influxdb/v1/tools/influx_inspect/#check-schema)and [`influx_inspect merge-schema`](/influxdb/v1/tools/influx_inspect/#merge-schema)commands to check for type conflicts between shards. * **New configuration options:** * Add [`total-buffer-bytes`](/influxdb/v1/administration/config/#total-buffer-bytes)configuration option to set the total number of bytes to allocate to subscription buffers. * Add [`termination-query-log`](/influxdb/v1/administration/config/#termination-query-log)configuration option to enable dumping running queries to log on `SIGTERM`. * Add [`max-concurrent-deletes`](/influxdb/v1/administration/config/#max-concurrent-deletes)configuration option to set delete concurrency. * Add [Flux query configuration settings](/influxdb/v1/administration/config/#flux-query-management-settings). * Add [`compact-series-file`](/influxdb/v1/administration/config/#compact-series-file)configuration option to enable or disable series file compaction on startup. * Add [`prom-read-auth-enabled` configuration option](/influxdb/v1/administration/config/#prom-read-auth-enabled)to authenticate Prometheus remote read. * **Flux improvements:** * Upgrade Flux to v0.194.5. * Update Flux components to use `flux/array` instead of `arrow/array`. * Add Flux query support to the `influx` REPL. * Add [new pushdown capabilities](/influxdb/v1/flux/guides/optimize-queries/#start-queries-with-pushdowns)to improve Flux query performance. * **InfluxDB v2 compatibility API updates:** * Partially support create, retrieve, update, delete, and list operations in the[`/v2/api/buckets` compatibility API](/influxdb/v1/api/buckets-v2-compatible/)and correctly handle requests to unsupported `/v2/api/buckets` endpoints. * Implement the [`v2/api/delete` compatibility API](/influxdb/v1/api/delete-v2-compatible/). * **Additional internal metrics:** * Ingress metrics by measurement. * Measurement metrics by login. * Series creation ingress metrics. * Add subscription buffer size usage metric. * **Build improvements:** * Enable static-pie builds for InfluxDB 1.x. * Make Windows and ARM64 builds possible. * **Miscellaneous additions and optimizations:** * Add multiple UDP writers. * Optimize series iteration. * Optimize saving changes to `fields.idx`. ### Bug Fixes * Prevent `GROUP BY` from returning multiple results per group in some circumstances. * Add `curl` dependency to InfluxDB packages for use in systemd scripts. * Use `defer` to unlock mutexes. * Ensure `influxd-ctl backup` creates a usable backup when only the `-shard`option is provided. * `influx_inspect verify -dir` no longer appends `/data` to the directory path. Files are checked recursively, which include files in the `/data` directory and other subdirectories. * Don’t rename files on mmap failure. * Fully clean up partially opened TSI. * Remember shards that fail `Open()` and avoid repeated attempts. * Improve error messages when opening index partitions. * Create `TSI MANIFEST` files atomically * Add paths to TSI log and index file errors. * Restore in-memory Manifest on write error. * Generalize test for Windows. * Use `copy` when a rename spans volumes. * Add tests for file rename across volumes. * Log errors in continuous query statistics storage. * Remove breaking argument validation for `_fieldKeys` iterator. * `ListBuckets` properly returns a wrapped response. * Don’t write skipped shard messages to the line protocol output destination. * Add warning if `fields.idxl` is encountered. * Prevent world-writable `MANIFEST` files. * Do not escape CSV output. * Avoid SIGBUS when reading non-standard series segment files. * Prevent retention service from creating orphaned shard files. * Sync index file before closing. * Abort processing a write request when encountering a precision error. * `MeasurementsCardinality` should not be less than 0. * Replace unprintable and invalid characters in errors. * Keep TSI reference or don’t close `TagValueSeriesIDIterator` on error. * Eliminate race condition on `Monitor.globalTags`. * Correctly handle `MaxSeriesPerDatabaseExceeded`. * Return underlying errors when creating a subscription. * Fix a race condition that caused restore commands to fail. * Create shards without overlapping time ranges. * Improve error messages for snapshots with no data. * Detect misquoted tag values and return an error. * Use `debug` log level in `influx_tools compact-shard`. * Allow limiting shards during Flux iterator reads. * When restoring data, force the `-db` parameter when `-newdb` is used. * Sync series segments to disk after writing. * Fix deadlock in `influx_inspect dumptsi`. * Prevent errors when restoring from a portable backup. * Copy names from mmapped memory before closing the iterator. * TSI should compact log files that are old or too large. * Compact old `tsl` files without new writes. * Enforce a hard limit on field size while parsing line protocol. * Ensure log formatting (JSON) is respected. * Ensure systemd handles HTTPS and 40x response codes and blocks indefinitely while waiting for API repsonses. * Avoid compaction queue stats flutter. * Require database authorization to see continuous queries. * Return correct count of `ErrNotExecuted`. * Address various `staticcheck` warnings. * Properly shutdown multiple HTTP servers. * Make error messaging for missing shards consistent. * Fix redundant registrations for prometheus collector metrics. * Improved performance for sorted merge iterator. * Grow tag index buffer when needed. * Fixes key collisions when serializing `/debug/vars`. * Return an error when trying to delete a nonexistent shard ID. * Improved CORS support: * Add User-Agent to allowed CORS headers. * Allow CORS in v2 compatibility endpoints. * Allow the `PATCH` request method. ### Other * Upgrade to Go 1.22.7. * Detect and use the Rust version defined in Flux (1.73). * Upgrade Flux to v0.194.5. * Upgrade multiple dependencies for maintenance and security. * Upgrade `xbuilder` for parity with InfluxDB Enterprise. * Upgrade to latest crossbuilder. * Use community maintained `golang-jwt` * Generate `influxdb.${CIRCLE_TAG}.digests` for each release. * Update contribution guidelines. * Remove deprecated `influx_stress` tool. ## v1.8.10 ### Bug fixes * Ensure `curl` dependency for InfluxDB packages. * Ensure snapshot service reads payload correctly. * Handle 40x errors by blocking indefinitely in systemd scripts. * Fix the following bugs in `influxd restore`: * Correctly validate parameters. * Improve error message when returning empty snapshots. * Fix temporary file deletion on Windows. ## v1.8.9 ### Bug fixes * Prevent silently dropped writes when there are overlapping shards. * Resolve `influxd restore -portable` backup bug. * Ensure systemd-startup script is executable by group and others. * Handle https in systemd wrapper, and prevent it from looping forever. * Log error instead of panic when restoring a backup to InfluxDB Enterprise cluster using InfluxDB OSS. ## v1.8.8 Due to encountering several issues with build dependencies in v.1.8.8, this version will not be released. ## v1.8.7 ### Bug fixes * Renamed ARM RPMs with yum-compatible names. * Convert ARM arch names for RPMs during builds via Docker. * Systemd unit now blocks on startup until HTTP endpoint is ready. * Updated protobuf libraries to current version. ## v1.8.6 This release is for InfluxDB Enterprise 1.8.6 customers only. No OSS-specific changes were made for InfluxDB 1.8.6–updates were made to the code base to support [InfluxDB Enterprise 1.8.6](/enterprise_influxdb/v1/about-the-project/release-notes/#v186). ## v1.8.5 ### Features * Add the ability to find which measurements or shards are contributing to disk size with the new [`influx_inspect report-disk`](/influxdb/v1/tools/influx_inspect/#report-disk)command. Useful for capacity planning and managing storage requirements. * Add support to [`influx_inspect export`](/influxdb/v1/tools/influx_inspect/#export)to write to standard out (`stdout`) by adding a hyphen after the[`-out`](/influxdb/v1/tools/influx_inspect/#--out-export_dir-or--out--) flag. Using this option writes to `stdout`, and sends error and status messages to standard error (`stderr`). * Update HTTP handler for `/query` to[log query text for POST requests](/influxdb/v1/administration/logs/#http-access-log-format). * Optimize shard lookups in groups containing only one shard. ### Bug fixes * Update meta queries (for example, `SHOW TAG VALUES`, `SHOW TAG KEYS`,`SHOW SERIES CARDINALITY`, `SHOW MEASUREMENT CARDINALITY`, and `SHOW MEASUREMENTS`) to check the query context when possible to respect timeout values set in the[`query-timeout` configuration parameter](/influxdb/v1/administration/config/#query-timeout). Note, meta queries will check the context less frequently than regular queries, which use iterators, because meta queries return data in batches. * Previously, successful writes were incorrectly incrementing the `WriteErr`statistics. Now, successful writes correctly increment the `writeOK` statistics. * Correct JSON marshalling error format. * Previously, a GROUP BY query with an offset that caused an interval to cross a daylight savings change inserted an extra output row off by one hour. Now, the correct GROUP BY interval start time is set before the time zone offset is calculated. * Improved error logging for TCP connection closures. * Fix `regexp` handling to comply with PromQL. * Previously, when a SELECT INTO query generated an unsupported value, for example, `+/- Inf`, the query failed silently. Now, an error occurs to notify that the value cannot be inserted. * Resolve the “snapshot in progress” error that occurred during a backup. * Fix data race when accessing tombstone statistics (`TombstoneStat`). * Minimize lock contention when adding new fields or measurements. * Resolve a bug causing excess resource usage when an error occurs while reporting an earlier error. ## v1.8.4 ### Features * Add `stat_total_allocated` to Flux logging. To ensure Flux logging is enabled, set both `flux-enabled` and `flux-log-enabled` to `true` in the [InfluxDB configuration file](/influxdb/v1/administration/config). For more information about InfluxDB logging, see [Log and trace with InfluxDB](/influxdb/v1/administration/logs). ### Bug fixes * Add durations to Flux logging, including the log compilation, execution, and total request duration. Previously, the following stats were incorrectly logging `0.000ms`: * `stat_total_duration` * `stat_compile_duration` * `stat_execute_duration` Now, these durations are logged correctly. ## v1.8.3 ### Features * Use latest version of InfluxQL package. * Add `-lponly` flag to [`influx export`](/influxdb/v2/reference/cli/influx/export/) sub-command. * Add the ability to [track number of values](/influxdb/v1/administration/monitor/measurements-internal/#valueswrittenok) written via the [`/debug/vars` HTTP endpoint](/influxdb/v1/api/debug/). * Update UUID library from [github.com/satori/go.uuid](https://github.com/satori/go.uuid) to [github.com/gofrs/uuid](https://github.com/gofrs/uuid). ### Bug fixes * ArrayFilterCursor truncation for multi-block data. * Multi-measurement queries now return all applicable series. * Lock map before writes. ## v1.8.2 ### Bug fixes * Revert configuration change to `DefaultSeriesIDSetCacheSize` that caused some environments to experience increased memory usage. ## v1.8.1 > [!Warning] > InfluxDB 1.8.1 introduced a bug that could potentially increase memory usage.**If you installed this release**, install [v1.8.2](#v182), which includes the > features, performance improvements, and bug fixes below. ### Features * Allow users to add custom HTTP response headers to comply with internal security policies. ### Performance improvements * InfluxQL query planner plans each field in parallel to improve planning time. * Improved performance of DELETE/DROP by batching tombstone writes. ### Bug fixes * Fix to Flux `buckets()` function causing a panic. * TSI: Addressed a couple of edge cases which resulted in segfaults. * HTTP: Simplify Authorizer. * Wait to delete epoch before dropping shard. * Improve error handling when creating snapshots. * Remove all Go 1.12 references from build. ## v1.8.0 ### Features #### Flux v0.65 ready for production use This release updates support for the Flux language and queries. To learn about Flux design principles and see how to get started with Flux, see[Introduction to Flux](/influxdb/v1/flux/). * Use the new [`influx -type=flux`](/influxdb/v1/tools/influx-cli/#flags) option to enable the Flux REPL shell for creating Flux queries. * Flux v0.65 includes the following capabilities: * Join data residing in multiple measurements, buckets, or data sources * Perform mathematical operations using data gathered across measurements/buckets * Manipulate Strings through an extensive library of string related functions * Shape data through `pivot()` and other functions * Group based on any data column: tags, fields, etc. * Window and aggregate based on calendar months, years * Join data across Influx and non-Influx sources * Cast booleans to integers * Query geo-temporal data (experimental) * Many additional functions for working with data > [!Note] > We’re evaluating the need for Flux query management controls equivalent to > existing InfluxQL [query management controls](/influxdb/v1/troubleshooting/query_management/#configuration-settings-for-query-management)based on your feedback. Please join the discussion on[InfluxCommunity](https://community.influxdata.com/),[Slack](https://influxcommunity.slack.com/), or [GitHub](https://github.com/influxdata/flux). > InfluxDB Enterprise customers, please contact [support@influxdata.com](mailto:support@influxdata.com). #### Forward compatibility * [InfluxDB 2.x API compatibility endpoints](/influxdb/v1/api/buckets-v2-compatible/) are now part of the InfluxDB 1.x line. This allows you to leverage the new InfluxDB 2.0 [client libraries](/influxdb/v1/tools/api_client_libraries/)for both writing and querying data with Flux. Take advantage of the latest client libraries while readying your implementation for a move to InfluxDB 2.0 Cloud when you’re ready to scale. #### Operational improvements * Add [`influx inspect verify-tombstone` command](/influxdb/v1/tools/influx_inspect/#verify-tombstone) * Add [offline series compaction to `influx_inspect buildtsi`](/influxdb/v1/administration/compact-series-file/). If you’re currently using the Time Series Index [(tsi1)](/influxdb/v1/concepts/time-series-index/), the index files grow over time and aren’t automatically compacted. This tool enables an administrator to perform a compaction while the database is offline. * Add support for connecting to a custom HTTP endpoint using `-url-prefix` in the [`influx` CLI](/influxdb/v1/tools/influx-cli/). This allows the Influx CLI to connect to an InfluxDB instance running behind a reverse proxy with a custom subpath `/` endpoint. #### Security enhancements * Add support for [TLS 1.3 configuration](/influxdb/v1/administration/config/#transport-layer-security-tls-settings) and update current list of [Go ciphers](https://golang.org/pkg/crypto/tls/#pkg-constants). * Expand support for bound parameters in queries to prevent injection attacks. #### Other updates * Update Go version to 1.13.8. ### Bug fixes * Skip `WriteSnapshot` during backup if `snapshotter` is busy. This fix eliminates contention between snapshot and backup processes, allowing backups to complete more reliably. * Rebuild the series index when data is actually deleted (not when deleted data is only found in cache or outside of time range). * Parse Accept header correctly. * Upgrade compaction error log from `Info` to `Warn`. * Remove double increment of `meta` index. * Improve series cardinality limit for `inmem` index. * Ensure all block data returned. * Reduce `influxd` and `influx` startup time if Flux isn’t used. * Fix bugs in `-compact-series-file` flag. * Fix a SIGSEGV when accessing `tsi` active log. * Verify precision in write requests. ## v1.7.10 ### Bug fixes * Fix failing corrupt data file renaming process. * Make shard digests safe for concurrent use. * Fix defect in TSI index where negative equality filters (`!=`) could result in no matching series. * Fix compaction logic on infrequent cache snapshots which resulted in frequent full compactions rather than [level compactions](/influxdb/v1/concepts/storage_engine/#compactions). * Fix for series key block data being truncated when read into an empty buffer. Ensure all block data is returned. * During compactions, skip TSM files with block read errors from previous compactions. ## v1.7.9 * This release is built using Go 1.12.10 which eliminates the[HTTP desync vulnerability](https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn). ### Bug fixes * Guard against compaction burst throughput limit. * Replace TSI compaction wait group with counter. * Update InfluxQL dependency. * Add option to authenticate debug/pprof and ping endpoints. * Honor `insecure-skip-verify` even if custom TLS config is specified. ### Features * Update Go version to 1.12.10. * Remove Godeps file. * Update Flux version to v0.50.2. ## v1.7.8 InfluxDB now rejects all non-UTF-8 characters. To successfully write data to InfluxDB, use only UTF-8 characters in database names, measurement names, tag sets, and field sets. ### Bugfixes * Fix Prometheus read panic. * Remove stray `fmt.Println` in `tsm1.StringArrayEncodeAll`. * Fix issue where fields re-appear after `DROP MEASUREMENT`. * Remove a dubugging `println` call. * Subquery ordering with aggregates in descending mode was wrong. * Fix the http handler to not mislabel series as partial. * Make `SHOW SERIES EXACT CARDINALITY` count only distinct series. * Fix time range exceeding min/max bounds. ### Features * Update Flux version to v0.36.2 ## 1.7.7 ### Known issues * The Flux Technical Preview was not advanced and remains at version 0.24.0. Next month’s maintenance release will update the preview. * After upgrading, customers have experienced an excessively large output additional lines due to a `Println` statement introduced in this release. For a possible workaround, see https://github.com/influxdata/influxdb/issues/14265#issuecomment-508875853. Next month’s maintenance release will address this. ### Bug fixes * Fix the sort order for aggregates so that they are sorted by tag and then time. * Use the timezone when evaluating time literals in subqueries. * Fix CSV decoder bug where empty tag values cause an array index panic. * Fix open/close race in `SeriesFile`. * Sync series segment after truncate. * Fix the ordering for selectors within a subquery with different outer tags. ## 1.7.6 If your InfluxDB OSS server is using the default in-memory index (`[data].index-version = "inmem"`), this release includes the fix for InfluxDB 1.7.5 servers that stopped responding without warning. ### Features * Upgrade Flux to `0.24.0` and remove the platform dependency. * If Flux is enabled, use Chronograf 1.7.11 or later. * When using Flux, there is a known issue that using `now` will cause a panic. The proper syntax is `now()`. * Track remote read requests to Prometheus remote read handler. ### Bug fixes * Ensure credentials are correctly passed when executing Flux HTTP requests in the `influx` CLI with the `-type=flux` option. * Back port of data generation improvements: renamed files for consistency between versions, added `time-interval` schema option, and updated schema example documentation. * Fix security vulnerability when `[http]shared-secret` configuration setting is blank. * Add nil check for `tagKeyValueEntry.setIDs()`. * Extend the Prometheus remote write endpoint to drop unsupported Prometheus values (`NaN`,`-Inf`, and `+Inf`) rather than reject the entire batch. * If write trace logging enabled (`[http] write-tracing = true`), then summaries of dropped values are logged. * If a batch of values contains values that are subsequently dropped, HTTP status code `204` is returned. * Update predicate key mapping to match InfluxDB `2.x` behavior. * Fix panic in Prometheus read API. * Add a version constraint for influxql. ## 1.7.5 **Update (2019-04-01):** If your InfluxDB OSS server is using the default in-memory index (`[data].index-version = "inmem"`), then do not upgrade to this release. Customers have reported that InfluxDB 1.7.5 stops responding without warning. For details, see [GitHub issue #13010](https://github.com/influxdata/influxdb/issues/13010). The [planned fix](https://github.com/influxdata/influxdb/issues/13053) will be available soon. ### Bug fixes * Update `tagKeyValue` mutex to write lock. * Fix some more shard epoch races. ## 1.7.4 ### Features * Allow TSI bitset cache to be configured. See: [InfluxDB Configuration `[data]`](/influxdb/v1/administration/config/#tsi-tsi1-index-settings) ### Bug fixes * Remove copy-on-write when caching bitmaps in TSI. * Use `systemd` for Amazon Linux 2. * Revert “Limit force-full and cold compaction size.” * Convert `TagValueSeriesIDCache` to use string fields. * Ensure that cached series id sets are Go heap backed. ## 1.7.3 ### Important update If you have not installed this release, then install the 1.7.4 release. **If you are currently running this release, then upgrade to the 1.7.4 release as soon as possible.** * A critical defect in the InfluxDB 1.7.3 release was discovered and our engineering team fixed the issue in the 1.7.4 release. Out of high concern for your data and projects, upgrade to the 1.7.4 release as soon as possible. * **Critical defect:** Shards larger than 16GB are at high risk for data loss during full compaction. The full compaction process runs when a shard go “cold” – no new data is being written into the database during the time range specified by the shard. * **Post-mortem analysis:** InfluxData engineering is performing a post-mortem analysis to determine how this defect was introduced. Their discoveries will be shared in a blog post. ### Breaking changes * Fix invalid UTF-8 bytes preventing shard opening. Treat fields and measurements as raw bytes. ### Features * Update Flux to `0.12.0` ### Bug fixes * Limit force-full and cold compaction size. * Add user authentication and authorization support for Flux HTTP requests. * Call `storage.Group` API to correctly map group mode. * Marked functions that always return floats as always returning floats. * Add support for optionally logging Flux queries. * Fix cardinality estimation error. ## 1.7.2 ### Bug fixes * Update to Flux 0.7.1. * Conflict-based concurrency resolution adds guards and an epoch-based system to coordinate modifications when deletes happen against writes to the same points at the same time. * Skip and warn that series file should not be in a retention policy directory. * Checks if measurement was removed from index, and if it was, then cleans up out of fields index. Also fix cleanup issue where only prefix was checked when matching measurements like “m1” and “m10”. * Error message to user that databases must be run in non-mixed index mode to allow deletes. * Update platform dependency to simplify Flux support in Enterprise. * Verify series file in presence of tombstones. * Fix `ApplyEnvOverrides` when a type that implements Unmarshaler is in a slice to not call `UnMarshaltext` when the environment variable is set to empty. * Drop NaN values when writing back points and fix the point writer to report the number of points actually written and omits the ones that were dropped. * Query authorizer was not properly passed to subqueries so rejections did not happen when a subquery was the one reading the value. Max series limit was not propagated downward. ## 1.7.1 ### Bug fixes * Simple8B `EncodeAll` incorrectly encodes entries: For a run of `1s`, if the 120th or 240th entry is not a `1`, the run will be incorrectly encoded as selector `0` (`240 1s`) or selector `1` (`120 1s`), resulting in a loss of data for the 120th or 240th value. Manifests itself as consuming significant CPU resources and as compactions running indefinitely. ## 1.7.0 ### Breaking changes Chunked query was added into the Go client v2 interface. If you compiled against the Go client v2 previously, you need to recompile using the updated interface. ### Features #### Flux v0.7 technical preview Support for the Flux language and queries has been added in this release. To begin exploring Flux 0.7 (technical preview): * Enable Flux using the new configuration setting[`[http] flux-enabled = true`](/influxdb/v1/administration/config/#flux-enabled). * Use the new [`influx -type=flux`](/influxdb/v1/tools/influx-cli/) option to enable the Flux REPL shell for creating Flux queries. * Read about Flux and the Flux language, enabling Flux, or jump into the getting started and other guides. #### Time Series Index (TSI) query performance and throughputs improvements * Faster index planning for queries against indexes with many series that share tag pairs. * Reduced index planning for queries that include previously queried tag pairs — the TSI index now caches partial index results for later reuse. * Performance improvements required a change in on-disk TSI format to be used. * **To take advantage of these improvements**: * Rebuild your indexes or wait for a TSI compaction of your indexes, at which point the new TSI format will be applied. * Hot shards and new shards immediately use the new TSI format. #### Other features * Enable the storage service by default. * Ensure read service regular expressions get optimized. * Add chunked query into the Go client v2. * Add `access-log-status-filters` config setting to create an access log filter. * Compaction performance improvements for Time Series Index (TSI). * Add roaring bitmaps to TSI index files. ### Bug fixes * Missing `hardwareAddr` in `uuid` v1 generation. * Fix the inherited interval for derivative and others. * Fix subquery functionality when a function references a tag from the subquery. * Strip tags from a subquery when the outer query does not group by that tag. ## 1.6.6 ### Bug fixes * Marked functions that always return floats as always returning floats. * Fix cardinality estimation error. * Update `tagKeyValue` mutex to write lock. ## 1.6.5 ### Features * Reduce allocations in TSI `TagSets` implementation. ### Bug fixes * Fix panic in `IndexSet`. * Pass the query authorizer to subqueries. * Fix TSM1 panic on reader error. * Limit database and retention policy names to 255 characters. * Update Go runtime to 1.10.6. ## 1.6.4 ### Features * Set maximum cache size using `-max-cache-size` in `buildtsi` when building TSI index. ### Bug fixes * Fix `tsi1` sketch locking. * Fix subquery functionality when a function references a tag from the subquery. * Strip tags from a subquery when the outer query does not group by that tag. * Add `-series-file` flag to `dumptsi` command help. * Cleanup failed TSM snapshots. * Fix TSM1 panic on reader error. * Fix series file tombstoning. * Fixing the stream iterator to not ignore the error. * Do not panic when a series ID iterator is nil. * Fix append of possible nil iterator. ## 1.6.3 ### Features * Remove TSI1 HLL sketches from heap. ### Bug fixes * Fix the inherited interval for derivative and others. The inherited interval from an outer query should not have caused an inner query to fail because inherited intervals are only implicitly passed to inner queries that support group by time functionality. Since an inner query with a derivative doesn’t support grouping by time and the inner query itself doesn’t specify a time, the outer query shouldn’t have invalidated the inner query. * Fix the derivative and others time ranges for aggregate data. The derivative function and others similar to it would preload themselves with data so that the first interval would be the start of the time range. That meant reading data outside of the time range. One change to the shard mapper made in v1.4.0 caused the shard mapper to constrict queries to the intervals given to the shard mapper. This was correct because the shard mapper can only deal with times it has mapped, but this broke the functionality of looking back into the past for the derivative and other functions that used that functionality. The query compiler has been updated with an additional attribute that records how many intervals in the past will need to be read so that the shard mapper can include extra times that it may not necessarily read from, but may be queried because of the above described functionality. ## 1.6.2 ### Features * Reduce allocations in TSI TagSets implementation. ### Bug fixes * Ensure orphaned series cleaned up with shard drop. ## 1.6.1 ### Features * Improve LogFile performance with bitset iterator. * Add TSI index cardinality report to `influx_inspect`. * Update to Go 1.10. * Improve performance of `buildtsi` and TSI planning. * Improve performance of read service for single measurements. * Remove max concurrent compaction limit. * Provide configurable TLS options. * Add option to hint MADV\_WILLNEED to kernel. ### Bug fixes * Improve series segment recovery. * Fix windows mmap on zero length file. * Ensure Filter iterators executed as late as possible. * Document UDP precision setting in config. * Allow tag keys to contain underscores. * Fix a panic when matching on a specific type of regular expression. ## 1.6.0 ### Breaking changes * If math is used with the same selector multiple times, it will now act as a selector rather than an aggregate. See [#9563](https://github.com/influxdata/influxdb/pull/9563) for details. * For data received from Prometheus endpoints, every Prometheus measurement is now stored in its own InfluxDB measurement rather than storing everything in the `_` measurement using the Prometheus measurement name as the `__name__` label. ### Features * Support proxy environment variables in the `influx` client. * Implement basic trigonometry functions. * Add ability to delete many series with predicate. * Implement `floor`, `ceil`, and `round` functions. * Add more math functions to InfluxQL. * Allow customizing the unix socket group and permissions created by the server. * Add `suppress-write-log` option to disable the write log when the log is enabled. * Add additional technical analysis algorithms. * Validate points on input. * Log information about index version during startup. * Add key sanitization to `deletetsm` command in `influx_inspect` utility. * Optimize the `spread` function to process points iteratively instead of in batch. * Allow math functions to be used in the condition. * Add HTTP write throttle settings: `max-concurrent-write-limit`, `max-enqueued-write-limit`, and `enqueued-write-timeout`. * Implement `SHOW STATS FOR indexes`. * Add `dumptsmwal` command to `influx_inspect` utility. * Improve the number of regex patterns that are optimized to static OR conditions. ### Bug fixes * Support setting the log level through the environment variable. * Fix panic when checking fieldsets. * Ensure correct number of tags parsed when commas used. * Fix data race in WAL. * Allow `SHOW SERIES` kill. * Revert “Use MADV\_WILLNEED when loading TSM files”. * Fix regression to allow now() to be used as the group by offset again. * Delete deleted shards in retention service. * Ignore index size in `Engine.DiskSize()`. * Enable casting values from a subquery. * Avoid a panic when using show diagnostics with text/csv. * Properly track the response bytes written for queries in all format types. * Remove error for series file when no shards exist. * Fix the validation for multiple nested distinct calls. * TSM: `TSMReader.Close` blocks until reads complete. * Return the correct auxiliary values for `top` and `bottom`. * Close TSMReaders from `FileStore.Close` after releasing FileStore mutex. ## 1.5.5 ### Features * Reduce allocations in TSI `TagSets` implementation. ### Bug fixes * Copy return value of `IndexSet.MeasurementNamesByExpr`. * Ensure orphaned series cleaned up with shard drop. * Fix the derivative and others time ranges for aggregate data. * Fix the stream iterator to not ignore errors. * Do not panic when a series ID iterator is `nil`. * Fix panic in `IndexSet`. * Pass the query authorizer to subqueries. * Fix TSM1 panic on reader error. ## 1.5.4 ### Features * Add `influx_inspect deletetsm` command for bulk deletes of measurements in raw TSM files. ### Bug fixes * Fix panic in readTombstoneV4. * buildtsi: Do not escape measurement names. ## 1.5.3 ### Features * Add `[http] debug-pprof-enabled` configuration setting immediately on startup. Useful for debugging startup performance issues. ### Bug fixes * Fix the validation for multiple nested `DISTINCT` calls. * Return the correct auxiliary values for `TOP` and `BOTTOM`. ## 1.5.2 ### Features * Check for root user when running `buildtsi`. * Adjustable TSI Compaction Threshold. ### Bug fixes * backport: check for failure case where backup directory has no manifest files. * Fix regression to allow `now()` to be used as the group by offset again. * Revert `Use MADV_WILLNEED when loading TSM files`. * Ignore index size in `Engine.DiskSize()`. * Fix `buildtsi` partition key. * Ensure that conditions are encoded correctly even if the AST is not properly formed. ## 1.5.1 ### Bug fixes * Allow time variable to be case insensitive again. * Support setting the log level through the environment variable. * Ensure correct number of tags parsed. * Fix panic when checking fieldsets. * Fix data race in WAL. ## 1.5.0 ### Breaking changes The default logging format has been changed. See [Logging and tracing in InfluxDB](/influxdb/v1/administration/logs) for details. ### Features * Improve CLI connection warnings. * Backup utility prints a list of backup files. * Backup and restore for OSS produces and consumes data in the Enterprise-compatible backup format. * Restore runs in online mode and does not delete existing databases. * Export functionality using `start` and `end` to filter exported data by `timestamp`. * Handle high cardinality deletes in TSM engine. * Improve in-memory index startup performance for high cardinality. * Add further TSI support for streaming and copying shards. * Schedule a full compaction after a successful import. * Add Prometheus `/metrics` endpoint. * Add ability to generate shard digests. * Allow setting the node ID in the InfluxDB CLI program. ### Bug fixes * Refuse extra arguments to influx CLI. * Fix space required after regex operator. Thanks @stop-start! * Fix `panic: sync: WaitGroup is reused before previous Wait has returned`. * Fix race condition in the merge iterator close method. * Fix query compilation so multiple nested distinct calls is allowable * Fix CLI to allow quoted database names in use statement. * Updated client `4xx` error message when response body length is zero. * Remove extraneous newlines from the log. * Allow lone Boolean literals in a condition expression. * Improve performance when writes exceed `max-values-per-tag` or `max-series`. * Prevent a panic when a query simultaneously finishes and is killed at the same time. * Fix missing sorting of blocks by time when compacting. * WAL: update `lastWriteTime` behavior ## 1.4.3 ### Configuration Changes #### `[data]` Section `cache-snapshot-memory-size`: default value changed from `25m` to `256m`. ### Bug fixes * Fix higher disk I/O utilization ## 1.4.2 Refer to the 1.4.0 breaking changes section if `influxd` fails to start with an `incompatible tsi1 index MANIFEST` error. ### Bug fixes * Fix `panic: runtime error: slice bounds out of range` when running `dep init` ## 1.4.1 ### Bug fixes * Fix descending cursors and range queries via IFQL RPC API. ## 1.4.0 ### TSI Index This feature remains experimental in this release. However, a number of improvements have been made and new meta query changes will allow for this feature to be explored at more depth than previously possible. It is not recommended for production use at this time. We appreciate all of the feedback we receive on this feature. Please keep it coming! ### Breaking changes You can no longer specify a different `ORDER BY` clause in a subquery than the one in the top level query. This functionality never worked properly, but was not explicitly forbidden. As part of the ongoing development of the `tsi1` index, the implementation of a Bloom Filter, used to efficiently determine if series are not present in the index, was altered. While this significantly increases the performance of the index and reduces its memory consumption, the existing `tsi1` indexes created while running previous versions of the database are not compatible with 1.4.0. Users with databases using the `tsi1` index must go through the following process to upgrade to 1.4.0: 1. Stop `influxd`. 2. Remove all `index` directories on databases using the `tsi1` index. With default configuration these can be found in`/var/lib/influxdb/data/DB_NAME/RP_NAME/SHARD_ID/index` or `~/.influxdb/data/DB_NAME/RP_NAME/SHARD_ID/index`. It’s worth noting at this point how many different `shard_ids` you visit. 3. Run the `influx_inspect inmem2tsi` tool using the shard’s data and WAL directories for `-datadir` and `-waldir`, respectively. Given the example in step (2) that would be`influx_inspect inmem2tsi -datadir /var/lib/influxdb/data/DB_NAME/RP_NAME/SHARD_ID -waldir /path/to/influxdb/wal/DB_NAME/RP_NAME/SHARD_ID`. 4. Repeat step (3) for each shard that needs to be converted. 5. Start `influxd`. Users with existing `tsi1` shards, who attempt to start version 1.4.0 without following the steps above, will find the shards refuse to open and will most likely see the following error message: `incompatible tsi1 index MANIFEST`. ### Configuration Changes #### `[collectd]` Section * `parse-multivalue-plugin` option was added with a default of `split`. When set to `split`, multivalue plugin data (e.g. `df free:5000,used:1000`) will be split into separate measurements (e.g., `df_free, value=5000` and `df_used, value=1000`). When set to `join`, multivalue plugin will be stored as a single multi-value measurement (e.g., `df, free=5000,used=1000`). ### Features * Add `influx_inspect inmem2tsi` command to convert existing in-memory (TSM-based) shards to the TSI (Time Series Index) format. * Add support for the Prometheus remote read and write APIs. * Support estimated and exact SHOW CARDINALITY commands for measurements, series, tag keys, tag key values, and field keys. * Improve `SHOW TAG KEYS` performance. * Add `EXPLAIN ANALYZE` command, which produces a detailed execution plan of a `SELECT` statement. * Improved compaction scheduling. * Support Ctrl+C to cancel a running query in the Influx CLI. * Allow human-readable byte sizes in configuration file. * Respect X-Request-Id/Request-Id headers. * Add ‘X-Influxdb-Build’ to http response headers so users can identify if a response is from an OSS or Enterprise service. * All errors from queries or writes are available via X-InfluxDB-Error header, and 5xx error messages will be written to server logs. * Add `parse-multivalue-plugin` to allow users to choose how multivalue plugins should be handled by the collectd service. * Make client errors more helpful on downstream errors. * Allow panic recovery to be disabled when investigating server issues. * Support http pipelining for `/query` endpoint. * Reduce allocations when reading data. * Mutex profiles are now available. * Batch up writes for monitor service. * Use system cursors for measurement, series, and tag key meta queries. * Initial implementation of explain plan. * Include the number of scanned cached values in the iterator cost. * Improve performance of `Include` and `Exclude` functions. * Report the task status for a query. * Reduce allocations, improve `readEntries` performance by simplifying loop * Separate importer log statements to stdout and stderr. * Improve performance of Bloom Filter in TSI index. * Add message pack format for query responses. * Implicitly decide on a lower limit for fill queries when none is present. * Streaming `inmem2tsi` conversion. * Sort & validate TSI key value insertion. * Handle nil MeasurementIterator. * Add long-line support to client importer. * Update to go 1.9.2. * InfluxDB now uses MIT licensed version of BurntSushi/toml. ### Bug fixes * Change the default stats interval to 1 second instead of 10 seconds. * illumos build broken on `syscall.Mmap`. * Prevent privileges on non-existent databases from being set. * `influxd backup` tool now separates out logging to `stdout` and `stderr`. Thanks @xginn8! * Dropping measurement used several GB disk space. * Fix the CQ start and end times to use Unix timestamps. * `influx` CLI case-sensitivity. * Fixed time boundaries for continuous queries with time zones. * Return query parsing errors in CSV formats. * Fix time zone shifts when the shift happens on a time zone boundary. * Parse time literals using the time zone in the `SELECT` statement. * Reduce CPU usage when checking series cardinality * Fix backups when snapshot is empty. * Cursor leak, resulting in an accumulation of `.tsm.tmp` files after compactions. * Improve condition parsing. * Ensure inputs are closed on error. Add runtime GC finalizer as additional guard to close iterators. * Fix merging bug on system iterators. * Force subqueries to match the parent queries ordering. * Fix race condition accessing `seriesByID` map. * Fix deadlock when calling `SeriesIDsAllOrByExpr`. * Fix `influx_inspect export` so it skips missing files. * Reduce how long it takes to walk the varrefs in an expression. * Address `panic: runtime error: invalid memory address or nil pointer dereference`. * Drop Series Cause Write Fail/Write Timeouts/High Memory Usage. * Fix increased memory usage in cache and wal readers. * An OSS read-only user should be able to list measurements on a database. * Ensure time and tag-based condition can be used with tsi1 index when deleting. * Prevent deadlock when doing math on the result of a subquery. * Fix a minor memory leak in batching points in TSDB. * Don’t assume `which` is present in package post-install script. * Fix missing man pages in new packaging output. * Fix use of `INFLUXD_OPTS` in service file. * Fix WAL panic: runtime error: makeslice: cap out of range. * Copy returned bytes from TSI meta functions. * Fix data deleted outside of time range. * Fix data dropped incorrectly during compaction. * Prevent deadlock during Collectd, Graphite, openTSDB, and UDP shutdown. * Remove the pidfile after the server has exited. * Return `query.ErrQueryInterrupted` for successful read on `InterruptCh`. * Fix race inside Measurement index. * Ensure retention service always removes local shards. * Handle utf16 files when reading the configuration file. * Fix `panic: runtime error: slice bounds out of range`. ## 1.3.7 ### Release Notes Bug fix identified via Community and InfluxCloud. The build artifacts are now consistent with v1.3.5. ### Bug fixes * Don’t assume `which` is present in package post-install script. * Fix use of `INFLUXD_OPTS` in service file. * Fix missing man pages in new packaging output. * Add RPM dependency on shadow-utils for `useradd`. * Fix data deleted outside of specified time range when using `delete` * Fix data dropped incorrectly during compaction. * Return `query.ErrQueryInterrupted` for a successful read on `InterruptCh`. * Copy returned bytes from TSI meta functions. ## v1.3.6 ### Release Notes Bug fix identified via Community and InfluxCloud. ### Bug fixes * Reduce how long it takes to walk the varrefs in an expression. * Address `panic: runtime error: invalid memory address or nil pointer dereference`. * Fix increased memory usage in cache and WAL readers for clusters with a large number of shards. * Prevent deadlock when doing math on the result of a subquery. * Fix several race conditions present in the shard and storage engine. * Fix race condition on cache entry. ### Release Notes Bug fix identified via Community and InfluxCloud. ### Bug fixes * Fix race condition accessing `seriesByID` map. * Fix deadlock when calling `SeriesIDsAllOrByExpr`. ## v1.3.5 ### Release Notes Bug fix identified via Community and InfluxCloud. ### Bug fixes * Fix race condition accessing `seriesByID` map. * Fix deadlock when calling `SeriesIDsAllOrByExpr`. ## v1.3.4 ### Release Notes Bug fix identified via Community and InfluxCloud. ### Bug fixes * Fixed time boundaries for continuous queries with time zones. * Fix time zone shifts when the shift happens on a time zone boundary. * Parse time literals using the time zone in the select statement. * Fix drop measurement not dropping all data. * Fix backups when snapshot is empty. * Eliminated cursor leak, resulting in an accumulation of .tsm.tmp files after compactions. * Fix Deadlock when dropping measurement and writing. * Ensure inputs are closed on error. Add runtime GC finalizer as additional guard to close iterators. * Fix leaking tmp file when large compaction aborted. ## v1.3.3 ### Release Notes Bug fix identified via Community and InfluxCloud. ### Bug fixes * Resolves a memory leak when NewReaderIterator creates a nilFloatIterator, the reader is not closed. ## v1.3.2 ### Release Notes Minor bug fixes were identified via Community and InfluxCloud. ### Bug fixes * Interrupt “in-progress” TSM compactions. * Prevent excessive memory usage when dropping series. * Significantly improve performance of SHOW TAG VALUES. ## v1.3.1 ### Release Notes Minor bug fixes were identified via Community and InfluxCloud. ### Bug fixes * Ensure temporary TSM files get cleaned up when compaction aborted. * Address deadlock issue causing 1.3.0 to become unresponsive. * Duplicate points generated via INSERT after DELETE. * Fix the CQ start and end times to use Unix timestamps. ## v1.3.0 ### Release Notes #### TSI Version 1.3.0 marks the first official release of the new InfluxDB time series index (TSI) engine. The TSI engine is a significant technical advancement in InfluxDB. It offers a solution to the [time-structured merge tree](/influxdb/v1/concepts/storage_engine/)engine’s [high series cardinality issue](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-does-series-cardinality-matter). With TSI, the number of series should be unbounded by the memory on the server hardware and the number of existing series will have a negligible impact on database startup time. See Paul Dix’s blogpost [Path to 1 Billion Time Series: InfluxDB High Cardinality Indexing Ready for Testing](https://www.influxdata.com/path-1-billion-time-series-influxdb-high-cardinality-indexing-ready-testing/)for additional information. TSI is disabled by default in version 1.3. To enable TSI, uncomment the [`index-version` setting](/influxdb/v1/administration/config#index-version)and set it to `tsi1`. The `index-version` setting is in the `[data]` section of the configuration file. Next, restart your InfluxDB instance. ``` [data] dir = "/var/lib/influxdb/data" index-version = "tsi1" ``` #### Continuous Query Statistics When enabled, each time a continuous query is completed, a number of details regarding the execution are written to the `cq_query` measurement of the internal monitor database (`_internal` by default). The tags and fields of interest are | tag / field | description | |-----------------|--------------------------------------------------| | `db` | name of database | | `cq` | name of continuous query | | `durationNS` | query execution time in nanoseconds | | `startTime` | lower bound of time range | | `endTime` | upper bound of time range | |`pointsWrittenOK`|number of points written to the target measurement| * `startTime` and `endTime` are UNIX timestamps, in nanoseconds. * The number of points written is also included in CQ log messages. ### Removals The admin UI is removed and unusable in this release. The `[admin]` configuration section will be ignored. ### Configuration Changes * The top-level config `bind-address` now defaults to `localhost:8088`. The previous default was just `:8088`, causing the backup and restore port to be bound on all available interfaces (i.e. including interfaces on the public internet). The following new configuration options are available. #### `[http]` Section * `max-body-size` was added with a default of 25,000,000, but can be disabled by setting it to 0. Specifies the maximum size (in bytes) of a client request body. When a client sends data that exceeds the configured maximum size, a `413 Request Entity Too Large` HTTP response is returned. #### `[continuous_queries]` Section * `query-stats-enabled` was added with a default of `false`. When set to `true`, continuous query execution statistics are written to the default monitor store. ### Features * Add WAL sync delay * Add chunked request processing back into the Go client v2 * Allow non-admin users to execute SHOW DATABASES * Reduce memory allocations by reusing gzip.Writers across requests * Add system information to /debug/vars * Add modulo operator to the query language. * Failed points during an import now result in a non-zero exit code * Expose some configuration settings via SHOW DIAGNOSTICS * Support single and multiline comments in InfluxQL * Support timezone offsets for queries * Add “integral” function to InfluxQL * Add “non\_negative\_difference” function to InfluxQL * Add bitwise AND, OR and XOR operators to the query language * Write throughput/concurrency improvements * Remove the admin UI * Update to go1.8.1 * Add max concurrent compaction limits * Add TSI support tooling * Track HTTP client requests for /write and /query with /debug/requests * Write and compaction stability * Add new profile endpoint for gathering all debug profiles and queries in single archive * Add nanosecond duration literal support * Optimize top() and bottom() using an incremental aggregator * Maintain the tags of points selected by top() or bottom() when writing the results. * Write CQ stats to the `_internal` database ### Bugfixes * Several statements were missing the DefaultDatabase method * Fix spelling mistake in HTTP section of config – shared-secret * History file should redact passwords before saving to history * Suppress headers in output for influx cli when they are the same * Add chunked/chunk size as setting/options in cli * Do not increment the continuous query statistic if no query is run * Forbid wildcards in binary expressions * Fix fill(linear) when multiple series exist and there are null values * Update liner dependency to handle docker exec * Bind backup and restore port to localhost by default * Kill query not killing query * KILL QUERY should work during all phases of a query * Simplify admin user check. * Significantly improve DROP DATABASE speed * Return an error when an invalid duration literal is parsed * Fix the time range when an exact timestamp is selected * Fix query parser when using addition and subtraction without spaces * Fix a regression when math was used with selectors * Ensure the input for certain functions in the query engine are ordered * Significantly improve shutdown speed for high cardinality databases * Fix racy integration test * Prevent overflowing or underflowing during window computation * Enabled golint for admin, httpd, subscriber, udp, thanks @karlding * Implicitly cast null to false in binary expressions with a Boolean * Restrict fill(none) and fill(linear) to be usable only with aggregate queries * Restrict top() and bottom() selectors to be used with no other functions * top() and bottom() now returns the time for every point * Remove default upper time bound on DELETE queries * Fix LIMIT and OFFSET for certain aggregate queries * Refactor the subquery code and fix outer condition queries * Fix compaction aborted log messages * TSM compaction does not remove .tmp on error * Set the CSV output to an empty string for null values * Compaction exhausting disk resources in InfluxDB * Small edits to the etc/config.sample.toml file * Points beyond retention policy scope are dropped silently * Fix TSM tmp file leaked on disk * Fix large field keys preventing snapshot compactions * URL query parameter credentials take priority over Authentication header * TSI branch has duplicate tag values * Out of memory when using HTTP API * Check file count before attempting a TSI level compaction. * index file fd leak in tsi branch * Fix TSI non-contiguous compaction panic ## v1.2.4 ### Bugfixes * Prefix partial write errors with `partial write:` to generalize identification in other subsystems. ## v1.2.3 ### Bugfixes * Redact passwords before saving them to the history file. * Add the missing DefaultDatabase method to several InfluxQL statements. * Fix segment violation in models.Tags.Get. * Simplify the admin user check. * Fix a regression when math was used with selectors. * Ensure the input for certain functions in the query engine are ordered. * Fix issue where deleted `time` field keys created unparseable points. ## v1.2.2 ### Release Notes ### Configuration Changes #### `[http]` Section * [`max-row-limit`](/influxdb/v1/administration/config#max-row-limit) now defaults to `0`. In versions 1.0 and 1.1, the default setting was `10000`, but due to a bug, the value in use in versions 1.0 and 1.1 was effectively `0`. In versions 1.2.0 through 1.2.1, we fixed that bug, but the fix caused a breaking change for Grafana and Kapacitor users; users who had not set `max-row-limit` to `0` experienced truncated/partial data due to the `10000` row limit. In version 1.2.2, we’ve changed the default `max-row-limit` setting to `0` to match the behavior in versions 1.0 and 1.1. ### Bug fixes * Change the default [`max-row-limit`](/influxdb/v1/administration/config#max-row-limit)setting from `10000` to `0` to prevent the absence of data in Grafana or Kapacitor. ## v1.2.1 ### Release Notes ### Bugfixes * Treat non-reserved measurement names with underscores as normal measurements. * Reduce the expression in a subquery to avoid a panic. * Properly select a tag within a subquery. * Prevent a panic when aggregates are used in an inner query with a raw query. * Points missing after compaction. * Point.UnmarshalBinary() bounds check. * Interface conversion: tsm1.Value is tsm1.IntegerValue, not tsm1.FloatValue. * Map types correctly when using a regex and one of the measurements is empty. * Map types correctly when selecting a field with multiple measurements where one of the measurements is empty. * Include IsRawQuery in the rewritten statement for meta queries. * Fix race in WALEntry.Encode and Values.Deduplicate * Fix panic in collectd when configured to read types DB from directory. * Fix ORDER BY time DESC with ordering series keys. * Fix mapping of types when the measurement uses a regular expression. * Fix LIMIT and OFFSET when they are used in a subquery. * Fix incorrect math when aggregates that emit different times are used. * Fix EvalType when a parenthesis expression is used. * Fix authentication when subqueries are present. * Expand query dimensions from the subquery. * Dividing aggregate functions with different outputs doesn’t panic. * Anchors not working as expected with case-insensitive regular expression. ## v1.2.0 ### Release Notes This release introduces a major new querying capability in the form of sub-queries, and provides several performance improvements, including a 50% or better gain in write performance on larger numbers of cores. The release adds some stability and memory-related improvements, as well as several CLI-related bug fixes. If upgrading from a prior version, please read the configuration changes in the following section before upgrading. ### Configuration Changes The following new configuration options are available, if upgrading to `1.2.0` from prior versions. #### `[[collectd]]` Section * `security-level` which defaults to `"none"`. This field also accepts `"sign"` and `"encrypt"` and enables different levels of transmission security for the collectd plugin. * `auth-file` which defaults to `"/etc/collectd/auth_file"`. Specifies where to locate the authentication file used to authenticate clients when using signed or encrypted mode. ### Deprecations The stress tool `influx_stress` will be removed in a subsequent release. We recommend using [`influx-stress`](https://github.com/influxdata/influx-stress) as a replacement. ### Features * Remove the override of GOMAXPROCS. * Uncomment section headers from the default configuration file. * Improve write performance significantly. * Prune data in meta store for deleted shards. * Update latest dependencies with Godeps. * Introduce syntax for marking a partial response with chunking. * Use X-Forwarded-For IP address in HTTP logger if present. * Add support for secure transmission via collectd. * Switch logging to use structured logging everywhere. * [CLI feature request] USE retention policy for queries. * Add clear command to CLI. * Adding ability to use parameters in queries in the v2 client using the `Parameters` map in the `Query` struct. * Allow add items to array config via ENV. * Support subquery execution in the query language. * Verbose output for SSL connection errors. * Cache snapshotting performance improvements ### Bugfixes * Fix potential race condition in correctness of tsm1\_cache memBytes statistic. * Fix broken error return on meta client’s UpdateUser and DropContinuousQuery methods. * Fix string quoting and significantly improve performance of `influx_inspect export`. * CLI was caching db/rp for insert into statements. * Fix CLI import bug when using self-signed SSL certificates. * Fix cross-platform backup/restore. * Ensures that all user privileges associated with a database are removed when the database is dropped. * Return the time from a percentile call on an integer. * Expand string and Boolean fields when using a wildcard with `sample()`. * Fix chuid argument order in init script. * Reject invalid subscription URLs. * CLI should use spaces for alignment, not tabs. * 0.12.2 InfluxDB CLI client PRECISION returns “Unknown precision…”. * Fix parse key panic when missing tag value. * Rentention Policy should not allow `INF` or `0` as a shard duration. * Return Error instead of panic when decoding point values. * Fix slice out of bounds panic when pruning shard groups. * Drop database will delete /influxdb/data directory. * Ensure Subscriber service can be disabled. * Fix race in storage engine. * InfluxDB should do a partial write on mismatched type errors. ## v1.1.5 ### Bugfixes * Redact passwords before saving them to the history file. * Add the missing DefaultDatabase method to several InfluxQL statements. ## v1.1.4 ### Bugfixes * Backport from 1.2.0: Reduce GC allocations. ## v1.1.3 ### Bugfixes * Remove Tags.shouldCopy, replace with forceCopy on series creation. ## v1.1.2 ### Bugfixes * Fix memory leak when writing new series over HTTP. * Fix series tag iteration segfault. * Fix tag dereferencing panic. ## v1.1.1 ### Features * Update Go version to 1.7.4. ### Bugfixes * Fix string fields w/ trailing slashes. * Quote the empty string as an ident. * Fix incorrect tag value in error message. ### Security [Go 1.7.4](https://golang.org/doc/devel/release.html#go1.7.minor) was released to address two security issues. This release includes these security fixes. ## v1.1.0 ### Release Notes This release is built with GoLang 1.7.3 and provides many performance optimizations, stability changes and a few new query capabilities. If upgrading from a prior version, please read the configuration changes below section before upgrading. ### Deprecations The admin interface is deprecated and will be removed in a subsequent release. The configuration setting to enable the admin UI is now disabled by default, but can be enabled if necessary. We recommend using [Chronograf](https://github.com/influxdata/chronograf) or [Grafana](https://github.com/grafana/grafana) as a replacement. ### Configuration Changes The following configuration changes may need to changed before upgrading to `1.1.0` from prior versions. #### `[admin]` Section * `enabled` now default to false. If you are currently using the admin interaface, you will need to change this value to `true` to re-enable it. The admin interface is currently deprecated and will be removed in a subsequent release. #### `[data]` Section * `max-values-per-tag` was added with a default of 100,000, but can be disabled by setting it to `0`. Existing measurements with tags that exceed this limit will continue to load, but writes that would cause the tags cardinality to increase will be dropped and a `partial write` error will be returned to the caller. This limit can be used to prevent high cardinality tag values from being written to a measurement. * `cache-max-memory-size` has been increased to from `524288000` to `1048576000`. This setting is the maximum amount of RAM, in bytes, a shard cache can use before it rejects writes with an error. Setting this value to `0` disables the limit. * `cache-snapshot-write-cold-duration` has been decreased from `1h` to `10m`. This setting determines how long values will stay in the shard cache while the shard is cold for writes. * `compact-full-write-cold-duration` has been decreased from `24h` to `4h`. The shorter duration allows cold shards to be compacted to an optimal state more quickly. ### Features The query language has been extended with a few new features: * Support regular expressions on fields keys in select clause. * New `linear` fill option. * New `cumulative_sum` function. * Support `ON` for `SHOW` commands. All Changes: * Filter out series within shards that do not have data for that series. * Rewrite regular expressions of the form host = /^server-a$/ to host = ‘server-a’, to take advantage of the tsdb index. * Improve compaction planning performance by caching tsm file stats. * Align binary math expression streams by time. * Reduce map allocations when computing the TagSet of a measurement. * Make input plugin services open/close idempotent. * Speed up shutdown by closing shards concurrently. * Add sample function to query language. * Add `fill(linear)` to query language. * Implement cumulative\_sum() function. * Update defaults in config for latest best practices. * UDP Client: Split large points. * Add stats for active compactions, compaction errors. * More man pages for the other tools we package and compress man pages fully. * Add max-values-per-tag to limit high tag cardinality data. * Update jwt-go dependency to version 3. * Support enable HTTP service over unix domain socket. * Add additional statistics to query executor. * Feature request: `influx inspect -export` should dump WAL files. * Implement text/csv content encoding for the response writer. * Support tools for running async queries. * Support ON and use default database for SHOW commands. * Correctly read in input from a non-interactive stream for the CLI. * Support `INFLUX_USERNAME` and `INFLUX_PASSWORD` for setting username/password in the CLI. * Optimize first/last when no group by interval is present. * Make regular expressions work on field and dimension keys in SELECT clause. * Change default time boundaries for raw queries. * Support mixed duration units. ### Bugfixes * Avoid deadlock when `max-row-limit` is hit. * Fix incorrect grouping when multiple aggregates are used with sparse data. * Fix output duration units for SHOW QUERIES. * Truncate the version string when linking to the documentation. * influx\_inspect: export does not escape field keys. * Fix issue where point would be written to wrong shard. * Fix retention policy inconsistencies. * Remove accidentally added string support for the stddev call. * Remove /data/process\_continuous\_queries endpoint. * Enable https subscriptions to work with custom CA certificates. * Reduce query planning allocations. * Shard stats include WAL path tag so disk bytes make more sense. * Panic with unread show series iterators during drop database. * Use consistent column output from the CLI for column formatted responses. * Correctly use password-type field in Admin UI. * Duplicate parsing bug in ALTER RETENTION POLICY. * Fix database locked up when deleting shards. * Fix mmap dereferencing. * Fix base64 encoding issue with /debug/vars stats. * Drop measurement causes cache max memory exceeded error. * Decrement number of measurements only once when deleting the last series from a measurement. * Delete statement returns an error when retention policy or database is specified. * Fix the dollar sign so it properly handles reserved keywords. * Exceeding max retention policy duration gives incorrect error message. * Drop time when used as a tag or field key. ## v1.0.2 ### Bugfixes * Fix RLE integer decoding producing negative numbers. * Avoid stat syscall when planning compactions. * Subscription data loss under high write load. * Do not automatically reset the shard duration when using ALTER RETENTION POLICY. * Ensure correct shard groups created when retention policy has been altered. ## v1.0.1 ### Bugfixes * Prevent users from manually using system queries since incorrect use would result in a panic. * Ensure fieldsCreated stat available in shard measurement. * Report cmdline and memstats in /debug/vars. * Fixing typo within example configuration file. * Implement time math for lazy time literals. * Fix database locked up when deleting shards. * Skip past points at the same time in derivative call within a merged series. * Read an invalid JSON response as an error in the Influx client. ## v1.0.0 ### Release Notes Initial release of InfluxDB. ### Breaking changes * `max-series-per-database` was added with a default of 1M but can be disabled by setting it to `0`. Existing databases with series that exceed this limit will continue to load but writes that would create new series will fail. * Config option `[cluster]` has been replaced with `[coordinator]`. * Support for config options `[collectd]` and `[opentsdb]` has been removed; use `[[collectd]]` and `[[opentsdb]]` instead. * Config option `data-logging-enabled` within the `[data]` section, has been renamed to `trace-logging-enabled`, and defaults to `false`. * The keywords `IF`, `EXISTS`, and `NOT` where removed for this release. This means you no longer need to specify `IF NOT EXISTS` for `DROP DATABASE` or `IF EXISTS` for `CREATE DATABASE`. If these are specified, a query parse error is returned. * The Shard `writePointsFail` stat has been renamed to `writePointsErr` for consistency with other stats. With this release the systemd configuration files for InfluxDB will use the system configured default for logging and will no longer write files to `/var/log/influxdb` by default. On most systems, the logs will be directed to the systemd journal and can be accessed by `journalctl -u influxdb.service`. Consult the systemd journald documentation for configuring journald. ### Features * Add mode function. * Support negative timestamps for the query engine. * Write path stats. * Add MaxSeriesPerDatabase config setting. * Remove IF EXISTS/IF NOT EXISTS from influxql language. * Update go package library dependencies. * Add tsm file export to influx\_inspect tool. * Create man pages for commands. * Return 403 Forbidden when authentication succeeds but authorization fails. * Added favicon. * Run continuous query for multiple buckets rather than one per bucket. * Log the CQ execution time when continuous query logging is enabled. * Trim BOM from Windows Notepad-saved config files. * Update help and remove unused config options from the configuration file. * Add NodeID to execution options. * Make httpd logger closer to Common (& combined) Log Format. * Allow any variant of the help option to trigger the help. * Reduce allocations during query parsing. * Optimize timestamp run-length decoding. * Adds monitoring statistic for on-disk shard size. * Add HTTP(s) based subscriptions. * Add new HTTP statistics to monitoring. * Speed up drop database. * Add Holt-Winter forecasting function. * Add support for JWT token authentication. * Add ability to create snapshots of shards. * Parallelize iterators. * Teach the http service how to enforce connection limits. * Support cast syntax for selecting a specific type. * Refactor monitor service to avoid expvar and write monitor statistics on a truncated time interval. * Dynamically update the documentation link in the admin UI. * Support wildcards in aggregate functions. * Support specifying a retention policy for the graphite service. * Add extra trace logging to tsm engine. * Add stats and diagnostics to the TSM engine. * Support regex selection in SHOW TAG VALUES for the key. * Modify the default retention policy name and make it configurable. * Update SHOW FIELD KEYS to return the field type with the field key. * Support bound parameters in the parser. * Add https-private-key option to httpd config. * Support loading a folder for collectd typesdb files. ### Bugfixes * Optimize queries that compare a tag value to an empty string. * Allow blank lines in the line protocol input. * Runtime: goroutine stack exceeds 1000000000-byte limit. * Fix alter retention policy when all options are used. * Concurrent series limit. * Ensure gzip writer is closed in influx\_inspect export. * Fix CREATE DATABASE when dealing with default values. * Fix UDP pointsRx being incremented twice. * Tombstone memory improvements. * Hardcode auto generated RP names to autogen. * Ensure IDs can’t clash when managing Continuous Queries. * Continuous full compactions. * Remove limiter from walkShards. * Copy tags in influx\_stress to avoid a concurrent write panic on a map. * Do not run continuous queries that have no time span. * Move the CQ interval by the group by offset. * Fix panic parsing empty key. * Update connection settings when changing hosts in CLI. * Always use the demo config when outputting a new config. * Minor improvements to init script. Removes sysvinit-utils as package dependency. * Fix compaction planning with large TSM files. * Duplicate data for the same timestamp. * Fix panic: truncate the slice when merging the caches. * Fix regex binary encoding for a measurement. * Fix fill(previous) when used with math operators. * Rename dumptsmdev to dumptsm in influx\_inspect. * Remove a double lock in the tsm1 index writer. * Remove FieldCodec from TSDB package. * Allow a non-admin to call “use” for the influx CLI. * Set the condition cursor instead of aux iterator when creating a nil condition cursor. * Update `stress/v2` to work with clusters, ssl, and username/password auth. Code cleanup. * Modify the max nanosecond time to be one nanosecond less. * Include sysvinit-tools as an rpm dependency. * Add port to all graphite log output to help with debugging multiple endpoints. * Fix panic: runtime error: index out of range. * Remove systemd output redirection. * Database unresponsive after DROP MEASUREMENT. * Address Out of Memory Error when Dropping Measurement. * Fix the point validation parser to identify and sort tags correctly. * Prevent panic in concurrent auth cache write. * Set X-Influxdb-Version header on every request (even 404 requests). * Prevent panic if there are no values. * Time sorting broken with overwritten points. * queries with strings that look like dates end up with date types, not string types. * Concurrent map read write panic. * Drop writes from before the retention policy time window. * Fix SELECT statement required privileges. * Filter out sources that do not match the shard database/retention policy. * Truncate the shard group end time if it exceeds MaxNanoTime. * Batch SELECT INTO / CQ writes. * Fix compaction planning re-compacting large TSM files. * Ensure client sends correct precision when inserting points. * Accept points with trailing whitespace. * Fix panic in SHOW FIELD KEYS. * Disable limit optimization when using an aggregate. * Fix panic: interface conversion: tsm1.Value is \*tsm1.StringValue, not \*tsm1.FloatValue. * Data race when dropping a database immediately after writing to it. * Make sure admin exists before authenticating query. * Print the query executor’s stack trace on a panic to the log. * Fix read tombstones: EOF. * Query-log-enabled in config not ignored anymore. * Ensure clients requesting gzip encoded bodies don’t receive empty body. * Optimize shard loading. * Queries slow down hundreds times after overwriting points. * SHOW TAG VALUES accepts != and !\~ in WHERE clause. * Remove old cluster code. * Ensure that future points considered in SHOW queries. * Fix full compactions conflicting with level compactions. * Overwriting points on large series can cause memory spikes during compactions. * Fix parseFill to check for fill ident before attempting to parse an expression. * Max index entries exceeded. * Address slow startup time. * Fix measurement field panic in tsm1 engine. * Queries against files that have just been compacted need to point to new files. * Check that retention policies exist before creating CQ. | tag / field | description | | --- | --- | | tag / field | description | | db | name of database | | cq | name of continuous query | | durationNS | query execution time in nanoseconds | | startTime | lower bound of time range | | endTime | upper bound of time range | | pointsWrittenOK | number of points written to the target measurement | --- # Third party software Source: https://docs.influxdata.com/influxdb/v1/about_the_project/third-party/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxData products contain third party software, which means the copyrighted, patented, or otherwise legally protected software of third parties that is incorporated in InfluxData products. Third party suppliers make no representation nor warranty with respect to such third party software or any portion thereof. Third party suppliers assume no liability for any claim that might arise with respect to such third party software, nor for a customer’s use of or inability to use the third party software. The [list of third party software components, including references to associated licenses and other materials](https://github.com/influxdata/influxdb/blob/1.8/DEPENDENCIES.md), is maintained on a version by version basis. --- # Additional InfluxDB resources Source: https://docs.influxdata.com/influxdb/v1/additional_resources/additional_resources/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. View these additional resources to learn more about InfluxDB. ## InfluxData Blog Check out the [InfluxData Blog](https://www.influxdata.com/blog/) for announcements, updates, and weekly [tech tips](https://www.influxdata.com/category/tech-tips/). ## Technical Papers InfluxData’s [Technical Papers](https://www.influxdata.com/_resources/techpapers-new/)series offer in-depth analysis on performance, time series, and benchmarking InfluxDB vs. other popular databases. ## Meetup Videos Check out our growing [Meetup videos](https://www.influxdata.com/_resources/videosnew//)collection for introductory content, how-tos, and more. ## Virtual Training Videos Watch the [virtual training videos](https://www.influxdata.com/_resources/videosnew/) from our weekly training webinar. ## Virtual Training Schedule Check out our [virtual training schedule](https://www.influxdata.com/virtual-training-courses/)to register for future webinars. ## Events Find out what’s happening at InfluxDB and sign up for upcoming [events](https://www.influxdata.com/events/). --- # Administer InfluxDB Source: https://docs.influxdata.com/influxdb/v1/administration/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The administration documentation contains all the information needed to administer a working InfluxDB installation. ## [Identify InfluxDB OSS v1 version](/influxdb/v1/administration/identify-version/) Learn how to identify your InfluxDB OSS v1 version using command-line tools, HTTP endpoints, and other methods. ## [Authentication and authorization in InfluxDB](/influxdb/v1/administration/authentication_and_authorization/) Set up and manage authentication and authorization in InfluxDB OSS. ## [Back up and restore data](/influxdb/v1/administration/backup_and_restore/) To prevent unexpected data loss, back up and restore InfluxDB OSS instances. ## [Compact a series file offline](/influxdb/v1/administration/compact-series-file/) Use the `influx_inspect buildtsi -compact-series-file` command to compact your series file and reduce its size on disk. ## [Configure InfluxDB OSS](/influxdb/v1/administration/config/) Learn about InfluxDB OSS configuration settings and environment variables. ## [Enable HTTPS with InfluxDB](/influxdb/v1/administration/https_setup/) Enable HTTPS and Transport Security Layer (TLS) secure communication between clients and your InfluxDB servers. ## [InfluxDB ports](/influxdb/v1/administration/ports/) Enabled and disabled ports in InfluxDB. ## [Log and trace with InfluxDB](/influxdb/v1/administration/logs/) Structured logging, access logging, tracing, and logging locations in InfluxDB. ## [Manage InfluxDB security](/influxdb/v1/administration/security/) Protect the data in your InfluxDB OSS instance. ## [Manage subscriptions in InfluxDB](/influxdb/v1/administration/subscription-management/) Manage subscriptions, which copy all written data to a local or remote endpoint, in InfluxDB OSS. ## [Monitor InfluxDB](/influxdb/v1/administration/monitor/) Monitor the health and performance of your InfluxDB server, or use InfluxDB and the TICK stack to monitor other systems. ## [Rebuild the TSI index](/influxdb/v1/administration/rebuild-tsi-index/) Use the `influxd_inspect buildtsi` command to rebuild your InfluxDB TSI index. ## [Stability and compatibility](/influxdb/v1/administration/stability_and_compatibility/) API and storage engine compatibility and stability in InfluxDB OSS. ## [Upgrade to InfluxDB 1.11.x](/influxdb/v1/administration/upgrading/) Upgrade to the latest version of InfluxDB. ## Downgrade To revert to a prior version, complete the same steps as when [Upgrading to InfluxDB 1.11.x](/influxdb/v1/administration/upgrading/), replacing 1.11.x with the version you want to downgrade to. After downloading the release, migrating your configuration settings, and enabling TSI or TSM, make sure to [rebuild your index](/influxdb/v1/administration/rebuild-tsi-index). > > > **Note:** Some versions of InfluxDB may have breaking changes that impact your ability to upgrade and downgrade. For example, you cannot downgrade from InfluxDB 1.3 or later to an earlier version. Please review the applicable version of release notes to check for compatibility issues between releases. > > --- # Authentication and authorization in InfluxDB Source: https://docs.influxdata.com/influxdb/v1/administration/authentication_and_authorization/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. This document covers setting up and managing authentication and authorization in InfluxDB. * [Authentication](#authentication) * [Set up Authentication](#set-up-authentication) * [Authenticate Requests](#authenticate-requests) * [Authorization](#authorization) * [User Types and Privileges](#user-types-and-privileges) * [User Management Commands](#user-management-commands) * [HTTP Errors](#authentication-and-authorization-http-errors) > [!Note] > Authentication and authorization should not be relied upon to prevent access and protect data from malicious actors. > If additional security or compliance features are desired, InfluxDB should be run behind a third-party service. If InfluxDB is > being deployed on a publicly accessible endpoint, we strongly recommend authentication be enabled. Otherwise the data will be > publicly available to any unauthenticated user. ## Authentication The InfluxDB API and the [command line interface](/influxdb/v1/tools/shell/) (CLI), which connects to the database using the API, include simple, built-in authentication based on user credentials. When you enable authentication, InfluxDB only executes HTTP requests that are sent with valid credentials. > [!Note] > Authentication only occurs at the HTTP request scope. > Plugins do not currently have the ability to authenticate requests and service > endpoints (for example, Graphite, collectd, etc.) are not authenticated. ### Set up authentication 1. **Create at least one [admin user](#admin-users)**. See the [authorization section](#authorization) for how to create an admin user. If you enable authentication and have no users, InfluxDB will **not** enforce authentication and will only accept the [query](#user-management-commands) that creates a new admin user. InfluxDB will enforce authentication once there is an admin user. 2. **Enable authentication in your configuration file**by setting the `auth-enabled` option to `true` in the `[http]` section: ``` [http] enabled = true bind-address = ":8086" auth-enabled = true # Set to true log-enabled = true write-tracing = false pprof-enabled = true pprof-auth-enabled = true debug-pprof-enabled = false ping-auth-enabled = true https-enabled = true https-certificate = "/etc/ssl/influxdb.pem" ``` > [!Note] > If `pprof-enabled` is set to `true`, set `pprof-auth-enabled` and `ping-auth-enabled`to `true` to require authentication on profiling and ping endpoints. 3. **Restart InfluxDB**. Once restarted, InfluxDB checks user credentials on every request and only processes requests that have valid credentials for an existing user. ### Authenticate requests #### Authenticate with the InfluxDB API There are two options for authenticating with the [InfluxDB API](/influxdb/v1/api/). If you authenticate with both Basic Authentication **and** the URL query parameters, the user credentials specified in the query parameters take precedence. The queries in the following examples assume that the user is an [admin user](#admin-users). See the section on [authorization](#authorization) for the different user types, their privileges, and more on user management. > > > **Note:** InfluxDB redacts passwords when you enable authentication. > > ##### Authenticate with Basic Authentication ```bash curl -G http://localhost:8086/query \ -u todd:influxdb4ever \ --data-urlencode "q=SHOW DATABASES" ``` ##### Authenticate with query parameters in the URL or request body Set `u` as the username and `p` as the password. ###### Credentials as query parameters ```bash curl -G "http://localhost:8086/query?u=todd&p=influxdb4ever" \ --data-urlencode "q=SHOW DATABASES" ``` ###### Credentials in the request body ```bash curl -G http://localhost:8086/query \ --data-urlencode "u=todd" \ --data-urlencode "p=influxdb4ever" \ --data-urlencode "q=SHOW DATABASES" ``` #### Authenticate with the CLI There are three options for authenticating with the [CLI](/influxdb/v1/tools/shell/). ##### Authenticate with environment variables Use the `INFLUX_USERNAME` and `INFLUX_PASSWORD` environment variables to provide authentication credentials to the `influx` CLI. ```bash export INFLUX_USERNAME=todd export INFLUX_PASSWORD=influxdb4ever echo $INFLUX_USERNAME $INFLUX_PASSWORD todd influxdb4ever influx Connected to http://localhost:8086 version 1.13.0 InfluxDB shell 1.13.0 ``` ##### Authenticate with CLI flags Use the `-username` and `-password` flags to provide authentication credentials to the `influx` CLI. ```bash influx -username todd -password influxdb4ever Connected to http://localhost:8086 version 1.13.0 InfluxDB shell 1.13.0 ``` ##### Authenticate with credentials in the influx shell Start the `influx` shell and run the `auth` command. Enter your username and password when prompted. ```bash > influx Connected to http://localhost:8086 version 1.13.0 InfluxDB shell 1.13.0 > auth username: todd password: > ``` #### Authenticate using JWT tokens For a more secure alternative to using passwords, include JWT tokens with requests to the InfluxDB API. This is currently only possible through the [InfluxDB HTTP API](/influxdb/v1/api/). 1. [Add a shared secret in your InfluxDB configuration file](#add-a-shared-secret-in-your-influxdb-configuration-file) 2. [Generate your JWT token](#generate-your-jwt-token) 3. [Include the token in HTTP requests](#include-the-token-in-http-requests) ##### Add a shared secret in your InfluxDB configuration file InfluxDB uses the shared secret to encode the JWT signature. By default, `shared-secret` is set to an empty string, in which case no JWT authentication takes place. Add a custom shared secret in your [InfluxDB configuration file](/influxdb/v1/administration/config/#shared-secret). The longer the secret string, the more secure it is: ```toml [http] shared-secret = "my super secret pass phrase" ``` Alternatively, to avoid keeping your secret phrase as plain text in your InfluxDB configuration file, set the value with the `INFLUXDB_HTTP_SHARED_SECRET` environment variable. ##### Generate your JWT token Use an authentication service to generate a secure token using your InfluxDB username, an expiration time, and your shared secret. There are online tools, such as [https://jwt.io/](https://jwt.io/), that will do this for you. The payload (or claims) of the token must be in the following format: ```json { "username": "myUserName", "exp": 1516239022 } ``` * **username** - The name of your InfluxDB user. * **exp** - The expiration time of the token in UNIX epoch time. For increased security, keep token expiration periods short. For testing, you can manually generate UNIX timestamps using [https://www.unixtimestamp.com/index.php](https://www.unixtimestamp.com/index.php). Encode the payload using your shared secret. You can do this with either a JWT library in your own authentication server or by hand at [https://jwt.io/](https://jwt.io/). The generated token follows this format: `
..` ##### Include the token in HTTP requests Include your generated token as part of the `Authorization` header in HTTP requests. Use the `Bearer` authorization scheme: ``` Authorization: Bearer ``` > [!Note] > Only unexpired tokens will successfully authenticate. Be sure your token has not expired. ###### Example query request with JWT authentication ```bash curl -G "http://localhost:8086/query?db=demodb" \ --data-urlencode "q=SHOW DATABASES" \ --header "Authorization: Bearer
.." ``` ## Authenticate Telegraf requests to InfluxDB Authenticating [Telegraf](/telegraf/v1/) requests to an InfluxDB instance with authentication enabled requires some additional steps. In the Telegraf configuration file (`/etc/telegraf/telegraf.conf`), uncomment and edit the `username` and `password` settings. ``` ############################################################################### # OUTPUT PLUGINS ############################################################################### # ... [[outputs.influxdb]] # ... username = "example-username" # Provide your username password = "example-password" # Provide your password # ... ``` Restart Telegraf and you’re all set! ## Authorization Authorization is only enforced once you’ve [enabled authentication](#set-up-authentication). By default, authentication is disabled, all credentials are silently ignored, and all users have all privileges. ### User types and privileges #### Admin users Admin users have `READ` and `WRITE` access to all databases and full access to the following administrative queries: ##### Database management * `CREATE DATABASE` * `DROP DATABASE` * `DROP SERIES` * `DROP MEASUREMENT` * `CREATE RETENTION POLICY` * `ALTER RETENTION POLICY` * `DROP RETENTION POLICY` * `CREATE CONTINUOUS QUERY` * `DROP CONTINUOUS QUERY` For more information about these commands, see [Database management](/influxdb/v1/query_language/manage-database/) and[Continuous queries](/influxdb/v1/query_language/continuous_queries/). ##### User management * Admin user management * [`CREATE USER`](#user-management-commands) * [`GRANT ALL PRIVILEGES`](#grant-administrative-privileges-to-an-existing-user) * [`REVOKE ALL PRIVILEGES`](#revoke-administrative-privileges-from-an-admin-user) * [`SHOW USERS`](#show-all-existing-users-and-their-admin-status) * Non-admin user management: * [`CREATE USER`](#user-management-commands) * [`GRANT [READ,WRITE,ALL]`](#grant-read-write-or-all-database-privileges-to-an-existing-user) * [`REVOKE [READ,WRITE,ALL]`](#revoke-read-write-or-all-database-privileges-from-an-existing-user) * General user management: * [`SET PASSWORD`](#reset-a-users-password) * [`DROP USER`](#drop-a-user) See [below](#user-management-commands) for a complete discussion of the user management commands. #### Non-admin users Non-admin users can have one of the following three privileges per database: * `READ` * `WRITE` * `ALL` (both `READ` and `WRITE` access) `READ`, `WRITE`, and `ALL` privileges are controlled per user per database. A new non-admin user has no access to any database until they are specifically [granted privileges to a database](#grant-read-write-or-all-database-privileges-to-an-existing-user) by an admin user. Non-admin users can [`SHOW`](/influxdb/v1/query_language/explore-schema/#show-databases) the databases on which they have `READ` and/or `WRITE` permissions. ### User management commands #### Admin user management When you enable HTTP authentication, InfluxDB requires you to create at least one admin user before you can interact with the system. ```sql CREATE USER admin WITH PASSWORD '' WITH ALL PRIVILEGES ``` ##### Create another admin user ```sql CREATE USER WITH PASSWORD '' WITH ALL PRIVILEGES ``` > [!Note] > Repeating the exact `CREATE USER` statement is idempotent. > If any values change the database will return a duplicate user error. > > ``` > > CREATE USER todd WITH PASSWORD '123456' WITH ALL PRIVILEGES > > CREATE USER todd WITH PASSWORD '123456' WITH ALL PRIVILEGES > > CREATE USER todd WITH PASSWORD '123' WITH ALL PRIVILEGES > ERR: user already exists > > CREATE USER todd WITH PASSWORD '123456' > ERR: user already exists > > CREATE USER todd WITH PASSWORD '123456' WITH ALL PRIVILEGES > > > ``` ##### `GRANT` administrative privileges to an existing user ```sql GRANT ALL PRIVILEGES TO ``` ##### `REVOKE` administrative privileges from an admin user ```sql REVOKE ALL PRIVILEGES FROM ``` ##### `SHOW` all existing users and their admin status ```sql SHOW USERS ``` ###### CLI Example ```sql > SHOW USERS user admin todd false paul true hermione false dobby false ``` #### Non-admin user management ##### `CREATE` a new non-admin user ```sql CREATE USER WITH PASSWORD '' ``` ###### CLI example ```js CREATE USER todd WITH PASSWORD 'influxdb41yf3' CREATE USER alice WITH PASSWORD 'wonder\'land' CREATE USER "rachel_smith" WITH PASSWORD 'asdf1234!' CREATE USER "monitoring-robot" WITH PASSWORD 'XXXXX' CREATE USER "$savyadmin" WITH PASSWORD 'm3tr1cL0v3r' ``` > [!Important] > ##### Important notes about providing user credentials > > * The user value must be wrapped in double quotes if it starts with a digit, is an InfluxQL keyword, contains a hyphen and or includes any special characters, for example: `!@#$%^&*()-` > * The password [string](/influxdb/v1/query_language/spec/#strings) must be wrapped in single quotes. > Do not include the single quotes when authenticating requests. > We recommend avoiding the single quote (`'`) and backslash (`\`) characters in passwords. > For passwords that include these characters, escape the special character with a backslash (for example, (`\'`) when creating the password and when submitting authentication requests. > * Repeating the exact `CREATE USER` statement is idempotent. If any values change the database will return a duplicate user error. See GitHub Issue [#6890](https://github.com/influxdata/influxdb/pull/6890) for details. > > ###### CLI example > > ``` > > CREATE USER "todd" WITH PASSWORD '123456' > > CREATE USER "todd" WITH PASSWORD '123456' > > CREATE USER "todd" WITH PASSWORD '123' > ERR: user already exists > > CREATE USER "todd" WITH PASSWORD '123456' > > CREATE USER "todd" WITH PASSWORD '123456' WITH ALL PRIVILEGES > ERR: user already exists > > CREATE USER "todd" WITH PASSWORD '123456' > > > ``` ##### `GRANT` `READ`, `WRITE` or `ALL` database privileges to an existing user ```sql GRANT [READ,WRITE,ALL] ON TO ``` CLI examples: `GRANT` `READ` access to `todd` on the `NOAA_water_database` database: ```sql GRANT READ ON "NOAA_water_database" TO "todd" ``` `GRANT` `ALL` access to `todd` on the `NOAA_water_database` database: ```sql GRANT ALL ON "NOAA_water_database" TO "todd" ``` ##### `REVOKE` `READ`, `WRITE`, or `ALL` database privileges from an existing user ``` REVOKE [READ,WRITE,ALL] ON FROM ``` CLI examples: `REVOKE` `ALL` privileges from `todd` on the `NOAA_water_database` database: ```sql REVOKE ALL ON "NOAA_water_database" FROM "todd" ``` `REVOKE` `WRITE` privileges from `todd` on the `NOAA_water_database` database: ```sql REVOKE WRITE ON "NOAA_water_database" FROM "todd" ``` > > > **Note:** If a user with `ALL` privileges has `WRITE` privileges revoked, they are left with `READ` privileges, and vice versa. > > ##### `SHOW` a user’s database privileges ```sql SHOW GRANTS FOR ``` CLI example: ```sql > SHOW GRANTS FOR "todd" database privilege NOAA_water_database WRITE another_database_name READ yet_another_database_name ALL PRIVILEGES one_more_database_name NO PRIVILEGES ``` #### General admin and non-admin user management ##### Reset a user’s password ```sql SET PASSWORD FOR = '' ``` CLI example: ```sql SET PASSWORD FOR "todd" = 'influxdb4ever' ``` > [!Note] > The password [string](/influxdb/v1/query_language/spec/#strings) must be wrapped in single quotes. > Do not include the single quotes when authenticating requests. > > We recommend avoiding the single quote (`'`) and backslash (`\`) characters in passwords > For passwords that include these characters, escape the special character with a backslash (for example, (`\'`) when creating the password and when submitting authentication requests. ##### `DROP` a user ```sql DROP USER ``` CLI example: ```sql DROP USER "todd" ``` ## Authentication and authorization HTTP errors Requests with no authentication credentials or incorrect credentials yield the `HTTP 401 Unauthorized` response. Requests by unauthorized users yield the `HTTP 403 Forbidden` response. --- # Back up and restore data Source: https://docs.influxdata.com/influxdb/v1/administration/backup_and_restore/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the InfluxDB OSS 1.13 `backup`, `restore`, `export`, and `import` utilities to prevent unexpected data loss and preserve the ability to restore data if it ever is lost. You can use these tools in your back up and restore procedures to: * Provide disaster recovery due to unexpected events * Migrate data to new environments or servers * Restore instances to a consistent state * Export and import data for debugging Depending on the volume of data to be protected and your application requirements, InfluxDB OSS 1.13 offers two methods, described below, for managing backups and restoring data: * [Backup and restore utilities](#backup-and-restore-utilities): Use for most applications * [Exporting and importing data](#exporting-and-importing-data): Use for very large datasets and to debug data > [!Note] > #### Back up and restore between InfluxDB Enterprise and OSS > > Use the `backup` and `restore` utilities in[InfluxDB Enterprise](/enterprise_influxdb/v1/administration/backup-and-restore/) and[InfluxDB OSS (version 1.5 and later)](#backup-and-restore-utilities) to: > > * Restore InfluxDB Enterprise backup files to InfluxDB OSS instances. > * Back up InfluxDB OSS data that can be restored in InfluxDB Enterprise clusters. ## Backup and restore utilities Use InfluxDB OSS 1.13 `backup` and `restore` utilities to: * Back up and restore multiple databases at a time. * Back up specific time ranges. * Create backup files compatible with InfluxDB Enterprise. Use the `backup` and `restore` utilities to back up and restore data for `influxd`instances of InfluxDB OSS version 1.4 and earlier, version 1.5 and later, and [InfluxDB Enterprise](/enterprise_influxdb/v1/). * [Backup formats](#backup-formats) * [Specify your backup format](#specify-your-backup-format) * [Determine your backup’s format](#determine-your-backups-format) * [Backup and restore requirements](#backup-and-restore-requirements) * [Important notes](#important-notes) * [Time-based backups](#time-based-backups) * [Back up data](#back-up-data) * [Backup examples (portable format)](?t=Portable+format#backup-examples) * [Backup examples (legacy format)](?t=Legacy+format#legacy-backup-examples) * [Restore data](#restore-data) * [Restore examples (portable format)](#restore-examples) * [Online restore examples (legacy format)](?t=Legacy+format#online-legacy-restore) * [Offline restore examples (legacy format)](?t=Legacy+format#offline-legacy-restore) * [Restore data to an existing database](#restore-data-to-an-existing-database) * [Configure backup and restore services](#configure-backup-and-restore-services) ## Backup formats The InfluxDB `backup` utility outputs data backups in one of two formats: **legacy** or **portable**. Each format provides different functionality and support for other versions of InfluxDB. The major difference between the formats is that the legacy format can only be used with InfluxDB OSS 1.x. The portable format is “portable” between InfluxDB 1.5–1.13 and [InfluxDB Enterprise](/enterprise_influxdb/v1/). > [!Note] > #### Use the portable format for InfluxDB 1.5 and later > > Use the portable format unless you need to be able to restore > the backup to InfluxDB 1.4 or earlier. | Backup functionality |Legacy format|Portable format| |------------------------------------|-------------|---------------| | Back up single databases | ✓ | ✓ | | Back up multiple databases | | ✓ | | Back up single retention polices | ✓ | ✓ | |Back up multiple retention policies | ✓ | ✓ | | Back up specific time ranges | ✓ | ✓ | | Back up remote databases | ✓ | ✓ | |Restore to **InfluxDB OSS 1.0–1.4** | ✓ | | |Restore to **InfluxDB OSS 1.5–1.13**| ✓ | ✓ | | Restore to **InfluxDB Enterprise** | | ✓ | ### Specify your backup format By default, the `backup` and `restore` utilities create and restore backups in the legacy format. To create or restore from a backup or in the portable format, include the`-portable` flag with your backup command. ```sh # Create a backup in the portable format influxd backup -portable /path/to/backup-destination # Restore from a portable backup influxd restore -portable /path/to/backup-destination ``` ### Determine your backup’s format Use the directory structure of the backup directory to determine the format of the backup. #### Portable format directory structure * backup-directory/ * 20060102T150405Z.meta(InfluxDB metadata) * 20060102T150405Z.s00.tar.gz(InfluxDB time series data *(shard)*) * 20060102T150405Z.s01.tar.gz(InfluxDB time series data *(shard)*) * 20060102T150405Z.manifest(Backup manifest) #### Legacy format directory structure * backup-directory/ * meta.00(InfluxDB metadata) * \.\.00000.00(InfluxDB time series data) * \.\.00001.00(InfluxDB time series data) ## Backup and restore requirements * The InfluxDB service (`influxd`) must be running *except when performing an[offline legacy restore](?t=Legacy+format#offline-legacy-restore)*. * Both the source and target InfluxDB instances must be the same InfluxDB version or differ by only a minor version. For example, you can back up data from InfluxDB 1.7 and restore it to an InfluxDB 1.8 instance. The only exception is restoring data from InfluxDB 1.8 into 1.11. ## Important notes ### Time-based backups When executing a backup with `-start` or `-end` flags, InfluxDB backs up shards, not individual points. Each shard is annotated with starting and ending timestamps for the time interval included in the shard. If a shard contains any points in the specified time range, the entire shard is included in the backup. Since most shards are highly compacted, extracting shards to inspect each point would be highly inefficient and add a significant burden to the underlying system. When restoring data from a time-based backup, you are likely to see data outside of the backed up time range. ## Back up data Use the [`influxd backup` utility](/influxdb/v1/tools/influxd/backup/) to create a backup of time series data and metadata stored in InfluxDB. > [!Note] > `backup` ignores WAL files and in-memory cache data. Provide the following for each [backup format](#backup-formats): [Portable format](#portable-format)[Legacy format](#legacy-format) * `-portable`: (Required) Specifies the portable backup format * `-host`: InfluxDB bind address *(Only required if creating a backup from a remote InfluxDB host)* * `-db`: Database name *(If no database name is specified, the command backs up all databases)* * `-rp`: Retention policy name*(If no retention policy is specified, the command backs up all retention policies)* * `-shard`: Shard ID *(If no shard ID is specified, the command backs up all shards. Requires a retention policy.)* * `-start`: Start time *(If no start time is specified, the command backs up data from all time.)* * `-stop`: Stop time *(If no stop time is specified, the command backs up data to now.)* * (Required) **Backup destination directory path** ### Backup examples * [Back up all databases](#back-up-all-databases) * [Back up all data from a remote InfluxDB instance](#back-up-all-data-from-a-remote-influxdb-instance) * [Back up a specific time range](#back-up-a-specific-time-range) * [Back up data from a specific time to now](#back-up-data-from-a-specific-time-to-now) * [Back up a specific database](#back-up-a-specific-database) * [Back up a specific retention policy](#back-up-a-specific-retention-policy) * [Back up a specific shard](#back-up-a-specific-shard) #### Back up all databases ```sh influxd backup -portable /path/to/backup-directory ``` #### Back up all data from a remote InfluxDB instance ```sh influxd backup -portable \ -host 203.0.113.0:8088 \ /path/to/backup-directory ``` #### Back up a specific time range ```sh influxd backup -portable \ -start 2022-01-01T00:00:00Z \ -stop 2022-02-01T00:00:00Z \ /path/to/backup-directory ``` *For important information about how time-based backups work, see [Time-based backups](#time-based-backups).* #### Back up data from a specific time to now ```sh influxd backup -portable \ -start 2022-01-01T00:00:00Z \ /path/to/backup-directory ``` *For important information about how time-based backups work, see [Time-based backups](#time-based-backups).* #### Back up a specific database ```sh influxd backup -portable \ -db example-db \ /path/to/backup-directory ``` #### Back up a specific retention policy ```sh influxd backup -portable \ -db example-db \ -rp example-retention-policy \ /path/to/backup-directory ``` #### Back up a specific shard > [!Note] > > [!Note] > To view shard IDs, use the [`SHOW SHARDS` InfluxQL statement](/influxdb/v1/query_language/spec/#show-shards). ```sh influxd backup -portable \ -rp example-retention-policy \ -shard 123 \ /path/to/backup-directory ``` * `-host`: InfluxDB bind address *(Only required if creating a backup from a remote InfluxDB host)* * `-db`: (Required) Database name * `-rp`: Retention policy name*(If no retention policy is specified, the command backs up all retention policies)* * `-shard`: Shard ID *(If no shard ID is specified, the command backs up all shards. Requires a retention policy.)* * `-start`: Start time *(If no start time is specified, the command backs up data from all time.)* * `-stop`: Stop time *(If no stop time is specified, the command backs up data to now.)* * (Required) **Backup destination directory path** ### Legacy backup examples * [Back up a database](#back-up-a-database) * [Back up a database from a remote InfluxDB instance](#back-up-a-database-from-a-remote-influxdb-instance) * [Back up a specific time range](#back-up-a-specific-time-range-legacy) * [Back up data from a specific time to now](#back-up-data-from-a-specific-time-to-now-legacy) * [Back up a specific retention policy](#back-up-a-specific-retention-policy-legacy) * [Back up a specific shard](#back-up-a-specific-shard-legacy) #### Back up a database ```sh influxd backup \ -db example-db \ /path/to/backup-directory ``` #### Back up a database from a remote InfluxDB instance ```sh influxd backup \ -db example-db \ -host 203.0.113.0:8088 \ /path/to/backup-directory ``` #### Back up a specific time range ```sh influxd backup \ -db example-db \ -start 2022-01-01T00:00:00Z \ -stop 2022-02-01T00:00:00Z \ /path/to/backup-directory ``` *For important information about how time-based backups work, see [Time-based backups](#time-based-backups).* #### Back up data from a specific time to now ```sh influxd backup \ -db example-db \ -start 2022-01-01T00:00:00Z \ /path/to/backup-directory ``` *For important information about how time-based backups work, see [Time-based backups](#time-based-backups).* #### Back up a specific retention policy ```sh influxd backup \ -db example-db \ -rp example-retention-policy \ /path/to/backup-directory ``` #### Back up a specific shard To view shard IDs, use the [`SHOW SHARDS` InfluxQL statement](/influxdb/v1/query_language/spec/#show-shards). ```sh influxd backup -portable \ -db example-db \ -rp example-retention-policy \ -shard 123 \ /path/to/backup-directory ``` ## Restore data Use the [`influxd restore` utility](/influxdb/v1/tools/influxd/restore/) to restore time series data and metadata to InfluxDB from an InfluxDB backup. > [!Note] > #### Cannot restore to an existing database > > The InfluxDB OSS 1.13 `restore` utility does **not** support incremental backups. > When restoring data to a running InfluxDB instance, the `restore` utility performs a full restore. > To preserve existing data, the `restore` utility does not allow restoring data > to a database that already exists.[\*](#note-offline-restore) > > If the target database already exists, the `restore` utility returns an error > similar to the following: > > ``` > error updating meta: DB metadata not changed. database may already exist > ``` > > There is a workaround for restoring data to an existing database. > For more information, see [Restore data to an existing database](#restore-data-to-an-existing-database). > > \* *Offline legacy restores can be used to restore data to an > existing database, but overwrite all data in the database.* Requirements and commands necessary to restore a backup depend on the[backup format](#backup-formats): [Portable format](#portable-format-2)[Legacy format](#legacy-format-2) * `-portable`: (Required) Specifies the portable backup format * `-host`: InfluxDB bind address *(Only required if restore a backup to a remote InfluxDB host)* * `-db`: Database name *(If no database name is specified, the command restores all databases)* * `-newdb`: New database name *(Required when restoring a database that already exists)* * `-rp`: Retention policy name*(If no retention policy is specified, the command restores all retention policies)* * `-newrp`: New retention policy name *(Required when restoring a retention policy that already exists)* * `-shard`: Shard ID *(If no shard ID is specified, the command backs up all shards. Requires a retention policy.)* * (Required) **Backup directory path** ### Restore examples * [Restore all databases](#restore-all-databases) * [Restore all data to a remote InfluxDB instance](#restore-all-data-to-a-remote-influxdb-instance) * [Restore a specific database](#restore-a-specific-database) * [Restore data to a database that already exists](#restore-data-to-a-database-that-already-exists) * [Restore a specific retention policy](#restore-a-specific-retention-policy) * [Restore data to a retention policy that already exists](#restore-data-to-a-retention-policy-that-already-exists) * [Restore a specific shard](#restore-a-specific-shard) #### Restore all databases ```sh influxd restore -portable /path/to/backup-directory ``` #### Restore all data to a remote InfluxDB instance ```sh influxd restore -portable \ -host 203.0.113.0:8088 \ /path/to/backup-directory ``` #### Restore a specific database ```sh influxd restore -portable \ -db example-db \ /path/to/backup-directory ``` #### Restore data to a database that already exists ```sh influxd restore -portable \ -db example-db \ -newdb example-new-db \ /path/to/backup-directory ``` #### Restore a specific retention policy ```sh influxd backup -portable \ -db example-db \ -rp example-retention-policy \ /path/to/backup-directory ``` #### Restore data to a retention policy that already exists ```sh influxd restore -portable \ -db example-db \ -rp example-rp \ -newrp example-new-rp \ /path/to/backup-directory ``` #### Restore a specific shard To restore a specific shard, you must specify a database and retention policy. To view shard IDs, use the [`SHOW SHARDS` InfluxQL statement](/influxdb/v1/query_language/spec/#show-shards). ```sh influxd backup -portable \ -db example-db \ -rp example-rp \ -shard 123 \ /path/to/backup-directory ``` Legacy backups can be restored to a running (online) or stopped (offline) InfluxDB instance. * [Online legacy restore](#online-legacy-restore) * [Offline legacy restore](#offline-legacy-restore) ### Online legacy restore * `-online`: (Required) Specifies that the target InfluxDB instance is running * `-host`: InfluxDB bind address *(Only required if restoring a backup from a remote InfluxDB host)* * `-db`: (Required) Database name * `-newdb`: New database name *(Required when restoring a database that already exists)* * `-rp`: Retention policy name*(If no retention policy is specified, the command backs up all retention policies)* * `-newrp`: New retention policy name *(Required when restoring a retention policy that already exists)* * `-shard`: Shard ID *(If no shard ID is specified, the command restores all shards. Requires a retention policy.)* * (Required) **Backup destination directory path** #### Online legacy restore examples * [Restore a database](#restore-a-database) * [Restore a database to a remote InfluxDB instance](#restore-a-database-to-a-remote-influxdb-instance) * [Restore data to a database that already exists](#restore-data-to-a-database-that-already-exists-legacy) * [Restore a specific retention policy](#restore-a-specific-retention-policy-legacy) * [Restore data to a retention policy that already exists](#restore-data-to-a-retention-policy-that-already-exists-legacy) * [Restore a specific shard](#restore-a-specific-shard-legacy) ##### Restore a database ```sh influxd restore -online \ -db example-db \ /path/to/backup-directory ``` ##### Restore a database to a remote InfluxDB instance ```sh influxd restore -online \ -db example-db \ -host 203.0.113.0:8088 \ /path/to/backup-directory ``` ##### Restore data to a database that already exists ```sh influxd restore -online \ -db example-db \ -newdb example-new-db \ /path/to/backup-directory ``` ##### Restore a specific retention policy ```sh influxd backup -online \ -db example-db \ -rp example-retention-policy \ /path/to/backup-directory ``` ##### Restore data to a retention policy that already exists ```sh influxd restore -online \ -db example-db \ -rp example-rp \ -newrp example-new-rp \ /path/to/backup-directory ``` ##### Restore a specific shard To restore a specific shard, you must specify a database and retention policy. To view shard IDs, use the [`SHOW SHARDS` InfluxQL statement](/influxdb/v1/query_language/spec/#show-shards). ```sh influxd backup -online \ -db example-db \ -rp example-rp \ -shard 123 \ /path/to/backup-directory ``` ### Offline legacy restore #### Offline restores overwrite data Offline restores are destructive and will overwrite all data in the destination database. #### Must be done on the same machine as InfluxDB Offline restores must be done from the machine where InfluxDB is running. They cannot be done remotely unless you console into the machine InfluxDB is running on and execute the restore process there. * `-db`: (Required) Database name * `-newdb`: New database name *(Required when restoring a database that already exists)* * `-rp`: Retention policy name*(If no retention policy is specified, the command backs up all retention policies)* * `-newrp`: New retention policy name *(Required when restoring a retention policy that already exists)* * `-datadir`: (Required) Location of destination data directory on disk *(See [InfluxDB file system layout](/influxdb/v1/concepts/file-system-layout/))* * `-metadir`: (Required) Location of destination meta directory on disk *(See [InfluxDB file system layout](/influxdb/v1/concepts/file-system-layout/))* * `-shard`: Shard ID *(If no shard ID is specified, the command backs up all shards. Requires a retention policy.)* * (Required) **Backup destination directory path** #### Offline legacy restore examples * [Restore a database](#restore-a-database-offline) * [Restore a specific retention policy](#restore-a-specific-retention-policy-offline) * [Restore a specific shard](#restore-a-specific-shard-offline) ##### Restore a database ```sh influxd restore \ -db example-db \ -datadir /path/to/data-directory \ -metadir /path/to/meta-directory \ /path/to/backup-directory ``` ##### Restore a specific retention policy ```sh influxd backup \ -db example-db \ -rp example-retention-policy \ -datadir /path/to/data-directory \ -metadir /path/to/meta-directory \ /path/to/backup-directory ``` ##### Restore a specific shard To restore a specific shard, you must specify a database and retention policy. To view shard IDs, use the [`SHOW SHARDS` InfluxQL statement](/influxdb/v1/query_language/spec/#show-shards). ```sh influxd backup \ -db example-db \ -rp example-rp \ -shard 123 \ -datadir /path/to/data-directory \ -metadir /path/to/meta-directory \ /path/to/backup-directory ``` ### Restore data to an existing database While backups can’t be restored directly to an existing database (except for [offline legacy restores](?t=Legacy+format#offline-legacy-restore)), you can do the following to restore data to an existing database: 1. Restore a backup to a temporary database. [Portable format](#portable-format-3)[Legacy format](#legacy-format-3) ``` influxd restore -portable \ -db example-db \ -newdb example-tmp-db \ /path/to/backup-directory/ ``` ``` influxd restore \ -db example-db \ -newdb example-tmp-db \ /path/to/backup-directory/ ``` 2. Use InfluxQL or Flux to query data from the temporary database and write it back to the existing database. [InfluxQL](#influxql)[Flux](#flux) ``` -- Repeat for each retention policy in the temporary database SELECT * INTO "example-db".autogen.:MEASUREMENT FROM "example-tmp-db".autogen./.*/ GROUP BY * ``` ``` // Repeat for each retention policy in the temporary database from(bucket: "example-tmp-db/autogen") |> range(start: 0) |> to(bucket: "example-db/autogen") ``` InfluxDB handles duplicate points added by this query as it normally does. For more information, see[How does InfluxDB handle duplicate points?](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-does-influxdb-handle-duplicate-points) 3. Use InfluxQL to delete the temporary database. ``` DROP DATABASE "example-tmp-db" ``` ## Configure backup and restore services InfluxDB OSS 1.13 `backup` and `restore` utilities execute over a TCP connection. The default IP and port used for backup and restore remote procedure calls (RPCs) are `127.0.0.1:8088`. **To customize the TCP IP and port the backup and restore services use**, uncomment and update the[`bind-address` configuration setting](/influxdb/v1/administration/config#rpc-bind-address)at the root level of your InfluxDB configuration file (`influxdb.conf`). ```toml # Bind address to use for the RPC service for backup and restore. bind-address = "127.0.0.1:8088" ``` ## Exporting and importing data For most applications, the [backup and restore utilities](#backup-and-restore-utilities) provide the tools you need for your backup and restore strategy. However, in some cases, the standard backup and restore utilities might not adequately handle the volumes of data in your application. As an alternative to the standard backup and restore utilities, use the InfluxDB `influx_inspect export` and `influx -import` commands to create backup and restore procedures for your disaster recovery and backup strategy. These commands can be executed manually or included in shell scripts that run the export and import operations at scheduled intervals. You can use the commands to export and import data between an InfluxDB OSS 1.13 instance and an [InfluxDB Enterprise](/enterprise_influxdb/v1/) cluster. * [Exporting data](#exporting-data) * [Importing data](#importing-data) * [Example: export and import for disaster recovery](#example-export-and-import-for-disaster-recovery) ### Exporting data Use the [`influx_inspect export` command](/influxdb/v1/tools/influx_inspect#export) to export data in line protocol format from your InfluxDB OSS 1.13 instance. Options include the following: * `-database`: Export all or specific databases * `-start` and `-end`: Filter with starting and ending timestamps * `-compress`: Use GNU zip (gzip) compression for smaller files and faster exports The following example shows how to export data filtered to one day and compressed for optimal speed and file size: ```bash influx_inspect export \ -database DATABASE_NAME \ -compress \ -start 2019-05-19T00:00:00.000Z \ -end 2019-05-19T23:59:59.999Z ``` The exported file contains the following: ```sh # DDL CREATE DATABASE WITH NAME # DML # CONTEXT-DATABASE: # CONTEXT-RETENTION-POLICY: ``` * `DDL`: an InfluxQL `CREATE` statement to create the target database when [importing the data](#importing-data) * `DML`: Context metadata that specifies the target database and retention policy for [importing the data](#importing-data) * the line protocol data For details on optional settings and usage, see [`influx_inspect export` command](/influxdb/v1/tools/influx_inspect#export). ### Importing data To import line protocol data from a file, use the [`influx -import` CLI command](/influxdb/v1/tools/influx-cli/use-influx-cli/#influx-arguments). In your import file, include the following sections: * *Optional*: **DDL (Data Definition Language)**: Contains the [InfluxQL commands](/influxdb/v1/query_language/manage-database/) for creating the relevant [database](/influxdb/v1/concepts/glossary/#database) and managing the [retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp). If your database and retention policy already exist, your file can skip this section. * **DML (Data Manipulation Language)**: Context metadata that specifies the database and (if desired) retention policy for the import and contains the data in [line protocol](/influxdb/v1/concepts/glossary/#influxdb-line-protocol). In the following example, the compressed data file (in GNU zip format) is imported into the database specified in the file’s `DML` metadata. ```bash influx -import -path -compressed ``` For details on using the `influx -import` command, see [Import data from a file](/influxdb/v1/tools/influx-cli/use-influx-cli/#import-data-from-a-file). ### Example: export and import for disaster recovery For an example of using the exporting and importing data approach for disaster recovery, see the presentation from Influxdays 2019 on [“Architecting for Disaster Recovery.”](https://www.youtube.com/watch?v=LyQDhSdnm4A). In this presentation, Capital One discusses the following: * Exporting data every 15 minutes from an active InfluxDB Enterprise cluster to an AWS S3 bucket. * Replicating the export file in the S3 bucket using the AWS S3 copy command. * Importing data every 15 minutes from the AWS S3 bucket to an InfluxDB Enterprise cluster available for disaster recovery. * Advantages of the export-import approach over the standard backup and restore utilities for large volumes of data. * Managing users and scheduled exports and imports with a custom administration tool. | Backup functionality | Legacy format | Portable format | | --- | --- | --- | | Backup functionality | Legacy format | Portable format | | Back up single databases | ✓ | ✓ | | Back up multiple databases | | ✓ | | Back up single retention polices | ✓ | ✓ | | Back up multiple retention policies | ✓ | ✓ | | Back up specific time ranges | ✓ | ✓ | | Back up remote databases | ✓ | ✓ | | Restore to InfluxDB OSS 1.0–1.4 | ✓ | | | Restore to InfluxDB OSS 1.5– 1.13 | ✓ | ✓ | | Restore to InfluxDB Enterprise | | ✓ | --- # Compact a series file offline Source: https://docs.influxdata.com/influxdb/v1/administration/compact-series-file/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the **Series File Compaction tool** to compact your series file and reduce its size on disk. This is useful for series files that grow quickly, for example, when series are frequently created and deleted. **To compact a series file:** 1. Stop the `influxd` process. 2. Run the following command, including your **data directory** and **WAL directory**: ``` # Syntax influx_inspect buildtsi -compact-series-file -datadir -waldir # Example influx_inspect buildtsi -compact-series-file -datadir /data -waldir /wal ``` 3. Restart the `influxd` process. 4. ***(InfluxDB Enterprise only)*** On each data node: 1. Complete steps 1-3. 2. Wait for the [hinted handoff queue (HHQ)](/enterprise_influxdb/v1/concepts/clustering/#hinted-handoff)to write all missed data to the node. 3. Continue to the next data node. --- # Configure InfluxDB OSS Source: https://docs.influxdata.com/influxdb/v1/administration/config/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The InfluxDB open source (OSS) configuration file contains configuration settings specific to a local node. #### Content * [Configuration overview](#configuration-overview) * [Environment variables](#environment-variables) * [InfluxDB environment variables (`INFLUXDB_*`)](#influxdb-environment-variables-influxdb) * [`GOMAXPROCS` environment variable](#gomaxprocs-environment-variable) * [Using the configuration file](#using-the-configuration-file) * [Configuration settings](#configuration-settings) * [Global settings](#global-settings) * [Metastore `[meta]`](#metastore-settings) * [Data `[data]`](#data-settings) * [Query management `[coordinator]`](#query-management-settings) * [Flux query management `[flux-controller]`](#flux-query-management-settings) * [Retention policies `[retention]`](#retention-policy-settings) * [Shard precreation `[shard-precreation]`](#shard-precreation-settings) * [Monitoring `[monitor]`](#monitoring-settings) * [HTTP endpoints `[http]`](#http-endpoints-settings) * [Subscriptions `[subscriber]`](#subscription-settings) * [Graphite `[[graphite]]`](#graphite-settings) * [CollectD `[[collectd]]`](#collectd-settings) * [OpenTSB `[[opentsdb]]`](#opentsdb-settings) * [UDP `[[udp]]`](#udp-settings) * [Continuous queries `[continuous_queries]`](#continuous-queries-settings) * [TLS `[tls]`](#transport-layer-security-tls-settings) ## Configuration overview InfluxDB is configured using the configuration file (`influxdb.conf`) and environment variables. If you do not uncomment a configuration option, the system uses its default setting. The configuration settings in this document are set to their default settings. Configuration settings that specify a duration support the following duration units: * `ns` *(nanoseconds)* * `us` or `µs` *(microseconds)* * `ms` *(milliseconds)* * `s` *(seconds)* * `m` *(minutes)* * `h` *(hours)* * `d` *(days)* * `w` *(weeks)* > [!Note] > Configuration file settings are documented here for the latest official release. > The [sample configuration file on GitHub](https://github.com/influxdb/influxdb/blob/1.8/etc/config.sample.toml)might be slightly newer. ## Environment variables All of the configuration settings in the configuration file can be specified either in the configuration file or in an environment variable. The environment variable overrides the equivalent option in the configuration file. If a configuration option is not specified in either the configuration file or in an environment variable, InfluxDB uses its internal default configuration. > [!Note] > If an environment variable has already been set, the equivalent configuration > setting in the configuration file is ignored. ### InfluxDB environment variables (`INFLUXDB_*`) The InfluxDB environment variables are documented below with the corresponding configuration file settings. All of the InfluxDB-specific environment variables are prefixed with `INFLUXDB_`. ### `GOMAXPROCS` environment variable > [!Note] > The GOMAXPROCS environment variable cannot be set using the InfluxDB > configuration file settings, like other environment variables. The `GOMAXPROCS` [Go language environment variable](https://golang.org/pkg/runtime/#hdr-Environment_Variables)can be used to set the maximum number of CPUs that can execute simultaneously. The default value of `GOMAXPROCS` is the number of CPUs (whatever your operating system considers to be a CPU) that are visible to the program *on startup*. For a 32-core machine, the `GOMAXPROCS` value would be `32`. You can override this value to be less than the maximum value, which can be useful in cases where you are running the InfluxDB along with other processes on the same machine and want to ensure that the database doesn’t completely starve those processes. > [!Note] > Setting `GOMAXPROCS=1` will eliminate all parallelization. ## Using the configuration file The InfluxDB system has internal defaults for all of the settings in the configuration file. To view the default configuration settings, use the`influxd config` command. The local InfluxDB configuration file is located here: * **Linux**: `/etc/influxdb/influxdb.conf` * **macOS**: `/usr/local/etc/influxdb.conf` * **Windows**: *Same directory as `influxd.exe`* Settings that are commented out are set to the internal system defaults. Uncommented settings override the internal defaults. Note that the local configuration file does not need to include every configuration setting. Use one of the following methods to start InfluxDB with your configuration file: * Point the process to the configuration file by using the `-config`option. For example: ``` influxd -config /etc/influxdb/influxdb.conf ``` * Set the environment variable `INFLUXDB_CONFIG_PATH` to the path of your configuration file and start the process. For example: ``` echo $INFLUXDB_CONFIG_PATH /etc/influxdb/influxdb.conf influxd ``` InfluxDB first checks for the `-config` option and then for the environment variable. ## Configuration settings > [!Note] > To set or override settings in a config section that allows multiple > configurations (any section with `[[double_brackets]]` in the header supports > multiple configurations), the desired configuration must be specified by ordinal > number. > For example, for the first set of `[[graphite]]` environment variables, > prefix the configuration setting name in the environment variable with the > relevant position number (in this case: `0`): > > ``` > INFLUXDB_GRAPHITE_0_BATCH_PENDING > INFLUXDB_GRAPHITE_0_BATCH_SIZE > INFLUXDB_GRAPHITE_0_BATCH_TIMEOUT > INFLUXDB_GRAPHITE_0_BIND_ADDRESS > INFLUXDB_GRAPHITE_0_CONSISTENCY_LEVEL > INFLUXDB_GRAPHITE_0_DATABASE > INFLUXDB_GRAPHITE_0_ENABLED > INFLUXDB_GRAPHITE_0_PROTOCOL > INFLUXDB_GRAPHITE_0_RETENTION_POLICY > INFLUXDB_GRAPHITE_0_SEPARATOR > INFLUXDB_GRAPHITE_0_TAGS > INFLUXDB_GRAPHITE_0_TEMPLATES > INFLUXDB_GRAPHITE_0_UDP_READ_BUFFER > ``` > > For the Nth Graphite configuration in the configuration file, the relevant > environment variables would be of the form `INFLUXDB_GRAPHITE_(N-1)_BATCH_PENDING`. > For each section of the configuration file the numbering restarts at zero. ## Global settings ### reporting-disabled InfluxData uses voluntarily reported data from running InfluxDB nodes primarily to track the adoption rates of different InfluxDB versions. This data helps InfluxData support the continuing development of InfluxDB. The `reporting-disabled` option toggles the reporting of data every 24 hours to `usage.influxdata.com`. Each report includes a randomly-generated identifier, OS, architecture, InfluxDB version, and the number of[databases](/influxdb/v1/concepts/glossary/#database),[measurements](/influxdb/v1/concepts/glossary/#measurement), and unique[series](/influxdb/v1/concepts/glossary/#series). Setting this option to`true` will disable reporting. > [!Note] > **Note:** No data from user databases is ever transmitted. **Default**: `false` **Environment variable**: `INFLUXDB_REPORTING_DISABLED` ### bind-address The `bind-address` option specifies the address and port to bind the RPC service for [backup and restore](/influxdb/v1/administration/backup_and_restore/). **Default**: `127.0.0.1:8088` **Environment variable**: `INFLUXDB_BIND_ADDRESS` ### compact-series-file The `compact-series-file` options determines if series files should be compacted on startup. If `true`, InfluxDB runs[`influxd_inspect -compact-series-file`](/influxdb/v1/tools/influx_inspect/#--compact-series-file-)before starting the `influxd` server. Default: `false` Environment variable: `INFLUXDB_COMPACT_SERIES_FILE` ## Metastore settings ### [meta] This section controls parameters for the InfluxDB metastore, which stores information on users, databases, retention policies, shards, and continuous queries. ### dir The directory where the metadata/raft database is stored. Files in the `meta`directory include `meta.db`, the InfluxDB metastore file. > [!Note] > The default directory for macOS installations is `/Users//.influxdb/meta`. **Default**: `/var/lib/influxdb/meta` **Environment variable**: `INFLUXDB_META_DIR` ### retention-autocreate Enables the automatic creation of the[`DEFAULT` retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp)`autogen` when a database is created. The retention policy `autogen` has an infinite duration and is also set as the database’s `DEFAULT` retention policy, which is used when a write or query does not specify a retention policy. Disable this setting to prevent the creation of this retention policy when creating databases. **Default**: `true` **Environment variable**: `INFLUXDB_META_RETENTION_AUTOCREATE` ### logging-enabled Enables the logging of messages from the meta service. **Default**: `true` **Environment variable**: `INFLUXDB_META_LOGGING_ENABLED` ## Data settings ### [data] The `[data]` settings control where the actual shard data for InfluxDB lives and how it is flushed from the Write-Ahead Log (WAL).`dir` may need to be changed to a suitable place for your system, but the WAL settings are an advanced configuration. The defaults should work for most systems. #### dir The InfluxDB directory where the TSM engine stores TSM files. This directory may be changed. > [!Note] > The default directory for macOS installations is `/Users//.influxdb/data`. **Default**: `/var/lib/influxdb/data` **Environment variable**: `INFLUXDB_DATA_DIR` #### wal-dir The location of the directory for[write ahead log (WAL)](/influxdb/v1/concepts/glossary/#wal-write-ahead-log)files. > [!Note] > For macOS installations, the default WAL directory is`/Users//.influxdb/wal`. **Default**: `/var/lib/influxdb/wal` **Environment variable**: `INFLUXDB_DATA_WAL_DIR` #### wal-fsync-delay The amount of time that a write waits before fsyncing. Use a duration greater than `0` to batch up multiple fsync calls. This is useful for slower disks or when experiencing [WAL](/influxdb/v1/concepts/glossary/#wal-write-ahead-log)write contention. > [!Note] > For non-SSD disks, InfluxData recommends values in the range of `0ms`-`100ms`. **Default**: `0s` **Environment variable**: `INFLUXDB_DATA_WAL_FSYNC_DELAY` #### index-version The type of shard index to use for new shards. The default (`inmem`) index is an in-memory index that is recreated at startup. To enable the Time Series Index (TSI) disk-based index, set the value to `tsi1`. **Default**: `inmem` **Environment variable**: `INFLUXDB_DATA_INDEX_VERSION` #### trace-logging-enabled Enables verbose logging of additional debug information within the TSM engine and WAL. The trace logging provides more useful output for debugging TSM engine issues. **Default**: `false` **Environment variable**: `INFLUXDB_DATA_TRACE_LOGGING_ENABLED` #### query-log-enabled Enables the logging of parsed queries before execution. The query log can be useful for troubleshooting, but logs any sensitive data contained within a query. **Default**: `true` **Environment variable**: `INFLUXDB_DATA_QUERY_LOG_ENABLED` #### strict-error-handling When set to `false`, any query attempting to insert an unsupported value, for example `+/-Inf` or `NaN`, fails to insert the unsupported value silently and proceeds to insert any valid points in the query. Set to `true` to provide more error checking. For example, a `SELECT INTO`query attempting to insert a `+/-Inf` value returns an error (rather than failing silently), and no points will be inserted. **Default**: `false` **Environment variable**: `INFLUXDB_DATA_STRICT_ERROR_HANDLING` #### validate-keys Validates incoming writes to ensure measurement keys and tag keys only have valid Unicode characters. This setting will incur a small overhead because every key must be checked. This will not validate field keys. **Default**: `false` **Environment variable**: `INFLUXDB_VALIDATE_KEYS` ### tar-stream-buffer-size The size (in bytes) of the tar buffer window while running tar streaming operations, such as renaming and copying tar files during backups. > [!Note] > Only change `tar-stream-buffer-size` from the default if backups have performance issues. Understand that this is a heuristic > and may require experimentation on your system. **Default**: `"1048576"` **Environment variable**: `INFLUXDB_TAR_STREAM_BUFFER_SIZE` ### Settings for the TSM engine #### cache-max-memory-size The maximum size that a shard cache can reach before it starts rejecting writes. Valid memory size suffixes are: `k`, `m`, or `g` (case-insensitive, 1024 = 1k). Values without a size suffix are in bytes. > [!Note] > Consider increasing this value if encountering`cache maximum memory size exceeded` errors. **Default**: `"1g"` **Environment variable**: `INFLUXDB_DATA_CACHE_MAX_MEMORY_SIZE` #### cache-snapshot-memory-size The size at which the engine will snapshot the cache and write it to a TSM file, freeing up memory. Valid memory size suffixes are: `k`, `m`, or `g` (case-insensitive, 1024 = 1k). Values without a size suffix are in bytes. **Default**: `"25m"` **Environment variable**: `INFLUXDB_DATA_CACHE_SNAPSHOT_MEMORY_SIZE` #### cache-snapshot-write-cold-duration The time interval at which the engine will snapshot the cache and write it to a new TSM file if the shard hasn’t received writes or deletes. **Default**: `"10m"` **Environment variable**: `INFLUXDB_DATA_CACHE_SNAPSHOT_WRITE_COLD_DURATION` #### compact-full-write-cold-duration The time interval at which the TSM engine will compact all TSM files in a shard if it hasn’t received a write or delete. **Default**: `"4h"` **Environment variable**: `INFLUXDB_DATA_COMPACT_FULL_WRITE_COLD_DURATION` #### max-concurrent-compactions The maximum number of concurrent full and level[compactions](/influxdb/v1/concepts/storage_engine/#compactions) that can run at one time. The default value of `0` results in 50% of the CPU cores being used at runtime for compactions. If explicitly set, the number of cores used for compaction is limited to the specified value. This setting does not apply to cache snapshotting. For more information on the `GOMAXPROCS` environment variable, see[`GOMAXPROCS` environment variable](#gomaxprocs-environment-variable) on this page. **Default**: `0` **Environment variable**: `INFLUXDB_DATA_MAX_CONCURRENT_COMPACTIONS` #### max-concurrent-deletes The maximum number of simultaneous `DELETE` calls on a shard. Default is `1`and should be left unchanged for most use cases. **Default**: `1` **Environment variable**: `INFLUXDB_DATA_MAX_CONCURRENT_DELETES` #### compact-throughput The maximum number of bytes per second TSM compactions write to disk. Short bursts are allowed to exceed this value, as determined by`compact-throughput-burst`. **Default**: `"48m"` (48 million) **Environment variable**: `INFLUXDB_DATA_COMPACT_THROUGHPUT` #### compact-throughput-burst The maximum number of bytes per second TSM compactions write to disk during brief bursts. **Default**: `"48m"` (48 million) **Environment variable**: `INFLUXDB_DATA_COMPACT_THROUGHPUT_BURST` #### aggressive-points-per-block The number of points per block to use during aggressive compaction. In certain cases, TSM files do not get fully compacted. This adjusts an internal parameter to help ensure these files do get fully compacted. **Default**: `10000` **Environment variable**: `INFLUXDB_AGGRESSIVE_POINTS_PER_BLOCK` #### tsm-use-madv-willneed If `true`, the MMap Advise value `MADV_WILLNEED` advises the kernel about how to handle the mapped memory region in terms of input/output paging and to expect access to the mapped memory region in the near future, with respect to TSM files. > [!Note] > Because this setting has been problematic on some kernels (including CentOS > and RHEL), the default is `false`. Changing the value to `true` might help > users who have slow disks in some cases. **Default**: `false` **Environment variable**: `INFLUXDB_DATA_TSM_USE_MADV_WILLNEED` ### In-memory (`inmem`) index settings #### max-series-per-database The maximum number of [series](/influxdb/v1/concepts/glossary/#series) allowed per database before writes are dropped. The default setting is `1000000` (one million). Change the setting to `0` to allow an unlimited number of series per database. If a point causes the number of series in a database to exceed`max-series-per-database`, InfluxDB will not write the point, and it returns a`500` with the following error: ``` {"error":"max series per database exceeded: "} ``` > [!Note] > Any existing databases with a series count that exceeds`max-series-per-database` will continue to accept writes to existing series, > but writes that create a new series will fail. **Default**: `1000000` **Environment variable**: `INFLUXDB_DATA_MAX_SERIES_PER_DATABASE` #### max-values-per-tag The maximum number of[tag values](/influxdb/v1/concepts/glossary/#tag-value) allowed per[tag key](/influxdb/v1/concepts/glossary/#tag-key). The default value is`100000` (one hundred thousand). Change the setting to `0` to allow an unlimited number of tag values per tag key. If a tag value causes the number of tag values of a tag key to exceed `max-values-per-tag`, then InfluxDB will not write the point, and it returns a `partial write` error. > [!Note] > Any existing tag keys with tag values that exceed `max-values-per-tag` will > continue to accept writes, but writes that create a new tag value will fail. **Default**: `100000` **Environment variable**: `INFLUXDB_DATA_MAX_VALUES_PER_TAG` ### TSI (`tsi1`) index settings #### max-index-log-file-size The threshold, in bytes, when an index write-ahead log (WAL) file will compact into an index file. Lower sizes will cause log files to be compacted more quickly and result in lower heap usage at the expense of write throughput. Higher sizes will be compacted less frequently, store more series in memory, and provide higher write throughput. Valid size suffixes are `k`, `m`, or `g`(case-insensitive, 1024 = 1k). Values without a size suffix are in bytes. **Default**: `"1m"` **Environment variable**: `INFLUXDB_DATA_MAX_INDEX_LOG_FILE_SIZE` #### series-id-set-cache-size The size of the internal cache used in the TSI index to store previously calculated series results. Cached results will be returned quickly from the cache rather than needing to be recalculated when a subsequent query with a matching tag key-value predicate is executed. Setting this value to `0` will disable the cache, which may lead to query performance issues. This value should only be increased if it is known that the set of regularly used tag key-value predicates across all measurements for a database is larger than 100. An increase in cache size may lead to an increase in heap usage. **Default**: `100` **Environment variable**: `INFLUXDB_DATA_SERIES_ID_SET_CACHE_SIZE` #### series-id-set-cache-max-size The upper bound, in number of entries, for adaptive growth of the[`series-id-set-cache-size`](#series-id-set-cache-size) cache. Set this together with `series-id-set-cache-target-hit-rate` to let the cache grow beyond its fixed size when the query hit rate falls below the target. A value of `0`disables adaptive sizing. > [!Note] > `series-id-set-cache-max-size` and `series-id-set-cache-target-hit-rate` must > both be set to enable adaptive sizing, or both left at `0` to disable it. > Setting only one prevents InfluxDB from starting. When adaptive sizing is > enabled, `series-id-set-cache-max-size` must be greater than`series-id-set-cache-size`, and `series-id-set-cache-size` must be greater > than `0`. A `series-id-set-cache-size` of `0` disables the cache entirely, > which takes precedence over adaptive sizing. **Default**: `0` **Environment variable**: `INFLUXDB_DATA_SERIES_ID_SET_CACHE_MAX_SIZE` #### series-id-set-cache-target-hit-rate The cache hit rate, as a fraction between `0.0` and `1.0` (exclusive), that adaptive sizing tries to reach for the[`series-id-set-cache-size`](#series-id-set-cache-size) cache. While the measured hit rate stays below this target and the cache is evicting entries, InfluxDB grows the cache capacity, up to `series-id-set-cache-max-size`. A value of `0.0` disables adaptive sizing. `1.0` isn’t a valid value because that hit rate can never be reached. **Default**: `0.0` **Environment variable**: `INFLUXDB_DATA_SERIES_ID_SET_CACHE_TARGET_HIT_RATE` #### series-id-set-cache-shrink-conservatism How reluctant the adaptive shrink policy is to release memory from the[`series-id-set-cache-size`](#series-id-set-cache-size) cache, expressed in standard deviations. Higher values make the cache hold onto capacity longer and resist rapid grow-and-shrink cycles. Lower values reclaim memory sooner after demand drops. InfluxDB only uses this setting when adaptive sizing is enabled. **Default**: `2.5` **Environment variable**: `INFLUXDB_DATA_SERIES_ID_SET_CACHE_SHRINK_CONSERVATISM` To monitor the series ID set cache, run `SHOW STATS` and check the`tsi1_cache` measurement. It reports `hit`, `miss`, `eviction`,`shrink_eviction`, `size`, and `capacity` fields. ## Query management settings ### [coordinator] This section contains configuration settings for query management. For more on managing queries, see[Query Management](/influxdb/v1/troubleshooting/query_management/). #### write-timeout The duration a write request waits until a “timeout” error is returned to the caller. **Default**: `"10s"` **Environment variable**: `INFLUXDB_COORDINATOR_WRITE_TIMEOUT` #### max-concurrent-queries The maximum number of running queries allowed on your instance. **Default**: `0` (unlimited) **Environment variable**: `INFLUXDB_COORDINATOR_MAX_CONCURRENT_QUERIES` #### query-timeout The maximum duration that a query is allowed to execute before InfluxDB kills the query. This setting is a [duration](#configuration-overview). **Default**: `"0s"` (no time restrictions) **Environment variable**: `INFLUXDB_COORDINATOR_QUERY_TIMEOUT` #### log-queries-after The maximum duration that a query can run until InfluxDB logs the query with a`Detected slow query` message. This setting is a [duration](#configuration-overview). **Default**: `"0s"` (never logs slow queries) **Environment variable**: `INFLUXDB_COORDINATOR_LOG_QUERIES_AFTER` #### max-select-point The maximum number of [points](/influxdb/v1/concepts/glossary/#point) that a`SELECT` statement can process. **Default**: `0` (unlimited) **Environment variable**: `INFLUXDB_COORDINATOR_MAX_SELECT_POINT` #### max-select-series The maximum number of[series](/influxdb/v1/concepts/glossary/#series) that a `SELECT` statement can process. > [!Note] > The default setting (`0`) allows the `SELECT` statement to process an > unlimited number of series. **Default**: `0` **Environment variable**: `INFLUXDB_COORDINATOR_MAX_SELECT_SERIES` #### max-select-buckets The maximum number of `GROUP BY time()` buckets that a query can process. > [!Note] > The default setting (`0`) allows a query to process an unlimited number > of buckets. **Default**: `0` **Environment variable**: `INFLUXDB_COORDINATOR_MAX_SELECT_BUCKETS` #### termination-query-log Prints a list of running queries when InfluxDB receives a `SIGTERM`(sent when a process exceeds a container memory limit or by the `kill`command). **Default**: `false` **Environment variable**: `INFLUXDB_COORDINATOR_TERMINATE_QUERY_LOG` ## Flux query management settings ### [flux-controller] The `flux-controller` settings control the behavior of Flux queries. #### query-concurrency The number of Flux queries that are allowed to execute concurrently. `0` means unlimited. **Default**: `0` **Environment variable**: `INFLUXDB_FLUX_CONTROLLER_QUERY_CONCURRENCY` #### query-initial-memory-bytes The initial number of bytes allocated for a Flux query when it is started. If this is unset, then the `query-max-memory-bytes` is used. `0` means unlimited. **Default**: `0` **Environment variable**: `INFLUXDB_FLUX_CONTROLLER_QUERY_INITIAL_MEMORY_BYTES` #### query-max-memory-bytes The maximum number of bytes (in table memory) a Flux query is allowed to use at any given time. `0` means unlimited. A query may not be able to use its entire quota of memory if requesting more memory would conflict with the maximum amount of memory that the controller can request. **Default**: `0` **Environment variable**: `INFLUXDB_FLUX_CONTROLLER_QUERY_MAX_MEMORY_BYTES` #### total-max-memory-bytes The maximum amount of memory the Flux query controller is allowed to allocate to Flux queries. `0` means unlimited. If this is unset, then this number is `query-concurrency * query-max-memory-bytes`. This number must be greater than or equal to`query-concurrency * query-initial-memory-bytes` and may be less than`query-concurrency * query-max-memory-bytes`. **Default**: `0` **Environment variable**: `INFLUXDB_FLUX_CONTROLLER_TOTAL_MAX_MEMORY_BYTES` #### query-queue-size The number of Flux queries that are allowed to be queued for execution before new queries are rejected. `0` means unlimited. > [!Note] > The default value of `0` allows an unlimited number of queries to be queued. **Default**: `0` **Environment variable**: `INFLUXDB_FLUX_CONTROLLER_QUERY_QUEUE_SIZE` ## Retention policy settings ### [retention] The `[retention]` settings control the enforcement of retention policies for evicting old data. #### enabled Set to `false` to prevent InfluxDB from enforcing retention policies. **Default**: `true` **Environment variable**: `INFLUXDB_RETENTION_ENABLED` #### check-interval The time interval at which InfluxDB checks to enforce a retention policy. **Default**: `"30m0s"` **Environment variable**: `INFLUXDB_RETENTION_CHECK_INTERVAL` ## Shard precreation settings ### [shard-precreation] The `[shard-precreation]` settings control the precreation of shards so that shards are available before data arrive. Only shards that, after creation, will have both a start- and end-time in the future are ever created. Shards that would be wholly or partially in the past are never precreated. #### enabled Determines whether the shard precreation service is enabled. **Default**: `true` **Environment variable**: `INFLUXDB_SHARD_PRECREATION_ENABLED` #### check-interval The time interval when the check to precreate new shards runs. **Default**: `"10m"` **Environment variable**: `INFLUXDB_SHARD_PRECREATION_CHECK_INTERVAL` #### advance-period The maximum period in the future for which InfluxDB precreates shards. The `30m` default should work for most systems. Increasing this setting too far in the future can cause inefficiencies. **Default**: `"30m"` **Environment variable**: `INFLUXDB_SHARD_PRECREATION_ADVANCE_PERIOD` ## Monitoring settings ### [monitor] The `[monitor]` section settings control the InfluxDB[system self-monitoring](https://github.com/influxdata/influxdb/blob/1.8/monitor/README.md). By default, InfluxDB writes the data to the `_internal` database. If that database does not exist, InfluxDB creates it automatically. The `DEFAULT` retention policy on the `_internal` database is seven days. If you want to use a retention policy other than the seven-day retention policy, you must [create](/influxdb/v1/query_language/manage-database/#retention-policy-management) it. #### store-enabled Set to `false` to disable recording statistics internally. If set to `false`, it will make it substantially more difficult to diagnose issues with your installation. **Default**: `true` **Environment variable**: `INFLUXDB_MONITOR_STORE_ENABLED` #### store-database The destination database for recorded statistics. **Default**: `_internal` **Environment variable**: `INFLUXDB_MONITOR_STORE_DATABASE` #### store-interval The time interval at which InfluxDB records statistics. The default value is every ten seconds (`10s`). **Default**: `10s` **Environment variable**: `INFLUXDB_MONITOR_STORE_INTERVAL` ## HTTP endpoints settings ### [http] The `[http]` section settings control how InfluxDB configures the HTTP endpoints. These are the primary mechanisms for getting data into and out of InfluxDB. Edit the settings in this section to enable HTTPS and authentication. For details on enabling HTTPS and authentication, see[Authentication and Authorization](/influxdb/v1/administration/authentication_and_authorization/). #### enabled Determines whether the HTTP endpoints are enabled. To disable access to the HTTP endpoints, set the value to `false`. Note that the InfluxDB[command line interface (CLI)](/influxdb/v1/tools/shell/) connects to the database using the InfluxDB API. **Default**: `true` **Environment variable**: `INFLUXDB_HTTP_ENABLED` #### flux-enabled Determines whether the Flux query endpoint is enabled. To enable the use of Flux queries, set the value to `true`. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_FLUX_ENABLED` #### flux-log-enabled Determines whether detailed logs for Flux queries are enabled. When set to `true`, InfluxDB logs detailed information about Flux queries, including: * `compiler_type`: Compiler used for processing the query (typically Flux). * `response_size`: Size of the response, in bytes. * `query`: The textual representation of the query. * `err`: Errors encountered while processing the query. * `stat_total_duration`: Total duration to process the query. * `stat_compile_duration`: Duration to compile the query. * `stat_execute_duration`: Duration to execute the query. * `stat_max_allocated`: Maximum allocated memory for the query. * `stat_total_allocated`: Total allocated memory for the query. To enable Flux logging, both `flux-enabled` and `flux-log-enabled` must be set to `true`. For more information about InfluxDB logging, see [Log and trace with InfluxDB](/influxdb/v1/administration/logs). **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_FLUX_LOG_ENABLED` #### bind-address The bind address (port) used by the HTTP service. **Default**: `":8086"` **Environment variable**: `INFLUXDB_HTTP_BIND_ADDRESS` #### auth-enabled Determines whether user authentication is enabled over HTTP and HTTPS. To require authentication, set the value to `true`. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_AUTH_ENABLED` #### realm The default realm sent back when issuing a basic authentication challenge. The realm is the JWT realm used by the HTTP endpoints. **Default**: `"InfluxDB"` **Environment variable**: `INFLUXDB_HTTP_REALM` #### log-enabled Determines whether HTTP request logging is enabled. To disable logging, set the value to `false`. **Default**: `true` **Environment variable**: `INFLUXDB_HTTP_LOG_ENABLED` #### suppress-write-log Determines whether the HTTP write request logs should be suppressed when the log is enabled. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_SUPPRESS_WRITE_LOG` #### access-log-path The path to the access log, which determines whether detailed write logging is enabled using `log-enabled = true`. Specifies whether HTTP request logging is written to the specified path when enabled. If `influxd` is unable to access the specified path, it will log an error and fall back to `stderr`. When HTTP request logging is enabled, this option specifies the path where log entries should be written. If unspecified, the default is to write to stderr, which intermingles HTTP logs with internal InfluxDB logging. If`influxd` is unable to access the specified path, it will log an error and fall back to writing the request log to `stderr`. **Default**: `""` **Environment variable**: `INFLUXDB_HTTP_ACCESS_LOG_PATH` #### access-log-status-filters Filters which requests should be logged. Each filter is of the pattern `nnn`,`nnx`, or `nxx` where `n` is a number and `x` is the wildcard for any number. To filter all `5xx` responses, use the string `5xx`. If multiple filters are used, then only one has to match. The default value is no filters, with every request being printed. **Default**: `[]` **Environment variable**: `INFLUXDB_HTTP_ACCESS_LOG_STATUS_FILTERS_x` ##### Examples ###### Set access log status filters using configuration settings ```txt access-log-status-filters = ["4xx", "5xx"] ``` `"4xx"` is in array position `0``"5xx"` is in array position `1` ###### Set access log status filters using environment variables The input values for the `access-log-status-filters` is an array. When using environment variables, the values can be supplied as follows. ``` INFLUXDB_HTTP_ACCESS_LOG_STATUS_FILTERS_0=4xx INFLUXDB_HTTP_ACCESS_LOG_STATUS_FILTERS_1=5xx ``` The `_n` at the end of the environment variable represents the array position of the entry. #### write-tracing Determines whether detailed write logging is enabled. Set to `true` to enable logging for the write payload. If set to `true`, this will duplicate every write statement in the logs and is thus not recommended for general use. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_WRITE_TRACING` #### pprof-enabled Determines whether the `/net/http/pprof` HTTP endpoint is enabled. Useful for troubleshooting and monitoring. **Default**: `true` **Environment variable**: `INFLUXDB_HTTP_PPROF_ENABLED` #### pprof-auth-enabled Enables authentication on `/debug` endpoints. If enabled, users need admin permissions to access the following endpoints: * `/debug/pprof` * `/debug/requests` * `/debug/vars` This setting has no effect if either[`auth-enabled`](#auth-enabled) or[`pprof-enabled`](#pprof-enabled) are set to `false`. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_PPROF_AUTH_ENABLED` #### debug-pprof-enabled Enables the default `/pprof` endpoint and binds against `localhost:6060`. Useful for debugging startup performance issues. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_DEBUG_PPROF_ENABLED` #### ping-auth-enabled Enables authentication on the `/ping`, `/metrics`, and deprecated `/status` endpoints. This setting has no effect if [`auth-enabled`](#auth-enabled) is set to `false`. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_PING_AUTH_ENABLED` #### prom-read-auth-enabled Enables authentication on the Prometheus remote read API. This setting has no effect if [`auth-enabled`](#auth-enabled) is set to `false`. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_PROM_READ_AUTH_ENABLED` #### user-query-bytes-enabled Enables per-user query response byte tracking. When enabled, InfluxDB records the number of bytes returned by queries for each user in the `userquerybytes` measurement, available through `SHOW STATS FOR 'userquerybytes'`, the `_internal` database, and the `/debug/vars` endpoint. Unauthenticated queries are attributed to `(anonymous)`. **Default**: `false`**Environment variable**: `INFLUXDB_HTTP_USER_QUERY_BYTES_ENABLED` #### user-write-bytes-enabled Enables per-user write request byte tracking, the write-path counterpart to[`user-query-bytes-enabled`](#user-query-bytes-enabled). When enabled, InfluxDB records the request body bytes for each v1, v2, and Prometheus remote write, per user, in the `userwritebytes` measurement, available through `SHOW STATS FOR 'userwritebytes'`, the `_internal`database, and the `/debug/vars` endpoint. Unauthenticated writes are attributed to `(anonymous)`. > [!Note] > Byte counts use each endpoint’s existing units: `/write` counts > decompressed (post-gzip) bytes, while the Prometheus remote write endpoint > counts compressed wire bytes. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_USER_WRITE_BYTES_ENABLED` #### http-headers User-supplied [HTTP response headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers). Configure this section to return[security headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers#security)such as `X-Frame-Options` or `Content Security Policy` where needed. Example: ```toml [http.headers] X-Frame-Options = "DENY" ``` #### https-enabled Determines whether HTTPS is enabled. To enable HTTPS, set the value to `true`. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_HTTPS_ENABLED` #### https-certificate The path of the SSL certificate file to use when HTTPS is enabled. **Default**: `"/etc/ssl/influxdb.pem"` **Environment variable**: `INFLUXDB_HTTP_HTTPS_CERTIFICATE` #### https-private-key Use a separate private key location. If only the `https-certificate` is specified, the `httpd` service will try to load the private key from the`https-certificate` file. If a separate `https-private-key` file is specified, the `httpd` service will load the private key from the `https-private-key` file. **Default**: `""`**Environment variable**: `INFLUXDB_HTTP_HTTPS_PRIVATE_KEY` #### https-insecure-certificate Skips file permission checking for `https-certificate` and `https-private-key` when `true`. **Default**: `false`**Environment variable**: `INFLUXDB_HTTP_HTTPS_INSECURE_CERTIFICATE` #### shared-secret The shared secret used to validate public API requests using JWT tokens. **Default**: `""` **Environment variable**: `INFLUXDB_HTTP_SHARED_SECRET` #### max-row-limit The maximum number of rows that the system can return in a[non-chunked](/influxdb/v1/tools/api#query-string-parameters) query. The default setting (`0`) allows for an unlimited number of rows. If the query results exceed a specified value, then InfluxDB includes a `"partial":true`tag in the response body. **Default**: `0` **Environment variable**: `INFLUXDB_HTTP_MAX_ROW_LIMIT` #### max-connection-limit The maximum number of connections that may be open at once. New connections that would exceed the limit are dropped. The default value of `0` disables the limit. **Default**: `0` **Environment variable**: `INFLUXDB_HTTP_MAX_CONNECTION_LIMIT` #### unix-socket-enabled Enable HTTP service over the UNIX domain socket. To enable HTTP service over the UNIX domain socket, set the value to `true`. **Default**: `false` **Environment variable**: `INFLUXDB_HTTP_UNIX_SOCKET_ENABLED` #### bind-socket The path of the UNIX domain socket. **Default**: `"/var/run/influxdb.sock"` **Environment variable**: `INFLUXDB_HTTP_UNIX_BIND_SOCKET` #### max-body-size The maximum size, in bytes, of a client request body. When an HTTP client sends data that exceeds the configured maximum size, a `413 Request Entity Too Large` HTTP response is returned. To disable the limit, set the value to `0`. **Default**: `25000000` **Environment variable**: `INFLUXDB_HTTP_MAX_BODY_SIZE` #### max-concurrent-write-limit The maximum number of writes that can be processed concurrently. To disable the limit, set the value to `0`. **Default**: `0` **Environment variable**: `INFLUXDB_HTTP_MAX_CONCURRENT_WRITE_LIMIT` #### max-enqueued-write-limit The maximum number of writes queued for processing. To disable the limit, set the value to `0`. **Default**: `0` **Environment variable**: `INFLUXDB_HTTP_MAX_ENQUEUED_WRITE_LIMIT` #### enqueued-write-timeout The maximum duration for a write to wait in the queue to be processed. To disable the limit, set this to `0` or set the `max-concurrent-write-limit`value to `0`. **Default**: `0` **Environment variable**: `INFLUXDB_HTTP_ENQUEUED_WRITE_TIMEOUT` ## Logging settings ### [logging] Controls how the logger emits logs to the output. #### format Determines which log encoder to use for logs. Valid values are `auto` (default),`logfmt`, and `json`. With the default `auto` option, if the output is to a TTY device (e.g., a terminal), a more user-friendly console encoding is used. If the output is to files, the auto option uses the `logfmt` encoding. The `logfmt` and`json` options are useful for integration with external tools. **Default**: `auto` **Environment variable**: `INFLUXDB_LOGGING_FORMAT` #### level The log level to be emitted. Valid values are `error`, `warn`, `info` (default), and `debug`. Logs that are equal to, or above, the specified level will be emitted. **Default**: `info` **Environment variable**: `INFLUXDB_LOGGING_LEVEL` #### suppress-logo Suppresses the logo output that is printed when the program is started. The logo is always suppressed if `STDOUT` is not a TTY. **Default**: `false` **Environment variable**: `INFLUXDB_LOGGING_SUPPRESS_LOGO` ## Subscription settings ### [subscriber] The `[subscriber]` section controls how [Kapacitor](/kapacitor/v1/) will receive data. #### enabled Determines whether the subscriber service is enabled. To disable the subscriber service, set the value to `false`. **Default**: `true` **Environment variable**: `INFLUXDB_SUBSCRIBER_ENABLED` #### http-timeout The duration that an HTTP write to a subscriber runs until it times out. **Default**: `"30s"` **Environment variable**: `INFLUXDB_SUBSCRIBER_HTTP_TIMEOUT` #### insecure-skip-verify Determines whether to allow insecure HTTPS connections to subscribers. This is useful when testing with self-signed certificates. **Default**: `false` **Environment variable**: `INFLUXDB_SUBSCRIBER_INSECURE_SKIP_VERIFY` #### ca-certs The path to the PEM-encoded CA certs file. If the value is an empty string (`""`), the default system certs will be used. **Default**: `""` **Environment variable**: `INFLUXDB_SUBSCRIBER_CA_CERTS` #### write-concurrency The number of writer goroutines processing the write channel. **Default**: `40` **Environment variable**: `INFLUXDB_SUBSCRIBER_WRITE_CONCURRENCY` #### write-buffer-size The number of in-flight writes buffered in the write channel. **Default**: `1000` **Environment variable**: `INFLUXDB_SUBSCRIBER_WRITE_BUFFER_SIZE` #### total-buffer-bytes The total size in bytes allocated to buffering across all subscriptions. Each named subscription receives an even division of the total. **Default**: `0` **Environment variable**: `INFLUXDB_SUBSCRIBER_TOTAL_BUFFER_BYTES` ## Graphite settings ### [[graphite]] This section controls one or many listeners for Graphite data. For more information, see [Graphite protocol support in InfluxDB](/influxdb/v1/supported_protocols/graphite/). #### enabled Set to `true` to enable Graphite input. **Default**: `false` **Environment variable**: `INFLUXDB_GRAPHITE_0_ENABLED` #### database The name of the database that you want to write to. **Default**: `"graphite"` **Environment variable**: `INFLUXDB_GRAPHITE_0_DATABASE` #### retention-policy The relevant retention policy. An empty string is equivalent to the database’s`DEFAULT` retention policy. **Default**: `""` **Environment variable**: `INFLUXDB_GRAPHITE_0_RETENTION_POLICY` #### bind-address The default port. **Default**: `":2003"` **Environment variable**: `INFLUXDB_GRAPHITE_0_BIND_ADDRESS` #### protocol Set to `tcp` or `udp`. **Default**: `"tcp"` **Environment variable**: `INFLUXDB_GRAPHITE_PROTOCOL` #### consistency-level The number of nodes that must confirm the write. If the requirement is not met, the return value will be either `partial write` if some points in the batch fail or `write failure` if all points in the batch fail. For more information, see the Query String Parameters for Writes section in the[InfluxDB line protocol syntax reference](/influxdb/v1/write_protocols/write_syntax/). **Default**: `"one"` **Environment variable**: `INFLUXDB_GRAPHITE_CONSISTENCY_LEVEL` > [!Important] > The next three settings control how batching works. > You should have this enabled otherwise you could get dropped metrics or poor performance. > Batching will buffer points in memory if you have many coming in. #### batch-size The input will flush if this many points get buffered. **Default**: `5000` **Environment variable**: `INFLUXDB_GRAPHITE_BATCH_SIZE` #### batch-pending The number of batches that may be pending in memory. **Default**: `10` **Environment variable**: `INFLUXDB_GRAPHITE_BATCH_PENDING` #### batch-timeout The input will flush at least this often even if it hasn’t reached the configured batch-size. **Default**: `"1s"` **Environment variable**: `INFLUXDB_GRAPHITE_BATCH_TIMEOUT` #### udp-read-buffer UDP read buffer size, `0` means OS default. The UDP listener will fail if set above the OS maximum. **Default**: `0` **Environment variable**: `INFLUXDB_GRAPHITE_UDP_READ_BUFFER` #### separator This string joins multiple matching ‘measurement’ values, providing more control over the final measurement name. **Default**: `"."` **Environment variable**: `INFLUXDB_GRAPHITE_SEPARATOR` ## CollectD settings ### [[collectd]] The `[[collectd]]` settings control the listener for `collectd` data. For more information, see [CollectD protocol support in InfluxDB](/influxdb/v1/supported_protocols/collectd/). #### enabled Set to `true` to enable `collectd` writes. **Default**: `false` **Environment variable**: `INFLUXDB_COLLECTD_ENABLED` #### bind-address The port. **Default**: `":25826"` **Environment variable**: `INFLUXDB_COLLECTD_BIND_ADDRESS` #### database The name of the database that you want to write to. This defaults to `collectd`. **Default**: `"collectd"` **Environment variable**: `INFLUXDB_COLLECTD_DATABASE` #### retention-policy The relevant retention policy. An empty string is equivalent to the database’s`DEFAULT` retention policy. **Default**: `""` **Environment variable**: `INFLUXDB_COLLECTD_RETENTION_POLICY` #### typesdb The collectd service supports either scanning a directory for multiple types db files, or specifying a single db file. A sample `types.db` file can be found[here](https://github.com/collectd/collectd/blob/master/src/types.db). **Default**: `"/usr/local/share/collectd"` **Environment variable**: `INFLUXDB_COLLECTD_TYPESDB` #### security-level **Default**: `"none"` **Environment variable**: `INFLUXDB_COLLECTD_SECURITY_LEVEL` #### auth-file **Default**: `"/etc/collectd/auth_file"` **Environment variable**: `INFLUXDB_COLLECTD_AUTH_FILE` > [!Important] > The next three settings control how batching works. > You should have this enabled otherwise you could get dropped metrics or poor performance. > Batching will buffer points in memory if you have many coming in.\* #### batch-size The input will flush if this many points get buffered. **Default**: `5000` **Environment variable**: `INFLUXDB_COLLECTD_BATCH_SIZE` #### batch-pending The number of batches that may be pending in memory. **Default**: `10` **Environment variable**: `INFLUXDB_COLLECTD_BATCH_PENDING` #### batch-timeout The input will flush at least this often even if it hasn’t reached the configured batch-size. **Default**: `"10s"` **Environment variable**: `INFLUXDB_COLLECTD_BATCH_TIMEOUT` #### read-buffer UDP read buffer size, `0` means OS default. The UDP listener will fail if set above the OS maximum. **Default**: `0` **Environment variable**: `INFLUXDB_COLLECTD_READ_BUFFER` #### parse-multivalue-plugin When set to `split`, multi-value plugin data (e.g., `df free:5000,used:1000`) will be split into separate measurements (e.g., `(df_free, value=5000)``(df_used, value=1000)`). When set to `join`, multi-value plugin data will be stored as a single multi-value measurement (e.g., `(df, free=5000,used=1000)`). **Default**: `"split"` **Environment variable**: `INFLUXDB_COLLECTD_PARSE_MULTIVALUE_PLUGIN` ## OpenTSDB settings ### [[opentsdb]] Controls the listener for OpenTSDB data. For more information, see [OpenTSDB protocol support in InfluxDB](/influxdb/v1/supported_protocols/opentsdb/). #### enabled Set to `true` to enable openTSDB writes. **Default**: `false` **Environment variable**: `INFLUXDB_OPENTSDB_0_ENABLED` #### bind-address The default port. **Default**: `":4242"` **Environment variable**: `INFLUXDB_OPENTSDB_BIND_ADDRESS` #### database The name of the database that you want to write to. If the database does not exist, it will be created automatically when the input is initialized. **Default**: `"opentsdb"` **Environment variable**: `INFLUXDB_OPENTSDB_DATABASE` #### retention-policy The relevant retention policy. An empty string is equivalent to the database’s `DEFAULT` retention policy. **Default**: `""` **Environment variable**: `INFLUXDB_OPENTSDB_RETENTION_POLICY` #### consistency-level Sets the write consistency level: `any`, `one`, `quorum`, or `all` for writes. **Default**: `"one"` **Environment variable**: `INFLUXDB_OPENTSDB_CONSISTENCY_LEVEL` #### tls-enabled Enable TLS for openTSDB writes. **Default**: `false` **Environment variable**: `INFLUXDB_OPENTSDB_TLS_ENABLED` #### certificate The path to the SSL certificate file to use when TLS is enabled. **Default**: `"/etc/ssl/influxdb.pem"` **Environment variable**: `INFLUXDB_OPENTSDB_CERTIFICATE` #### log-point-errors Log an error for every malformed point. **Default**: `true` **Environment variable**: `INFLUXDB_OPENTSDB_0_LOG_POINT_ERRORS` > [!Important] > The next three settings control how batching works. > You should have this enabled otherwise you could get dropped metrics or poor performance. > Only points metrics received over the telnet protocol undergo batching.\* #### batch-size The input will flush if this many points get buffered. **Default**: `1000` **Environment variable**: `INFLUXDB_OPENTSDB_BATCH_SIZE` #### batch-pending The number of batches that may be pending in memory. **Default**: `5` **Environment variable**: `INFLUXDB_OPENTSDB_BATCH_PENDING` #### batch-timeout The input will flush at least this often even if it hasn’t reached the configured batch-size. **Default**: `"1s"` **Environment variable**: `INFLUXDB_OPENTSDB_BATCH_TIMEOUT` ## UDP settings ### [[udp]] The `[[udp]]` settings control the listeners for InfluxDB line protocol data using UDP. For more information, see [UDP protocol support in InfluxDB](/influxdb/v1/supported_protocols/udp/). #### enabled Determines whether UDP listeners are enabled. To enable writes over UDP, set the value to `true`. **Default**: `false` **Environment variable**: `INFLUXDB_UDP_ENABLED` #### bind-address An empty string is equivalent to `0.0.0.0`. **Default**: `":8089"` **Environment variable**: `INFLUXDB_UDP_BIND_ADDRESS` #### database The name of the database that you want to write to. **Default**: `"udp"` **Environment variable**: `INFLUXDB_UDP_DATABASE` #### retention-policy The relevant retention policy for your data. An empty string is equivalent to the database’s `DEFAULT` retention policy. **Default**: `""` **Environment variable**: `INFLUXDB_UDP_RETENTION_POLICY` > [!Important] > The next three settings control how batching works. > You should have this enabled otherwise you could get dropped metrics or poor performance. > Batching will buffer points in memory if you have many coming in.\* #### batch-size The input will flush if this many points get buffered. **Default**: `5000` **Environment variable**: `INFLUXDB_UDP_0_BATCH_SIZE` #### batch-pending The number of batches that may be pending in memory. **Default**: `10` **Environment variable**: `INFLUXDB_UDP_0_BATCH_PENDING` #### batch-timeout The input will flush at least this often even if it hasn’t reached the configured batch-size. **Default**: `"1s"` **Environment variable**: `INFLUXDB_UDP_BATCH_TIMEOUT` #### read-buffer UDP read buffer size, `0` means OS default. The UDP listener will fail if set above the OS maximum. **Default**: `0` **Environment variable**: `INFLUXDB_UDP_BATCH_SIZE` #### precision [Time precision](/influxdb/v1/query_language/spec/#durations) used when decoding time values. Defaults to `nanoseconds`, which is the default of the database. **Default**: `""` **Environment variable**: `INFLUXDB_UDP_PRECISION` ## Continuous queries settings ### [continuous\_queries] The `[continuous_queries]` settings control how[continuous queries (CQs)](/influxdb/v1/concepts/glossary/#continuous-query-cq)run within InfluxDB. Continuous queries are automated batches of queries that execute over recent time intervals. InfluxDB executes one auto-generated query per `GROUP BY time()` interval. #### enabled Set to `false` to disable CQs. **Default**: `true` **Environment variable**: `INFLUXDB_CONTINUOUS_QUERIES_ENABLED` #### log-enabled Set to `false` to disable logging for CQ events. **Default**: `true` **Environment variable**: `INFLUXDB_CONTINUOUS_QUERIES_LOG_ENABLED` #### query-stats-enabled When set to `true`, continuous query execution statistics are written to the default monitor store. **Default**: `false` **Environment variable**: `INFLUXDB_CONTINUOUS_QUERIES_QUERY_STATS_ENABLED` #### run-interval The interval at which InfluxDB checks to see if a CQ needs to run. Set this option to the lowest interval at which your CQs run. For example, if your most frequent CQ runs every minute, set `run-interval` to `1m`. **Default**: `"1s"` **Environment variable**: `INFLUXDB_CONTINUOUS_QUERIES_RUN_INTERVAL` ## Transport Layer Security (TLS) settings ### [tls] Global configuration settings for Transport Layer Security (TLS) in InfluxDB. For more information, see [Enabling HTTPS](/influxdb/v1/administration/https_setup/). If the TLS configuration settings is not specified, InfluxDB supports all of the cipher suite IDs listed and all TLS versions implemented in the [Constants section of the Go `crypto/tls` package documentation](https://golang.org/pkg/crypto/tls/#pkg-constants), depending on the version of Go used to build InfluxDB. Use the `SHOW DIAGNOSTICS` command to see the version of Go used to build InfluxDB. ### Recommended server configuration for “modern compatibility” InfluxData recommends configuring your InfluxDB server’s TLS settings for “modern compatibility.” This provides a higher level of security and assumes that backward compatibility is not required. Our recommended TLS configuration settings for `ciphers`, `min-version`, and `max-version` are based on Mozilla’s “modern compatibility” TLS server configuration described in[Security/Server Side TLS](https://wiki.mozilla.org/Security/Server_Side_TLS). InfluxData’s recommended TLS settings for “modern compatibility” are specified in the following configuration settings example: ```toml [tls] ciphers = [ "TLS_AES_128_GCM_SHA256", "TLS_AES_256_GCM_SHA384", "TLS_CHACHA20_POLY1305_SHA256" ] min-version = "tls1.3" max-version = "tls1.3" ``` > [!Important] > The order of the cipher suite IDs in the `ciphers` setting determines which > algorithms are selected by priority. The TLS `min-version` and the`max-version` settings restrict support to TLS 1.3. #### ciphers Specifies the set of cipher suite IDs to negotiate. If not specified, `ciphers`supports all existing cipher suite IDs listed in the Go `crypto/tls` package. This is consistent with the behavior within previous releases. In this example, only the two specified cipher suite IDs would be supported. **Default**: `["TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305", "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"]` **Environment variable**: `INFLUXDB_TLS_CIPHERS` #### min-version Minimum version of the TLS protocol that will be negotiated. Valid values include: `tls1.0`, `tls1.1`, `tls1.2`, and `tls1.3`. If not specified,`min-version` is the minimum TLS version specified in the[Go `crypto/tls` package](https://golang.org/pkg/crypto/tls/#pkg-constants). In this example, `tls1.0` specifies the minimum version as TLS 1.0, which is consistent with the behavior of previous InfluxDB releases. **Default**: `"tls1.0"` **Environment variable**: `INFLUXDB_TLS_MIN_VERSION` #### max-version The maximum version of the TLS protocol that will be negotiated. Valid values include: `tls1.0`, `tls1.1`, `tls1.2`, and `tls1.3`. If not specified,`max-version` is the maximum TLS version specified in the[Go `crypto/tls` package](https://golang.org/pkg/crypto/tls/#pkg-constants). In this example, `tls1.3` specifies the maximum version as TLS 1.3, which is consistent with the behavior of previous InfluxDB releases. **Default**: `tls1.3`**Environment variable**: `INFLUXDB_TLS_MAX_VERSION` #### advanced-expiration Sets how far in advance to log warnings about TLS certificate expiration. **Default**: `5d`**Environment variable**: `INFLUXDB_TLS_ADVANCED_EXPIRATION` --- # Enable HTTPS with InfluxDB Source: https://docs.influxdata.com/influxdb/v1/administration/https_setup/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Enabling HTTPS encrypts the communication between clients and the InfluxDB server. When configured with a signed certificate, HTTPS can also verify the authenticity of the InfluxDB server to connecting clients. InfluxData [strongly recommends](/influxdb/v1/administration/security/) enabling HTTPS, especially if you plan on sending requests to InfluxDB over a network. ## Requirements To enable HTTPS with InfluxDB, you’ll need an existing or new InfluxDB instance and a Transport Layer Security (TLS) certificate (also known as a Secured Sockets Layer (SSL) certificate). InfluxDB supports three types of TLS certificates: * **Single domain certificates signed by a Certificate Authority** Single domain certificates provide cryptographic security to HTTPS requests and allow clients to verify the identity of the InfluxDB server. With this certificate option, every InfluxDB instance requires a unique single domain certificate. * **Wildcard certificates signed by a Certificate Authority** Wildcard certificates provide cryptographic security to HTTPS requests and allow clients to verify the identity of the InfluxDB server. Wildcard certificates can be used across multiple InfluxDB instances on different servers. * **Self-signed certificates** Self-signed certificates are *not* signed by a Certificate Authority (CA).[Generate a self-signed certificate](#step-1-generate-a-self-signed-certificate) on your own machine. Unlike CA-signed certificates, self-signed certificates only provide cryptographic security to HTTPS requests. They do not allow clients to verify the identity of the InfluxDB server. With this certificate option, every InfluxDB instance requires a unique self-signed certificate. Regardless of your certificate’s type, InfluxDB supports certificates composed of a private key file (`.key`) and a signed certificate file (`.crt`) file pair, as well as certificates that combine the private key file and the signed certificate file into a single bundled file (`.pem`). The following two sections outline how to set up HTTPS with InfluxDB [using a CA-signed certificate](#set-up-https-with-a-ca-certificate) and [using a self-signed certificate](#set-up-https-with-a-self-signed-certificate)on Ubuntu 16.04. Steps may vary for other operating systems. ## Set up HTTPS with a CA certificate #### Step 1: Install the certificate Place the private key file (`.key`) and the signed certificate file (`.crt`) or the single bundled file (`.pem`) in the `/etc/ssl` directory. #### Step 2: Set certificate file permissions Users running InfluxDB must have read permissions on the TLS certificate. > > > ***Note***: You may opt to set up multiple users, groups, and permissions. Ultimately, make sure all users running InfluxDB have read permissions for the TLS certificate. > > Run the following command to give InfluxDB read and write permissions on the certificate files. ```bash sudo chown influxdb:influxdb /etc/ssl/ sudo chmod 644 /etc/ssl/ sudo chmod 600 /etc/ssl/ ``` #### Step 3: Review the TLS configuration settings By default, InfluxDB supports the values for TLS `ciphers`, `min-version`, and `max-version` listed in the [Constants section of the Go `crypto/tls` package documentation](https://golang.org/pkg/crypto/tls/#pkg-constants) and depends on the version of Go used to build InfluxDB. You can configure InfluxDB to support a restricted list of TLS cipher suite IDs and versions. For more information, see [Transport Layer Security (TLS) configuration settings](/influxdb/v1/administration/config#transport-layer-security-tls-settings). #### Step 4: Enable HTTPS in the InfluxDB configuration file HTTPS is disabled by default. Enable HTTPS in the `[http]` section of the configuration file (`/etc/influxdb/influxdb.conf`) by setting: * `https-enabled` to `true` * `https-certificate` to `/etc/ssl/.crt` (or to `/etc/ssl/.pem`) * `https-private-key` to `/etc/ssl/.key` (or to `/etc/ssl/.pem`) ```toml [http] [...] # Determines whether HTTPS is enabled. https-enabled = true [...] # The SSL certificate to use when HTTPS is enabled. https-certificate = ".pem" # Use a separate private key location. https-private-key = ".pem" ``` #### Step 5: Restart the InfluxDB service Restart the InfluxDB process for the configuration changes to take effect: ```bash sudo systemctl restart influxdb ``` #### Step 6: Verify the HTTPS setup Verify that HTTPS is working by connecting to InfluxDB with the [CLI tool](/influxdb/v1/tools/shell/): ```bash influx -ssl -host .com ``` A successful connection returns the following: ```bash Connected to https://.com:8086 version 1.x.x InfluxDB shell version: 1.x.x > ``` That’s it! You’ve successfully set up HTTPS with InfluxDB. ## Set up HTTPS with a self-signed certificate #### Step 1: Generate a self-signed certificate The following command generates a private key file (`.key`) and a self-signed certificate file (`.crt`) which remain valid for the specified `NUMBER_OF_DAYS`. It outputs those files to the InfluxDB database’s default certificate file paths and gives them the required permissions. ```bash sudo openssl req -x509 -nodes -newkey rsa:2048 -keyout /etc/ssl/influxdb-selfsigned.key -out /etc/ssl/influxdb-selfsigned.crt -days ``` When you execute the command, it will prompt you for more information. You can choose to fill out that information or leave it blank; both actions generate valid certificate files. Run the following command to give InfluxDB read and write permissions on the certificate. ```bash chown influxdb:influxdb /etc/ssl/influxdb-selfsigned.* ``` #### Step 2: Review the TLS configuration settings By default, InfluxDB supports the values for TLS `ciphers`, `min-version`, and `max-version` listed in the [Constants section of the Go `crypto/tls` package documentation](https://golang.org/pkg/crypto/tls/#pkg-constants) and depends on the version of Go used to build InfluxDB. You can configure InfluxDB to support a restricted list of TLS cipher suite IDs and versions. For more information, see [Transport Layer Security (TLS) settings `[tls]`](/influxdb/v1/administration/config#transport-layer-security-tls-settings). #### Step 3: Enable HTTPS in the configuration file HTTPS is disabled by default. Enable HTTPS in the `[http]` section of the configuration file (`/etc/influxdb/influxdb.conf`) by setting: * `https-enabled` to `true` * `https-certificate` to `/etc/ssl/influxdb-selfsigned.crt` * `https-private-key` to `/etc/ssl/influxdb-selfsigned.key` ``` [http] [...] # Determines whether HTTPS is enabled. https-enabled = true [...] # The TLS or SSL certificate to use when HTTPS is enabled. https-certificate = "/etc/ssl/influxdb-selfsigned.crt" # Use a separate private key location. https-private-key = "/etc/ssl/influxdb-selfsigned.key" ``` > > > If setting up HTTPS for [InfluxDB Enterprise](/enterprise_influxdb), you also need to configure insecure TLS connections between both meta and data nodes in your cluster. > Instructions are provided in the [InfluxDB Enterprise HTTPS Setup guide](/enterprise_influxdb/v1/guides/https_setup/#set-up-https-with-a-self-signed-certificate). > > #### Step 4: Restart InfluxDB Restart the InfluxDB process for the configuration changes to take effect: ```bash sudo systemctl restart influxdb ``` #### Step 5: Verify the HTTPS setup Verify that HTTPS is working by connecting to InfluxDB with the [CLI tool](/influxdb/v1/tools/shell/): ```bash influx -ssl -unsafeSsl -host .com ``` A successful connection returns the following: ```bash Connected to https://.com:8086 version 1.x.x InfluxDB shell version: 1.x.x > ``` That’s it! You’ve successfully set up HTTPS with InfluxDB. ## Connect Telegraf to a secured InfluxDB instance Connecting [Telegraf](/telegraf/v1/) to an InfluxDB instance that’s using HTTPS requires some additional steps. In the Telegraf configuration file (`/etc/telegraf/telegraf.conf`), edit the `urls`setting to indicate `https` instead of `http` and change `localhost` to the relevant domain name. If you’re using a self-signed certificate, uncomment the `insecure_skip_verify`setting and set it to `true`. ``` ############################################################################### # OUTPUT PLUGINS # ############################################################################### # Configuration for InfluxDB server to send metrics to [[outputs.influxdb]] ## The full HTTP or UDP endpoint URL for your InfluxDB instance. ## Multiple urls can be specified as part of the same cluster, ## this means that only ONE of the urls will be written to each interval. # urls = ["udp://localhost:8089"] # UDP endpoint example urls = ["https://.com:8086"] # ... other output options elided ... ## Optional SSL Config # ... other SSL options elided ... insecure_skip_verify = true # <-- Update only if you're using a self-signed certificate ``` Next, restart Telegraf and you’re all set! --- # Identify InfluxDB OSS v1 version Source: https://docs.influxdata.com/influxdb/v1/administration/identify-version/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Identifying which InfluxDB product and version you’re using is essential for accessing the correct documentation, troubleshooting issues, and planning upgrades. > [!Note] > #### Identify your InfluxDB version > > If you are unsure which InfluxDB product you are using, use our interactive version detector to help identify it: > > [Detect my InfluxDB version](#) > *(Interactive feature in HTML: Opens version detector modal)* ## Quick detection methods ### InfluxDB OSS v1 / Enterprise v1 detection Check the version using `influxd`: ```bash influxd version ``` Check the `/ping` endpoint headers: ```bash curl -i http://localhost:8086/ping ``` Look for the `x-influxdb-version` header (for example, `1.11.7`). For Enterprise v1, the `x-influxdb-build` header will show `Enterprise`. ## Understanding InfluxDB products For a complete comparison of InfluxDB versions and deployment options, see the [InfluxDB platform overview](/platform/). ## Troubleshooting ### Can’t access your InfluxDB instance? If you can’t directly access your InfluxDB instance: 1. Use the [interactive version detector](#identify-your-influxdb-version) above or [Ask InfluxData AI](#) 2. Answer questions about your setup: * License type (paid or free) * Hosting model (cloud or self-hosted) * Server age * Query language preferences 3. Get product recommendations based on your answers ### Still not sure? If you’re still uncertain, see the [Support and feedback](#bug-reports-and-feedback) options. #### Related --- # Log and trace with InfluxDB Source: https://docs.influxdata.com/influxdb/v1/administration/logs/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. **Content** * [Logging locations](#logging-locations) * [HTTP access logging](#http-access-logging) * [Structured logging](#structured-logging) * [Tracing](#tracing) ## Logging locations InfluxDB writes log output, by default, to `stderr`. Depending on your use case, this log information can be written to another location. ### Running InfluxDB directly When you run InfluxDB directly, using `influxd`, all logs are written to `stderr`. You can also redirect the log output, as you would any output to `stderr`, like in this example: ``` influxd 2>$HOME/my_log_file ``` ### Launched as a service [systemd](#systemd-2)[sysinit](#sysinit) #### systemd Most Linux systems direct logs to the `systemd` journal. To access these logs, use this command: ```sh sudo journalctl -u influxdb.service ``` For more information, see the [`journald.conf` manual page](https://www.freedesktop.org/software/systemd/man/journald.conf.html). #### sysvinit On Linux systems not using systemd, InfluxDB writes all log data and `stderr` to `/var/log/influxdb/influxd.log`. You can override this location by setting the environment variable `STDERR` in a start-up script at `/etc/default/influxdb`. (If this file doesn’t exist, you need to create it.) For example, if `/etc/default/influxdb` contains: ```sh STDERR=/dev/null ``` all log data is discarded. Likewise, you can direct output to `stdout` by setting `STDOUT` in the same file.`stdout` is sent to `/dev/null` by default when InfluxDB is launched as a service. InfluxDB must be restarted to use any changes to `/etc/default/influxdb`. > > > #### Log location on macOS #### > > > > On macOs, InfluxDB stores logs at `/usr/local/var/log/influxdb.log` by default. > > ### Using logrotate You can use [logrotate](http://manpages.ubuntu.com/manpages/cosmic/en/man8/logrotate.8.html) to rotate the log files generated by InfluxDB on systems where logs are written to flat files. If using the package install on a `sysvinit` system, the config file for logrotate is installed in `/etc/logrotate.d`. You can view the file [here](https://github.com/influxdb/influxdb/blob/1.8/scripts/logrotate). ## HTTP access logging Use the HTTP access log to log HTTP request traffic separately from the other InfluxDB log output. ### HTTP access log format Find details about your GET and POST requests in the HTTP access log. Useful to help identify bad queries and improve performance. Refer to the table below for details included in the log. The following examples show the HTTP access log format: #### GET request format ``` [httpd] 172.13.8.13,172.39.5.169 - - [21/Jul/2019:03:01:27 +0000] "GET /query?db=metrics&q=SELECT+MEAN%28value%29+as+average_cpu%2C+MAX%28value%29+as+peak_cpu+FROM+%22foo.load%22+WHERE+time+%3E%3D+now%28%29+-+1m+AND+org_id+%21%3D+%27%27+AND+group_id+%21%3D+%27%27+GROUP+BY+org_id%2Cgroup_id HTTP/1.0" 200 11450 "-" "Baz Service" d6ca5a13-at63-11o9-8942-000000000000 9337349 ``` #### POST request format ``` [httpd] 127.0.0.1 - - [19/Mar/2021:14:09:51 -0700] "POST /query?pretty=true HTTP/1.1 {'db': 'TestData', 'BadDataBase'}, {'q': 'SELECT * FROM a'}, {'p': '[REDACTED]'}" 200 1217 "-" "curl/7.68.0" 71dcf8eb-88f7-11eb-8003-002b67685b12 1300 ``` > [!Note] > Note: One or more password `p` values are replaced by a single `[REDACTED]`. #### HTTP access log | HTTP access log | Example | |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------| | Host | `172.13.8.13,172.39.5.169` | | Time of log event | `[21/Jul/2019:03:01:27 +0000]` | | Request method | `GET` or `POST` | | Username | `user` | | HTTP API call being made\* |`/query?db=metrics%26q=SELECT%20used_percent%20FROM%20%22telegraf.autogen.mem%22%20WHERE%20time%20%3E=%20now()%20-%201m%20`| | Request protocol | `HTTP/1.0` | | HTTP response code | `200` | | Size of response in bytes | `11450` | | Referrer | `-` | | User agent | `Baz Service` | | Request ID | `d4ca9a10-ab63-11e9-8942-000000000000` | | Response time in microseconds | `9357049` | |\* This field shows the database being accessed and the query being run. For more details, see [InfluxDB API reference](/influxdb/v1/api/). Note that this field is URL-encoded.| | ### Redirecting HTTP access logging When HTTP request logging is enabled, the HTTP logs are intermingled by default with internal InfluxDB logging. By redirecting the HTTP request log entries to a separate file, both log files are easier to read, monitor, and debug. **To redirect HTTP request logging:** Locate the `[http]` section of your InfluxDB configuration file and set the `access-log-path` option to specify the path where HTTP log entries should be written. **Notes:** * If `influxd` is unable to access the specified path, it will log an error and fall back to writing the request log to `stderr`. * The `[httpd]` prefix is stripped when HTTP request logging is redirected to a separate file, allowing access log parsing tools (like [lnav](https://lnav.org)) to render the files without additional modification. * To rotate the HTTP request log file, use the `copytruncate` method of `logrotate` or similar to leave the original file in place. ## Structured logging Structured logging enables machine-readable and more developer-friendly log output formats. The two structured log formats, `logfmt` and `json`, provide easier filtering and searching with external tools and simplifies integration of InfluxDB logs with Splunk, Papertrail, Elasticsearch, and other third party tools. The InfluxDB logging configuration options (in the `[logging]` section) now include the following options: * `format`: `auto` (default) | `logfmt` | `json` * `level`: `error` | `warn` | `info` (default) | `debug` * `suppress-logo`: `false` (default) | `true` For details on these logging configuration options and their corresponding environment variables, see [Logging options](/influxdb/v1/administration/config#logging-settings) in the configuration file documentation. ### Logging formats Three logging `format` options are available: `auto`, `logfmt`, and `json`. The default logging format setting, `format = "auto"`, lets InfluxDB automatically manage the log encoding format: * When logging to a file, the `logfmt` is used * When logging to a terminal (or other TTY device), a user-friendly console format is used. The `json` format is available when specified. ### Examples of log output: **Logfmt** ``` ts=2018-02-20T22:48:11.291815Z lvl=info msg="InfluxDB starting" version=unknown branch=unknown commit=unknown ts=2018-02-20T22:48:11.291858Z lvl=info msg="Go runtime" version=go1.10 maxprocs=8 ts=2018-02-20T22:48:11.291875Z lvl=info msg="Loading configuration file" path=/Users/user_name/.influxdb/influxdb.conf ``` **JSON** ``` {"lvl":"info","ts":"2018-02-20T22:46:35Z","msg":"InfluxDB starting, version unknown, branch unknown, commit unknown"} {"lvl":"info","ts":"2018-02-20T22:46:35Z","msg":"Go version go1.10, GOMAXPROCS set to 8"} {"lvl":"info","ts":"2018-02-20T22:46:35Z","msg":"Using configuration at: /Users/user_name/.influxdb/influxdb.conf"} ``` **Console/TTY** ``` 2018-02-20T22:55:34.246997Z info InfluxDB starting {"version": "unknown", "branch": "unknown", "commit": "unknown"} 2018-02-20T22:55:34.247042Z info Go runtime {"version": "go1.10", "maxprocs": 8} 2018-02-20T22:55:34.247059Z info Loading configuration file {"path": "/Users/user_name/.influxdb/influxdb.conf"} ``` ### Logging levels The `level` option sets the log level to be emitted. Valid logging level settings are `error`, `warn`, `info`(default), and `debug`. Logs that are equal to, or above, the specified level will be emitted. ### Logo suppression The `suppress-logo` option can be used to suppress the logo output that is printed when the program is started. The logo is always suppressed if `STDOUT` is not a TTY. ## Tracing Logging has been enhanced to provide tracing of important InfluxDB operations. Tracing is useful for error reporting and discovering performance bottlenecks. ### Logging keys used in tracing #### Tracing identifier key The `trace_id` key specifies a unique identifier for a specific instance of a trace. You can use this key to filter and correlate all related log entries for an operation. All operation traces include consistent starting and ending log entries, with the same message (`msg`) describing the operation (e.g., “TSM compaction”), but adding the appropriate `op_event` context (either `start` or `end`). For an example, see [Finding all trace log entries for an InfluxDB operation](#finding-all-trace-log-entries-for-an-influxdb-operation). **Example:** `trace_id=06R0P94G000` #### Operation keys The following operation keys identify an operation’s name, the start and end timestamps, and the elapsed execution time. ##### `op_name` Unique identifier for an operation. You can filter on all operations of a specific name. **Example:** `op_name=tsm1_compact_group` ##### `op_event` Specifies the start and end of an event. The two possible values, `(start)` or `(end)`, are used to indicate when an operation started or ended. For example, you can grep by values in `op_name` AND `op_event` to find all starting operation log entries. For an example of this, see [Finding all starting log entries](#finding-all-starting-operation-log-entries). **Example:** `op_event=start` ##### `op_elapsed` Amount of time the operation spent executing. Logged with the ending trace log entry. Time unit displayed depends on how much time has elapsed – if it was seconds, it will be suffixed with `s`. Valid time units are `ns`, `µs`, `ms`, and `s`. **Example:** `op_elapsed=0.352ms` #### Log identifier context key The log identifier key (`log_id`) lets you easily identify *every* log entry for a single execution of an `influxd` process. There are other ways a log file could be split by a single execution, but the consistent `log_id` eases the searching of log aggregation services. **Example:** `log_id=06QknqtW000` #### Database context keys `db_instance`: Database name `db_rp`: Retention policy name `db_shard_id`: Shard identifier `db_shard_group` Shard group identifier ### Tooling Here are a couple of popular tools available for processing and filtering log files output in `logfmt` or `json` formats. #### hutils The [hutils](https://blog.heroku.com/hutils-explore-your-structured-data-logs), provided by Heroku, is a collection of command line utilities for working with logs with `logfmt` encoding, including: * `lcut`: Extracts values from a `logfmt` trace based on a specified field name. * `lfmt`: Prettifies `logfmt` lines as they emerge from a stream, and highlights their key sections. * `ltap`: Accesses messages from log providers in a consistent way to allow easy parsing by other utilities that operate on `logfmt` traces. * `lviz`: Visualizes `logfmt` output by building a tree out of a dataset combining common sets of key-value pairs into shared parent nodes. #### lnav (Log File Navigator) The [lnav (Log File Navigator)](http://lnav.org) is an advanced log file viewer useful for watching and analyzing your log files from a terminal. The lnav viewer provides a single log view, automatic log format detection, filtering, timeline view, pretty-print view, and querying logs using SQL. ### Operations The following operations, listed by their operation name (`op_name`) are traced in InfluxDB internal logs and available for use without changes in logging level. #### Initial opening of data files The `tsdb_open` operation traces include all events related to the initial opening of the `tsdb_store`. #### Retention policy shard deletions The `retention.delete_check` operation includes all shard deletions related to the retention policy. #### TSM snapshotting in-memory cache to disk The `tsm1_cache_snapshot` operation represents the snapshotting of the TSM in-memory cache to disk. #### TSM compaction strategies The `tsm1_compact_group` operation includes all trace log entries related to TSM compaction strategies and displays the related TSM compaction strategy keys: * `tsm1_strategy`: `level` | `full` * `tsm1_level`: `1` | `2` | `3` * `tsm1_optimize`: `true` | `false` #### Series file compactions The `series_partition_compaction` operation includes all trace log entries related to series file compactions. #### Continuous query execution (if logging enabled) The `continuous_querier_execute` operation includes all continuous query executions, if logging is enabled. #### TSI log file compaction The `tsi1_compact_log_file` #### TSI level compaction The `tsi1_compact_to_level` operation includes all trace log entries for TSI level compactions. ### Tracing examples #### Finding all trace log entries for an InfluxDB operation In the example below, you can see the log entries for all trace operations related to a “TSM compaction” process. Note that the initial entry shows the message “TSM compaction (start)” and the final entry displays the message “TSM compaction (end)”. [Note: Log entries were grepped using the `trace_id` value and then the specified key values were displayed using `lcut` (an hutils tool).] ``` $ grep "06QW92x0000" influxd.log | lcut ts lvl msg strategy level 2018-02-21T20:18:56.880065Z info TSM compaction (start) full 2018-02-21T20:18:56.880162Z info Beginning compaction full 2018-02-21T20:18:56.880185Z info Compacting file full 2018-02-21T20:18:56.880211Z info Compacting file full 2018-02-21T20:18:56.880226Z info Compacting file full 2018-02-21T20:18:56.880254Z info Compacting file full 2018-02-21T20:19:03.928640Z info Compacted file full 2018-02-21T20:19:03.928687Z info Finished compacting files full 2018-02-21T20:19:03.928707Z info TSM compaction (end) full ``` #### Finding all starting operation log entries To find all starting operation log entries, you can grep by values in `op_name` AND `op_event`. In the following example, the grep returned 101 entries, so the result below only displays the first entry. In the example result entry, the timestamp, level, strategy, trace\_id, op\_name, and op\_event values are included. ``` $ grep -F 'op_name=tsm1_compact_group' influxd.log | grep -F 'op_event=start' ts=2018-02-21T20:16:16.709953Z lvl=info msg="TSM compaction" log_id=06QVNNCG000 engine=tsm1 level=1 strategy=level trace_id=06QV~HHG000 op_name=tsm1_compact_group op_event=start ... ``` Using the `lcut` utility (in hutils), the following command uses the previous `grep` command, but adds an `lcut` command to only display the keys and their values for keys that are not identical in all of the entries. The following example includes 19 examples of unique log entries displaying selected keys: `ts`, `strategy`, `level`, and `trace_id`. ``` $ grep -F 'op_name=tsm1_compact_group' influxd.log | grep -F 'op_event=start' | lcut ts strategy level trace_id | sort -u 2018-02-21T20:16:16.709953Z level 1 06QV~HHG000 2018-02-21T20:16:40.707452Z level 1 06QW0k0l000 2018-02-21T20:17:04.711519Z level 1 06QW2Cml000 2018-02-21T20:17:05.708227Z level 2 06QW2Gg0000 2018-02-21T20:17:29.707245Z level 1 06QW3jQl000 2018-02-21T20:17:53.711948Z level 1 06QW5CBl000 2018-02-21T20:18:17.711688Z level 1 06QW6ewl000 2018-02-21T20:18:56.880065Z full 06QW92x0000 2018-02-21T20:20:46.202368Z level 3 06QWFizW000 2018-02-21T20:21:25.292557Z level 1 06QWI6g0000 2018-02-21T20:21:49.294272Z level 1 06QWJ_RW000 2018-02-21T20:22:13.292489Z level 1 06QWL2B0000 2018-02-21T20:22:37.292431Z level 1 06QWMVw0000 2018-02-21T20:22:38.293320Z level 2 06QWMZqG000 2018-02-21T20:23:01.293690Z level 1 06QWNygG000 2018-02-21T20:23:25.292956Z level 1 06QWPRR0000 2018-02-21T20:24:33.291664Z full 06QWTa2l000 2018-02-21T21:12:08.017055Z full 06QZBpKG000 2018-02-21T21:12:08.478200Z full 06QZBr7W000 ``` | HTTP access log | Example | | --- | --- | | HTTP access log | Example | | Host | 172.13.8.13,172.39.5.169 | | Time of log event | [21/Jul/2019:03:01:27 +0000] | | Request method | GET or POST | | Username | user | | HTTP API call being made* | /query?db=metrics%26q=SELECT%20used_percent%20FROM%20%22telegraf.autogen.mem%22%20WHERE%20time%20%3E=%20now()%20-%201m%20 | | Request protocol | HTTP/1.0 | | HTTP response code | 200 | | Size of response in bytes | 11450 | | Referrer | - | | User agent | Baz Service | | Request ID | d4ca9a10-ab63-11e9-8942-000000000000 | | Response time in microseconds | 9357049 | | * This field shows the database being accessed and the query being run. For more details, see InfluxDB API reference . Note that this field is URL-encoded. | | --- # Monitor InfluxDB Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Monitoring is the act of observing changes in data over time. “Monitoring” covers two different things: * **Monitor InfluxDB itself**—track the health and performance of your InfluxDB server. * **Monitor other systems with InfluxDB**—use InfluxDB and the wider TICK stack (Telegraf, Chronograf, Kapacitor) as your monitoring backend for infrastructure and applications. ## Monitor InfluxDB itself * [`_internal` measurements and fields](/influxdb/v1/administration/monitor/measurements-internal/) * [SHOW STATS](/influxdb/v1/administration/monitor/show-stats/) * [SHOW DIAGNOSTICS](/influxdb/v1/administration/monitor/show-diagnostics/) * [Performance metrics commands and the `/metrics` endpoint](/influxdb/v1/administration/monitor/performance-metrics-commands/) * [Monitoring dashboards](/influxdb/v1/administration/monitor/monitoring-dashboards/) * [Internal vs. external monitoring](/influxdb/v1/administration/monitor/internal-vs-external/) and [watcher of watchers setup](/influxdb/v1/administration/monitor/external-monitor-setup/) * [Kapacitor measurements](/influxdb/v1/administration/monitor/kapacitor-measurements/) ## Monitor other systems with InfluxDB * [Monitor Kubernetes](/influxdb/v1/administration/monitor/monitor-kubernetes/) --- # Configure a watcher of watchers system to monitor InfluxDB 1.x servers Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/external-monitor-setup/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The flexibility and portability of InfluxData’s TICK stack make it easy to use in different monitoring solutions, including monitoring the TICK stack with another TICK stack. This guide walks through setting up an external TICK stack monitor to which important metrics are sent and monitored. The following terms are used throughout this guide: * **Primary** - The *monitored* TICK stack or cluster for which uptime is most important. * **Monitor** - The *monitoring* TICK stack to which monitoring data is sent and processed. *This guide assumes a primary node or cluster is already running.* ## Install a monitor Install a separate TICK stack to act as your monitor. Your monitor should be on hardware separate from your primary cluster. Installation instructions for the TICK stack are provided in the [installation guides](/platform/install-and-deploy/). > [!Note] > In order for your monitor to receive data from your primary cluster, the primary > must be able to connect to your monitor’s API endpoint via HTTP or UDP. ## Install Telegraf on each node [Install the `telegraf` agent](/telegraf/v1/introduction/installation/#installation)on each node in your primary InfluxDB cluster you would like to monitor. ### Send data collected by Telegraf to your monitor [Generate a Telegraf configuration file](/telegraf/v1/introduction/installation/#configuration)and modify the InfluxDB output `url` setting to include the URL of your monitor’s InfluxDB API endpoint. ***telegraf.conf*** ```toml # ... [[outputs.influxdb]] ## The full HTTP or UDP URL for your InfluxDB instance. urls = ["http://monitor-url.com:8086"] # ... ``` ### Configure Telegraf input plugins By default, Telegraf is configured to collect the following system metrics from the host machine: * CPU * Disk * Disk IO * Memory * Processes * Swap * System (load, number of CPUs, number of users, uptime, etc.) Use other [Telegraf input plugins](/telegraf/v1/plugins/#input-plugins) to collect a variety of metrics. #### Monitor InfluxDB performance metrics To monitor the internal performance of InfluxDB, enable the InfluxDB input plugin in the Telegraf configuration files used to run Telegraf **on InfluxDB instances**. The InfluxDB input plugin pulls [InfluxDB internal metrics](/influxdb/v1/administration/monitor/measurements-internal/)from the local InfluxDB `/debug/vars` endpoint. ```toml # ... [[inputs.influxdb]] # ... ## Multiple URLs from which to read InfluxDB-formatted JSON ## Default is "http://localhost:8086/debug/vars". urls = [ "http://localhost:8086/debug/vars" ] # ... ``` #### Monitor Kapacitor performance metrics To monitor the internal performance of Kapacitor, enable the Kapacitor input plugin in the Telegraf configuration files used to run Telegraf **on Kapacitor instances**. The Kapacitor input plugin pulls [Kapacitor internal metrics](/influxdb/v1/administration/monitor/kapacitor-measurements/)from the local Kapacitor `/debug/vars` endpoint. ```toml # ... [[inputs.kapacitor]] # ... ## Multiple URLs from which to read Kapacitor-formatted JSON ## Default is "http://localhost:9092/kapacitor/v1/debug/vars". urls = [ "http://localhost:9092/kapacitor/v1/debug/vars" ] # ... ``` ### (Optional) Namespace monitoring data If Telegraf is running on your monitor instance, it will store your monitor’s own metrics in the `telegraf` database by default. To keep your monitor’s internal data separate from your other monitoring data, configure your local Telegraf agent to write to a database other than `telegraf` using the `database` setting under `[[outputs.influxdb]]` in your `telelgraf.conf`. ```toml # ... [[outputs.influxdb]] # ... ## The target database for metrics; will be created as needed. database = "monitor_local" # ... ``` ### (Optional) Update primary hostnames Telegraf’s default behavior is to include a `host` tag on each data point using the `os.hostname` provided by the host machine. Customize the hostname by updating the `hostname` setting under the `[agent]`section in your `telegraf.conf`. ***Example custom hostname in telegraf.conf*** ```toml [agent] # ... ## Override default hostname, if empty use os.Hostname() hostname = "primary_influxdb_1" # ... ``` ## Start Telegraf With Telegraf installed and configured on each of your primary nodes, start Telegraf using your custom configuration file. ```bash telegraf -config path/to/telegraf.conf ``` ## Create Kapacitor monitoring alerts Monitoring data should now be flowing from your primary cluster to your monitor where it can be processed by your monitor’s Kapacitor component.[Create Kapacitor alerts](/kapacitor/v1/working/alerts/) that alert you of issues detected in any of the monitored metrics. --- # Considerations for monitoring the 1.x TICK stack Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/internal-vs-external/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. One of the primary use cases for InfluxData’s TICK stack is infrastructure monitoring, including using the TICK stack to monitor itself or another TICK stack. These are the two main approaches to monitoring your TICK stack: * **[Internal monitoring](#internal-monitoring)** - A TICK stack that monitors itself. * **[“Watcher of watchers” approach](#the-watcher-of-watchers-approach)** - A TICK stack monitored by another TICK stack. ## Internal monitoring > [!Warning] > Not recommended for production environments. By default, the InfluxData platform is configured to monitor itself. Telegraf collects metrics from the host on which it’s running for things such as CPU usage, memory usage, disk usage, etc., and stores them in the `telegraf` database in InfluxDB. InfluxDB also reports performance metrics about itself, such as continuous query statistics, internal goroutine statistics, write statistics, series cardinality, and others, and stores them in the `_internal` database.*For the recommendation about `_internal` databases, see [Disable the `_internal` database in production clusters](#disable-the-_internal-database-in-production-clusters) below.* [Monitoring dashboards](/influxdb/v1/administration/monitor/monitoring-dashboards/) are available that visualize the default metrics provided in each of these databases. You can also [configure Kapacitor alerts](/kapacitor/v1/working/alerts/)to monitor and alert on each of these metrics. ### Pros of internal monitoring #### Simple setup Internal monitoring requires no additional setup or configuration changes. The TICK stack monitors itself out of the box. ### Cons of internal monitoring #### No hardware separation When using internal monitoring, if your TICK stack goes offline, your monitor does as well. Any configured alerts will not be sent and you will not be notified of any issues. Because of this, **internal monitoring is not recommended for production use cases.** ## The “watcher of watchers” approach > [!Note] > Recommended for production environments. A “watcher of watchers” approach for monitoring InfluxDB OSS and InfluxDB cluster nodes offers monitoring of your InfluxDB resources while ensuring that the monitoring statistics are available remotely in case of data loss. This usually takes the form of an Enterprise cluster being monitored by an OSS TICK stack. It consists of Telegraf agents installed on each node in your primary cluster reporting metrics for their respective hosts to a monitoring TICK stack installed on a separate server or cluster. *For information about setting up an external monitoring TICK stack, see [Setup an external monitor](/influxdb/v1/administration/monitor/external-monitor-setup/).* [Monitoring dashboards](/influxdb/v1/administration/monitor/monitoring-dashboards/) are available that visualize the default metrics provided by the Telegraf agents. You can also [configure Kapacitor alerts](/kapacitor/v1/working/alerts/)to monitor and alert on each of these metrics. ### Pros of external monitoring #### Hardware separation With a monitor running separate from your primary TICK stack, issues that occur in the primary stack will not affect the monitor. If your primary TICK stack goes down or has issues, your monitor will be able to detect them and alert you. ### Cons of external monitoring #### Slightly more setup There is more setup involved with external monitoring, but the benefits far outweigh the extra time required, especially for production use cases. ## Recommendations ### Disable the `_internal` database in production clusters InfluxData does **not** recommend using the `_internal` database in a production cluster. It creates unnecessary overhead, particularly for busy clusters, that can overload an already loaded cluster. Metrics stored in the `_internal` database primarily measure workload performance, which should only be tested in non-production environments. To disable the `_internal` database, set [`store-enabled`](/influxdb/v1/administration/config/#monitoring-settings)to `false` under the `[monitor]` section of your `influxdb.conf`. ***influxdb.conf*** ```toml # ... [monitor] # ... # Whether to record statistics internally. store-enabled = false #... ``` --- # Kapacitor measurements and fields Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/kapacitor-measurements/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Kapacitor exposes internal performance through the `/kacapitor/v1/debug/vars` endpoint. Use the Telegraf `kapacitor` plugins to capture these metrics and store them in InfluxDB. Enable the Kapacitor input plugin in your Telegraf configuration file: ```toml # ... [[inputs.kapacitor]] ## Multiple URLs from which to read Kapacitor-formatted JSON ## Default is "http://localhost:9092/kapacitor/v1/debug/vars". urls = [ "http://localhost:9092/kapacitor/v1/debug/vars" ] # ... ``` ### Visualize Kapacitor metrics Use the [Kapacitor Monitor dashboard](/influxdb/v1/administration/monitor/monitoring-dashboards/#monitor-kapacitor)to visualize Kapacitor metrics. ## Kapacitor measurements & fields Kapacitor exposes the following measurements and fields through the`/kacapitor/v1/debug/vars` endpoint. * [kapacitor](#kapacitor) * [num\_enabled\_tasks](#num-enabled-tasks) * [num\_subscriptions](#num-subscriptions) * [num\_tasks](#num-tasks) * [kapacitor\_edges](#kapacitor-edges) * [collected](#collected) * [emitted](#emitted) * [kapacitor\_ingress](#kapacitor-ingress) * [points\_received](#points-received) * [kapacitor\_load](#kapacitor-load) * [errors](#errors) * [kapacitor\_memstats](#kapacitor-memstats) * [alloc\_bytes](#alloc-bytes) * [buck\_hash\_sys\_bytes](#buck-hash-sys-bytes) * [frees](#frees) * [gc\_sys\_bytes](#gc-sys-bytes) * [gcc\_pu\_fraction](#gcc-pu-fraction) * [heap\_alloc\_bytes](#heap-alloc-bytes) * [heap\_idle\_bytes](#heap-idle-bytes) * [heap\_in\_use\_bytes](#heap-in-use-bytes) * [heap\_objects](#heap-objects) * [heap\_released\_bytes](#heap-released-bytes) * [heap\_sys\_bytes](#heap-sys-bytes) * [last\_gc\_ns](#last-gc-ns) * [lookups](#lookups) * [mallocs](#mallocs) * [mcache\_in\_use\_bytes](#mcache-in-use-bytes) * [mcache\_sys\_bytes](#mcache-sys-bytes) * [mspan\_in\_use\_bytes](#mspan-in-use-bytes) * [mspan\_sys\_bytes](#mspan-sys-bytes) * [next\_gc\_ns](#next-gc-ns) * [num\_gc](#num-gc) * [other\_sys\_bytes](#other-sys-bytes) * [pause\_total\_ns](#pause-total-ns) * [stack\_in\_use\_bytes](#stack-in-use-bytes) * [stack\_sys\_bytes](#stack-sys-bytes) * [sys\_bytes](#sys-bytes) * [total\_alloc\_bytes](#total-alloc-bytes) * [kapacitor\_nodes](#kapacitor-nodes) * [alerts\_inhibited](#alerts-inhibited) * [alerts\_triggered](#alerts-triggered) * [avg\_exec\_time\_ns](#avg-exec-time-ns) * [crits\_triggered](#crits-triggered) * [errors](#errors) * [infos\_triggered](#infos-triggered) * [oks\_triggered](#oks-triggered) * [points\_written](#points-written) * [warns\_triggered](#warns-triggered) * [write\_errors](#write-errors) * [kapacitor\_topics](#kapacitor-topics) * [collected](#collected) ### kapacitor The `kapacitor` measurement stores fields with information related to[Kapacitor tasks](/kapacitor/v1/introduction/getting-started/#kapacitor-tasks)and [subscriptions](/kapacitor/v1/administration/subscription-management/). #### num\_enabled\_tasks The number of enabled Kapacitor tasks. #### num\_subscriptions The number of Kapacitor/InfluxDB subscriptions. #### num\_tasks The total number of Kapacitor tasks. ### kapacitor\_edges The `kapacitor_edges` measurement stores fields with information related to[edges](/kapacitor/v1/reference/tick/introduction/#pipelines)in Kapacitor TICKscripts. #### collected The number of messages collected by TICKscript edges. #### emitted The number of messages emitted by TICKscript edges. ### kapacitor\_ingress The `kapacitor_ingress` measurement stores fields with information related to data coming into Kapacitor. #### points\_received The number of points received by Kapacitor. ### kapacitor\_load The `kapacitor_load` measurement stores fields with information related to the[Kapacitor Load Directory service](/kapacitor/v1/guides/load_directory/). #### errors The number of errors reported from the load directory service. ### kapacitor\_memstats The `kapacitor_memstats` measurement stores fields related to Kapacitor memory usage. #### alloc\_bytes The number of bytes of memory allocated by Kapacitor that are still in use. #### buck\_hash\_sys\_bytes The number of bytes of memory used by the profiling bucket hash table. #### frees The number of heap objects freed. #### gc\_sys\_bytes The number of bytes of memory used for garbage collection system metadata. #### gcc\_pu\_fraction The fraction of Kapacitor’s available CPU time used by garbage collection since Kapacitor started. #### heap\_alloc\_bytes The number of reachable and unreachable heap objects garbage collection has not freed. #### heap\_idle\_bytes The number of heap bytes waiting to be used. #### heap\_in\_use\_bytes The number of heap bytes in use. #### heap\_objects The number of allocated objects. #### heap\_released\_bytes The number of heap bytes released to the operating system. #### heap\_sys\_bytes The number of heap bytes obtained from `system`. #### last\_gc\_ns The nanosecond epoch time of the last garbage collection. #### lookups The total number of pointer lookups. #### mallocs The total number of mallocs. #### mcache\_in\_use\_bytes The number of bytes in use by mcache structures. #### mcache\_sys\_bytes The number of bytes used for mcache structures obtained from `system`. #### mspan\_in\_use\_bytes The number of bytes in use by mspan structures. #### mspan\_sys\_bytes The number of bytes used for mspan structures obtained from `system`. #### next\_gc\_ns The nanosecond epoch time of the next garbage collection. #### num\_gc The number of completed garbage collection cycles. #### other\_sys\_bytes The number of bytes used for other system allocations. #### pause\_total\_ns The total number of nanoseconds spent in garbage collection “stop-the-world” pauses since Kapacitor started. #### stack\_in\_use\_bytes The number of bytes in use by the stack allocator. #### stack\_sys\_bytes The number of bytes obtained from `system` for stack allocator. #### sys\_bytes The number of bytes of memory obtained from `system`. #### total\_alloc\_bytes The total number of bytes allocated, even if freed. ### kapacitor\_nodes The `kapacitor_nodes` measurement stores fields related to events that occur in[TICKscript nodes](/kapacitor/v1/reference/nodes/). #### alerts\_inhibited The total number of alerts inhibited by TICKscripts. #### alerts\_triggered The total number of alerts triggered by TICKscripts. #### avg\_exec\_time\_ns The average execution time of TICKscripts in nanoseconds. #### crits\_triggered The number of critical (`crit`) alerts triggered by TICKscripts. #### errors The number of errors caused by TICKscripts. #### infos\_triggered The number of info (`info`) alerts triggered by TICKscripts. #### oks\_triggered The number of ok (`ok`) alerts triggered by TICKscripts. #### points\_written The number of points written to InfluxDB or back to Kapacitor. #### warns\_triggered The number of warning (`warn`) alerts triggered by TICKscripts. #### working\_cardinality The total number of unique series processed. #### write\_errors The number of errors that occurred when writing to InfluxDB or other write endpoints. ### kapacitor\_topics The `kapacitor_topics` measurement stores fields related to[Kapacitor topics](/kapacitor/v1/working/using_alert_topics/). #### collected The number of events collected by Kapacitor topics. --- # InfluxDB _internal measurements and fields Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/measurements-internal/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. By default, InfluxDB writes internal runtime and performance metrics to the`_internal` database. This requires no configuration—the `monitor` service and its `store-enabled` setting default to `true`. Use these metrics to monitor InfluxDB and InfluxDB Enterprise and to create alerts to notify you when problems arise. Query the `_internal` database directly with the InfluxDB UI or `influx` CLI, using `SHOW STATS` and `SHOW DIAGNOSTICS` or standard InfluxQL. You can also export these metrics to third-party monitoring tools such as Prometheus and Grafana—see [Store internal metrics in an external monitor](#store-internal-metrics-in-an-external-monitor)and [Visualize InfluxDB internal metrics](#visualize-influxdb-internal-metrics) below. ### Disable the `_internal` database in production InfluxData does **not** recommend using the `_internal` database in a production cluster. It creates unnecessary overhead, particularly for busy clusters, that can overload an already loaded cluster. Metrics stored in the `_internal` database primarily measure workload performance and should only be tested in non-production environments. To disable the `_internal` database, set [`store-enabled`](/influxdb/v1/administration/config/#monitoring-settings)to `false` under the `[monitor]` section of your **InfluxDB configuration file**. ```toml # ... [monitor] # ... # Whether to record statistics internally. store-enabled = false #... ``` ### Store internal metrics in an external monitor To monitor InfluxDB `_internal` metrics in a production cluster, use Telegraf and the [`influxdb` input plugin](/telegraf/v1/input-plugins/influxdb/)to capture these metrics from the InfluxDB `/debug/vars` endpoint and store them in an external InfluxDB monitoring instance. For more information, see [Configure a Watcher of Watchers](/influxdb/v1/administration/monitor/external-monitor-setup/). InfluxDB also exposes Go runtime metrics in Prometheus exposition format on the`/metrics` HTTP endpoint. Third-party tools that support Prometheus-format scraping, such as Prometheus and Grafana, can collect these metrics directly without going through Telegraf. > [!Note] > When using the “watcher of watchers (WoW)” configuration, InfluxDB > metric field keys are prepended with `influxdb_`, but are otherwise identical > to those listed in [internal measurements and fields](#influxdb-_internal-measurements-and-fields). ## Visualize InfluxDB internal metrics Use the [InfluxDB OSS Monitor dashboard](/influxdb/v1/administration/monitor/monitoring-dashboards/#monitor-influxdb-oss)or the [InfluxDB Enterprise Monitor dashboard](/influxdb/v1/administration/monitor/monitoring-dashboards/#monitor-influxdb-enterprise)to visualize InfluxDB `_internal` metrics. ## InfluxDB \_internal measurements and fields > [!Note] > `_internal` series only appear after the corresponding subsystem has run at > least once, so a given instance may not report every field below. * [ae](#ae-enterprise-only) (Enterprise only) * [bytesRx](#bytesrx) * [errors](#errors) * [jobs](#jobs) * [jobsActive](#jobsactive) * [cluster](#cluster-enterprise-only) (Enterprise only) * [copyShardReq](#copyshardreq) * [createIteratorReq](#createiteratorreq) * [expandSourcesReq](#expandsourcesreq) * [fieldDimensionsReq](#fielddimensionsreq) * [iteratorCostReq](#iteratorcostreq) * [openConnections](#openconnections) * [removeShardReq](#removeshardreq) * [storeReadFilterReq](#storereadfilterreq) * [storeReadGroupReq](#storereadgroupreq) * [storeReadWindowAggregateReq](#storereadwindowaggregatereq) * [writeShardFail](#writeshardfail) * [writeShardPointsReq](#writeshardpointsreq) * [writeShardReq](#writeshardreq) * [cq](#cq) * [queryFail](#queryfail) * [queryOk](#queryok) * [database](#database) * [numMeasurements](#nummeasurements) * [numSeries](#numseries) * [hh](#hh-enterprise-only) (Enterprise only) * [bytesRead](#bytesread) * [bytesWritten](#byteswritten) * [queueBytes](#queuebytes) * [queueDepth](#queuedepth) * [writeBlocked](#writeblocked) * [writeDropped](#writedropped) * [writeNodeReq](#writenodereq) * [writeNodeReqFail](#writenodereqfail) * [writeNodeReqPoints](#writenodereqpoints) * [writeShardReq](#writeshardreq) * [writeShardReqPoints](#writeshardreqpoints) * [hh\_database](#hh_database-enterprise-only) (Enterprise only) * [bytesRead](#bytesread) * [bytesWritten](#byteswritten) * [queueBytes](#queuebytes) * [queueDepth](#queuedepth) * [writeBlocked](#writeblocked) * [writeDropped](#writedropped) * [writeNodeReq](#writenodereq) * [writeNodeReqFail](#writenodereqfail) * [writeNodeReqPoints](#writenodereqpoints) * [writeShardReq](#writeshardreq) * [writeShardReqPoints](#writeshardreqpoints) * [hh\_node](#hh_node-enterprise-only) (Enterprise only) * [queueTotalSize](#queuetotalsize) * [hh\_processor](#hh_processor-enterprise-only) (Enterprise only) * [bytesRead](#bytesread) * [bytesWritten](#byteswritten) * [queueBytes](#queuebytes) * [queueDepth](#queuedepth) * [writeBlocked](#writeblocked) * [writeDropped](#writedropped) * [writeNodeReq](#writenodereq) * [writeNodeReqFail](#writenodereqfail) * [writeNodeReqPoints](#writenodereqpoints) * [writeShardReq](#writeshardreq) * [writeShardReqPoints](#writeshardreqpoints) * [httpd](#httpd) * [authFail](#authfail) * [clientError](#clienterror) * [fluxQueryReq](#fluxqueryreq) * [fluxQueryReqDurationNs](#fluxqueryreqdurationns) * [fluxQueryRespBytes](#fluxqueryrespbytes) * [pingReq](#pingreq) * [pointsWrittenDropped](#pointswrittendropped) * [pointsWrittenFail](#pointswrittenfail) * [pointsWrittenOK](#pointswrittenok) * [promReadReq](#promreadreq) * [promWriteReq](#promwritereq) * [queryReq](#queryreq) * [queryReqDurationNs](#queryreqdurationns) * [queryRespBytes](#queryrespbytes) * [recoveredPanics](#recoveredpanics) * [req](#req) * [reqActive](#reqactive) * [reqDurationNs](#reqdurationns) * [serverError](#servererror) * [statusReq](#statusreq) * [valuesWrittenOK](#valueswrittenok) * [writeReq](#writereq) * [writeReqActive](#writereqactive) * [writeReqBytes](#writereqbytes) * [writeReqDurationNs](#writereqdurationns) * [localStore](#localstore) * [pointsWritten](#pointswritten) * [seriesCreated](#seriescreated) * [valuesWritten](#valueswritten) * [queryExecutor](#queryexecutor) * [queriesActive](#queriesactive) * [queriesExecuted](#queriesexecuted) * [queriesFailed](#queriesfailed) * [queriesFinished](#queriesfinished) * [queriesSlow](#queriesslow) * [queryDurationNs](#querydurationns) * [recoveredPanics](#recoveredpanics) * [rpc](#rpc-enterprise-only) (Enterprise only) * [idleStreams](#idlestreams) * [liveConnections](#liveconnections) * [liveStreams](#livestreams) * [rpcCalls](#rpccalls) * [rpcFailures](#rpcfailures) * [rpcReadBytes](#rpcreadbytes) * [rpcRetries](#rpcretries) * [rpcWriteBytes](#rpcwritebytes) * [singleUse](#singleuse) * [singleUseOpen](#singleuseopen) * [totalConnections](#totalconnections) * [totalStreams](#totalstreams) * [runtime](#runtime) * [Alloc](#alloc) * [Frees](#frees) * [HeapAlloc](#heapalloc) * [HeapIdle](#heapidle) * [HeapInUse](#heapinuse) * [HeapObjects](#heapobjects) * [HeapReleased](#heapreleased) * [HeapSys](#heapsys) * [Lookups](#lookups) * [Mallocs](#mallocs) * [NumGC](#numgc) * [NumGoroutine](#numgoroutine) * [PauseTotalNs](#pausetotalns) * [Sys](#sys) * [TotalAlloc](#totalalloc) * [shard](#shard) * [diskBytes](#diskbytes) * [fieldsCreate](#fieldscreate) * [seriesCreate](#seriescreate) * [writeBytes](#writebytes) * [writePointsDropped](#writepointsdropped) * [writePointsErr](#writepointserr) * [writePointsOk](#writepointsok) * [writeReq](#writereq) * [writeReqErr](#writereqerr) * [writeReqOk](#writereqok) * [writeValuesOk](#writevaluesok) * [subscriber](#subscriber) * [createFailures](#createfailures) * [memUsage](#memusage) * [pointsWritten](#pointswritten) * [writeFailures](#writefailures) * [tsm1\_cache](#tsm1_cache) * [WALCompactionTimeMs](#walcompactiontimems) * [cacheAgeMs](#cacheagems) * [cachedBytes](#cachedbytes) * [diskBytes](#diskbytes) * [memBytes](#membytes) * [snapshotCount](#snapshotcount) * [writeDropped](#writedropped) * [writeErr](#writeerr) * [writeOk](#writeok) * [tsm1\_engine](#tsm1_engine) * [cacheCompactionDuration](#cachecompactionduration) * [cacheCompactionErr](#cachecompactionerr) * [cacheCompactions](#cachecompactions) * [cacheCompactionsActive](#cachecompactionsactive) * [compactionPlannerFindGenerations](#compactionplannerfindgenerations) * [compactionPlannerPlan](#compactionplannerplan) * [compactionPlannerPlanLevel1](#compactionplannerplanlevel1) * [compactionPlannerPlanLevel2](#compactionplannerplanlevel2) * [compactionPlannerPlanLevel3](#compactionplannerplanlevel3) * [compactionPlannerPlanOptimize](#compactionplannerplanoptimize) * [tsmFullCompactionDuration](#tsmfullcompactionduration) * [tsmFullCompactionErr](#tsmfullcompactionerr) * [tsmFullCompactionQueue](#tsmfullcompactionqueue) * [tsmFullCompactions](#tsmfullcompactions) * [tsmFullCompactionsActive](#tsmfullcompactionsactive) * [tsmLevel1CompactionDuration](#tsmlevel1compactionduration) * [tsmLevel1CompactionErr](#tsmlevel1compactionerr) * [tsmLevel1CompactionQueue](#tsmlevel1compactionqueue) * [tsmLevel1Compactions](#tsmlevel1compactions) * [tsmLevel1CompactionsActive](#tsmlevel1compactionsactive) * [tsmLevel2CompactionDuration](#tsmlevel2compactionduration) * [tsmLevel2CompactionErr](#tsmlevel2compactionerr) * [tsmLevel2CompactionQueue](#tsmlevel2compactionqueue) * [tsmLevel2Compactions](#tsmlevel2compactions) * [tsmLevel2CompactionsActive](#tsmlevel2compactionsactive) * [tsmLevel3CompactionDuration](#tsmlevel3compactionduration) * [tsmLevel3CompactionErr](#tsmlevel3compactionerr) * [tsmLevel3CompactionQueue](#tsmlevel3compactionqueue) * [tsmLevel3Compactions](#tsmlevel3compactions) * [tsmLevel3CompactionsActive](#tsmlevel3compactionsactive) * [tsmOptimizeCompactionDuration](#tsmoptimizecompactionduration) * [tsmOptimizeCompactionErr](#tsmoptimizecompactionerr) * [tsmOptimizeCompactionQueue](#tsmoptimizecompactionqueue) * [tsmOptimizeCompactions](#tsmoptimizecompactions) * [tsmOptimizeCompactionsActive](#tsmoptimizecompactionsactive) * [tsm1\_filestore](#tsm1_filestore) * [diskBytes](#diskbytes) * [numFiles](#numfiles) * [tsm1\_wal](#tsm1_wal) * [currentSegmentDiskBytes](#currentsegmentdiskbytes) * [oldSegmentsDiskBytes](#oldsegmentsdiskbytes) * [writeErr](#writeerr) * [writeOk](#writeok) * [userquerybytes](#userquerybytes) * [userQueryRespBytes](#userqueryrespbytes) * [userwritebytes](#userwritebytes) * [userWriteReqBytes](#userwritereqbytes) * [write](#write) * [pointReq](#pointreq) * [pointReqHH](#pointreqhh-enterprise-only) (Enterprise only) * [pointReqLocal](#pointreqlocal) * [pointReqRemote](#pointreqremote-enterprise-only) (Enterprise only) * [pointsWrittenOK](#pointswrittenok-1) * [req](#req) * [subWriteDrop](#subwritedrop) * [subWriteOk](#subwriteok) * [valuesWrittenOK](#valueswrittenok-1) * [writeDrop](#writedrop) * [writeError](#writeerror) * [writeOk](#writeok) * [writePartial](#writepartial-enterprise-only) (Enterprise only) * [writeTimeout](#writetimeout) [](#) ### ae (Enterprise only) The measurement statistics related to the Anti-Entropy (AE) engine in InfluxDB Enterprise clusters. #### bytesRx The number of bytes received by the data node. #### errors The total number of anti-entropy jobs that have resulted in errors. #### jobs The total number of jobs executed by the data node. #### jobsActive The number of active (currently executing) jobs. ### cluster (Enterprise only) The `cluster` measurement tracks statistics related to the clustering features of the data nodes in InfluxDB Enterprise. The tags on the series indicate the source host of the stat. #### copyShardReq The number of internal requests made to copy a shard from one data node to another. #### createIteratorReq The number of read requests from other data nodes in the cluster. #### expandSourcesReq The number of remote node requests made to find measurements on this node that match a particular regular expression. Indicates a SELECT from a regex initiated on a different data node, which then sent an internal request to this node. There is not currently a statistic tracking how many queries with a regex, instead of a fixed measurement, were initiated on a particular node. #### fieldDimensionsReq The number of remote node requests for information about the fields and associated types, and tag keys of measurements on this data node. #### iteratorCostReq The number of internal requests for iterator cost. #### openConnections Tracks the number of open connections being handled by the data node (including counting logical connections multiplexed onto a single yamux connection). #### removeShardReq The number of internal requests to delete a shard from this data node. Exclusively incremented by use of the `influxd-ctl remove shard` command. #### storeReadFilterReq The number of internal storage read filter requests from other data nodes in the cluster. #### storeReadGroupReq The number of internal storage read group requests from other data nodes in the cluster. #### storeReadWindowAggregateReq The number of internal storage read window aggregate requests from other data nodes in the cluster. #### writeShardFail The total number of internal write requests from a remote node that failed. It’s the cousin of InfluxDB shard stat `writeReqErr`. A write request over HTTP is received by Node A. Node A does not have the shard locally, so it creates an internal request to Node B instructing what to write and to which shard. If Node B sees the request and if anything goes wrong, Node B increments its own `writeShardFail`. Depending on what went wrong, in most circumstances Node B would also increment its `writeReqErr` stat inherited from InfluxDB OSS. If Node A had the shard locally, there would be no internal request to write data to a remote node, so `writeShardFail` would not be incremented. #### writeShardPointsReq The number of points in every internal write request from any remote node, regardless of success. #### writeShardReq The number of internal write requests from a remote data node, regardless of success. ### cq The measurement statistics related to continuous queries (CQs). #### queryFail The total number of continuous queries that executed but failed. #### queryOk The total number of continuous queries that executed successfully. Note that this value may be incremented in some cases where a CQ is initiated but does not actually run, for example, due to misconfigured resample intervals. ### database #### numMeasurements The current number of measurements in the specified database. The series cardinality values are estimates, based on [HyperLogLog++ (HLL++)](https://github.com/influxdata/influxdb/blob/master/pkg/estimator/hll/hll.go). The numbers returned by the estimates when there are thousands or millions of measurements or series should be accurate within a relatively small margin of error. #### numSeries The current series cardinality of the specified database. The series cardinality values are estimates, based on [HyperLogLog++ (HLL++)](https://github.com/influxdata/influxdb/blob/master/pkg/estimator/hll/hll.go). The numbers returned by the estimates when there are thousands or millions of measurements or series should be accurate within a relatively small margin of error. ### hh (Enterprise only) The `hh` measurement statistics track events resulting in new hinted handoff (HH) processors in InfluxDB Enterprise. The `hh` measurement has one additional tag: * `path` - The path to the durable hinted handoff queue on disk. #### bytesRead The size, in bytes, of points read from the hinted handoff queue and sent to its destination data node. > [!Note] > > [!Note] > > [!Note] > Resets to zero after crash or restart, even if the HH queue was non-empty. #### bytesWritten The total number of bytes written to the hinted handoff queue. Note that this statistic only tracks bytes written during the lifecycle of the current process. Upon restart or a crash, this statistic resets to zero, even if the hinted handoff queue was not empty. #### queueBytes The total number of bytes remaining in the hinted handoff queue. #### queueDepth The total number of segments in the hinted handoff queue. The HH queue is a sequence of 10MB “segment” files. #### writeBlocked The number of writes blocked because the number of concurrent HH requests exceeds the limit. #### writeDropped The number of writes dropped from the HH queue because the write appeared to be corrupted. #### writeNodeReq The total number of write requests that succeeded in writing a batch to the destination node. #### writeNodeReqFail The total number of write requests that failed in writing a batch of data from the hinted handoff queue to the destination node. #### writeNodeReqPoints The total number of points successfully written from the HH queue to the destination node. #### writeShardReq The number of initial write requests handled by the hinted handoff engine for a remote node. Subsequent write requests to this node, destined for the same remote node, do not increment this statistics. This statistic resets to `0` upon restart of `influxd`, regardless of the state the last time the process was alive. It is incremented when the HH “supersystem” is instructed to enqueue a write for the node, and the “subsystem” for the destination node doesn’t exist and has to be created, and the “subsystem” created successfully. If HH files are on disk for a remote node at process startup, the branch that increments this stat will not be reached. #### writeShardReqPoints The number of write requests for each point in the initial request to the hinted handoff engine for a remote node. ### hh\_database (Enterprise only) The `hh_database` measurement aggregates all hinted handoff queues for a single database and node. This allows accurate reporting of total queue size for a single database to a target node. The `hh_database` measurement has two additional tags: * `db` — The name of the database * `node` — The node identifier #### bytesRead The size, in bytes, of points read from the hinted handoff queue and sent to its destination data node. Note that if the data node process is restarted while there is data in the HH queue,`bytesRead` may settle to a number larger than `bytesWritten`. Hinted handoff writes occur in concurrent batches as determined by the[`retry-concurrency`](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#retry-concurrency) setting. If an individual write succeeds, the metric is incremented. If any write out of the whole batch fails, the entire batch is considered unsuccessful, and every part of the batch will be retried later. This was not the intended behavior of this stat. The other situation where `bytesRead` could be larger would be after a restart of the process. Say at startup there were 1000 bytes still enqueued in HH from the previous run of the process. Immediately after a restart, both `bytesRead` and `bytesWritten` are set to zero. Assuming HH is properly depleted, and no future writes require HH, then the stats will read 1000 bytes read and 0 bytes written. Resets to zero after crash or restart, even if the HH queue was non-empty. #### bytesWritten The total number of bytes written to the hinted handoff queue. Note that this statistic only tracks bytes written during the lifecycle of the current process. Upon restart or a crash, this statistic resets to zero, even if the hinted handoff queue was not empty. #### queueBytes The total number of bytes remaining in the hinted handoff queue. This statistic should accurately and absolutely track the number of bytes of encoded data waiting to be sent to the remote node. This statistic should remain correct across restarts, unlike `bytesRead` and `bytesWritten` (see [#780](https://github.com/influxdata/docs.influxdata.com/issues/780)). #### queueDepth The total number of segments in the hinted handoff queue. The HH queue is a sequence of 10MB “segment” files. This is a coarse-grained statistic that roughly represents the amount of data queued for a remote node. The `queueDepth` values can give you a sense of when a queue is growing or shrinking. #### writeBlocked The number of writes blocked because the number of concurrent HH requests exceeds the limit. #### writeDropped The number of writes dropped from the HH queue because the write appeared to be corrupted. #### writeNodeReq The total number of write requests that succeeded in writing a batch to the destination node. #### writeNodeReqFail The total number of write requests that failed in writing a batch of data from the hinted handoff queue to the destination node. #### writeNodeReqPoints The total number of points successfully written from the HH queue to the destination node. #### writeShardReq The total number of every write batch request enqueued into the hinted handoff queue. #### writeShardReqPoints The total number of points enqueued into the hinted handoff queue. ### hh\_node (Enterprise only) Available in InfluxDB Enterprise 1.9.8 and later. The `hh_node` measurement stores hinted handoff statistics for all queues (shards) for a given node. The `hh_node` measurement has one additional tag: * `node` - The destination node for the recorded metrics. #### queueTotalSize Total bytes of disk space used by all hinted handoff queues for a single node. Tracks the disk usage of all hinted handoff queues for a given node (not the bytes waiting to be processed). Due to the implementation of the hinted handoff queue, a lag occurs between when bytes are processed and when they’re removed from the disk. `queueTotalSize` is used to determine when a node’s hinted handoff queue has reached the maximum size configured in the [hinted-handoff max-size](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#max-size) parameter. ### hh\_processor (Enterprise only) The `hh_processor` measurement stores statistics for a single queue (shard). In InfluxDB Enterprise, there is a hinted handoff processor on each data node. The `hh_processor` measurement has two additional tags: * `node` - The destination node for the recorded metrics. * `path` - The path to the durable hinted handoff queue on disk. > [!Note] > The `hh_processor` statistics against a host are only accurate for the lifecycle of the current process. > If the process crashes or restarts, `bytesRead` and `bytesWritten` are reset to zero, even if the HH queue was non-empty. #### bytesRead The size, in bytes, of points read from the hinted handoff queue and sent to its destination data node. Note that if the data node process is restarted while there is data in the HH queue,`bytesRead` may settle to a number larger than `bytesWritten`. Hinted handoff writes occur in concurrent batches as determined by the[`retry-concurrency`](/enterprise_influxdb/v1/administration/configure/config-data-nodes/#retry-concurrency) setting. If an individual write succeeds, the metric is incremented. If any write out of the whole batch fails, the entire batch is considered unsuccessful, and every part of the batch will be retried later. This was not the intended behavior of this stat. The other situation where `bytesRead` could be larger would be after a restart of the process. Say at startup there were 1000 bytes still enqueued in HH from the previous run of the process. Immediately after a restart, both `bytesRead` and `bytesWritten` are set to zero. Assuming HH is properly depleted, and no future writes require HH, then the stats will read 1000 bytes read and 0 bytes written. Resets to zero after crash or restart, even if the HH queue was non-empty. #### bytesWritten The total number of bytes written to the hinted handoff queue. Note that this statistic only tracks bytes written during the lifecycle of the current process. Upon restart or a crash, this statistic resets to zero, even if the hinted handoff queue was not empty. #### queueBytes The total number of bytes remaining in the hinted handoff queue. This statistic should accurately and absolutely track the number of bytes of encoded data waiting to be sent to the remote node. This statistic should remain correct across restarts, unlike `bytesRead` and `bytesWritten`(see [#780](https://github.com/influxdata/docs.influxdata.com/issues/780)). #### queueDepth The total number of segments in the hinted handoff queue. The HH queue is a sequence of 10MB “segment” files. This is a coarse-grained statistic that roughly represents the amount of data queued for a remote node. The `queueDepth` values can give you a sense of when a queue is growing or shrinking. #### writeBlocked The number of writes blocked because the number of concurrent HH requests exceeds the limit. #### writeDropped The number of writes dropped from the HH queue because the write appeared to be corrupted. #### writeNodeReq The total number of write requests that succeeded in writing a batch to the destination node. #### writeNodeReqFail The total number of write requests that failed in writing a batch of data from the hinted handoff queue to the destination node. #### writeNodeReqPoints The total number of points successfully written from the HH queue to the destination node. #### writeShardReq The total number of every write batch request enqueued into the hinted handoff queue. #### writeShardReqPoints The total number of points enqueued into the hinted handoff queue. ### httpd The `httpd` measurement stores fields related to the InfluxDB HTTP server. #### authFail The number of HTTP requests that were aborted due to authentication being required, but not supplied or incorrect. #### clientError The number of HTTP responses due to client errors, with a `4XX` HTTP status code. #### fluxQueryReq The number of Flux query requests served. #### fluxQueryReqDurationNs The duration (wall-time), in nanoseconds, spent executing Flux query requests. #### fluxQueryRespBytes The sum of all bytes returned in Flux query responses. #### pingReq The number of times InfluxDB HTTP server served the `/ping` HTTP endpoint. #### pointsWrittenDropped The number of points dropped by the storage engine. #### pointsWrittenFail The number of points accepted by the HTTP `/write` endpoint, but unable to be persisted. #### pointsWrittenOK The number of points successfully accepted and persisted by the HTTP `/write` endpoint. #### promReadReq The number of read requests to the Prometheus `/read` endpoint. #### promWriteReq The number of write requests to the Prometheus `/write` endpoint. #### queryReq The number of query requests. #### queryReqDurationNs The total query request duration, in nanosecond (ns). #### queryRespBytes The total number of bytes returned in query responses. #### recoveredPanics The total number of panics recovered by the HTTP handler. #### req The total number of HTTP requests served. #### reqActive The number of currently active requests. #### reqDurationNs The duration (wall time), in nanoseconds, spent inside HTTP requests. #### serverError The number of HTTP responses due to server errors. #### statusReq The number of status requests served using the HTTP `/status` endpoint. #### valuesWrittenOK The number of values (fields) successfully accepted and persisted by the HTTP `/write` endpoint. #### writeReq The number of write requests served using the HTTP `/write` endpoint. #### writeReqActive The number of currently active write requests. #### writeReqBytes The total number of bytes of line protocol data received by write requests, using the HTTP `/write` endpoint. #### writeReqDurationNs The duration (wall time), in nanoseconds, of write requests served using the `/write` HTTP endpoint. ### localStore The `localStore` measurement statistics are related to writes handled by the local storage engine. #### pointsWritten The total number of points written to the local store. #### seriesCreated The total number of series created in the local store. #### valuesWritten The total number of values (fields) written to the local store. ### queryExecutor The `queryExecutor` statistics related to usage of the Query Executor of the InfluxDB engine. #### queriesActive The number of active queries currently being handled. #### queriesExecuted The number of queries executed (started). #### queriesFailed The number of queries that failed to execute due to errors. #### queriesFinished The number of queries that have finished executing. #### queriesSlow The number of queries that exceeded the configured slow-query threshold. #### queryDurationNs The duration (wall time), in nanoseconds, of every query executed. If one query took 1000 ns from start to finish, and another query took 500 ns from start to finish and ran before the first query finished, the statistic would increase by 1500. #### recoveredPanics The number of panics recovered by the Query Executor. ### rpc (Enterprise only) The `rpc` measurement statistics are related to the use of RPC calls within InfluxDB Enterprise clusters. #### idleStreams The number of idle multiplexed streams across all live TCP connections. #### liveConnections The current number of live TCP connections to other nodes. #### liveStreams The current number of live multiplexed streams across all live TCP connections. #### rpcCalls The total number of RPC calls made to remote nodes. #### rpcFailures The total number of RPC failures, which are RPCs that did not recover. #### rpcReadBytes The total number of RPC bytes read. #### rpcRetries The total number of RPC calls that retried at least once. #### rpcWriteBytes The total number of RPC bytes written. #### singleUse The total number of single-use connections opened using Dial. #### singleUseOpen The number of single-use connections currently open. #### totalConnections The total number of TCP connections that have been established. #### totalStreams The total number of streams established. ### runtime The `runtime` measurement statistics include a subset of MemStats records statistics about the Go memory allocator. The `runtime` statistics can be useful to determine poor memory allocation strategies and related performance issues. The [Go runtime package](https://golang.org/pkg/runtime/) contains operations that interact with Go’s runtime system, including functions used to control goroutines. It also includes the low-level type information used by the [Go reflect package](https://golang.org/pkg/reflect/). #### Alloc The currently allocated number of bytes of heap objects. #### Frees The cumulative number of freed (live) heap objects. #### HeapAlloc The size, in bytes, of all heap objects. #### HeapIdle The number of bytes of idle heap objects. #### HeapInUse The number of bytes in in-use spans. #### HeapObjects The number of allocated heap objects. #### HeapReleased The number of bytes of physical memory returned to the OS. #### HeapSys The number of bytes of heap memory obtained from the OS. Measures the amount of virtual address space reserved for the heap. #### Lookups The number of pointer lookups performed by the runtime. Primarily useful for debugging runtime internals. #### Mallocs The total number of heap objects allocated. The total number of live objects is [Frees](#frees). #### NumGC The number of completed GC (garbage collection) cycles. #### NumGoroutine The total number of Go routines. #### PauseTotalNs The total duration, in nanoseconds, of total GC (garbage collection) pauses. #### Sys The total number of bytes of memory obtained from the OS. Measures the virtual address space reserved by the Go runtime for the heap, stacks, and other internal data structures. #### TotalAlloc The total number of bytes allocated for heap objects. This statistic does not decrease when objects are freed. ### shard The `shard` measurement statistics are related to working with shards in InfluxDB OSS and InfluxDB Enterprise. Each series also carries an `indexType` tag (`inmem` or `tsi1`), along with `database`, `engine`, `id`, `path`, `retentionPolicy`, and `walPath` tags. #### diskBytes The size, in bytes, of the shard, including the size of the data directory and the WAL directory. #### fieldsCreate The number of fields created. #### seriesCreate Then number of series created. #### writeBytes The number of bytes written to the shard. #### writePointsDropped The number of requests to write points t dropped from a write. Also, `http.pointsWrittentDropped` incremented when a point is dropped from a write (see [#780](https://github.com/influxdata/docs.influxdata.com/issues/780)). #### writePointsErr The number of requests to write points that failed to be written due to errors. #### writePointsOk The number of points written successfully. #### writeReq The total number of write requests. #### writeReqErr The total number of write requests that failed due to errors. #### writeReqOk The total number of successful write requests. #### writeValuesOk The total number of values (fields) successfully written to the shard. ### subscriber The `subscriber` measurement statistics are related to the usage of InfluxDB subscriptions. #### createFailures The number of subscriptions that failed to be created. #### memUsage The number of bytes of memory currently used by the subscriber’s internal write buffers. #### pointsWritten The total number of points that were successfully written to subscribers. #### writeFailures The total number of batches that failed to be written to subscribers. ### tsm1\_cache The `tsm1_cache` measurement statistics are related to the usage of the TSM cache. The following query example calculates various useful measurements related to the TSM cache. ```sql SELECT max(cacheAgeMs) / 1000.000 AS CacheAgeSeconds, max(memBytes) AS MaxMemBytes, max(diskBytes) AS MaxDiskBytes, max(snapshotCount) AS MaxSnapShotCount, (last(cachedBytes) - first(cachedBytes)) / (last(WALCompactionTimeMs) - first(WALCompactionTimeMs)) - 1000.000 AS CompactedBytesPerSecond, last(cachedBytes) AS CachedBytes, (last(cachedBytes) - first(cachedBytes))/300 as CacheThroughputBytesPerSecond FROM _internal.monitor.tsm1_cache WHERE time > now() - 1h GROUP BY time(5m), path ``` #### cacheAgeMs The duration, in milliseconds, since the cache was last snapshotted at sample time. This statistic indicates how busy the cache is. Large numbers indicate a cache which is idle with respect to writes. #### cachedBytes The total number of bytes that have been written into snapshots. This statistic is updated during the creation of a snapshot. The purpose of this statistic is to allow calculation of cache throughput between any two instances of time. The ratio of the difference between two samples of this statistic divided by the interval separating the samples is a measure of the cache throughput (more accurately, the rate at which data is being snapshotted). When combined with the `diskBytes`and `memBytes` statistics, it can also be used to calculate the rate at which data is entering the cache and rate at which is being purged from the cache. If the entry rate exceeds the exit rate for a sustained period of time, there is an issue that needs to be addressed. #### diskBytes The size, in bytes, of on-disk snapshots. #### memBytes The size, in bytes, of in-memory cache. #### snapshotCount The current level (number) of active snapshots. In a healthy system, this number should be between 0 and 1. A system experiencing transient write errors might expect to see this number rise. #### WALCompactionTimeMs The duration, in milliseconds, that the commit lock is held while compacting snapshots. The expression `(cachedBytes - diskBytes) / WALCompactionTime` provides an indication of how fast the WAL logs are being committed to TSM files. The ratio of the difference between the start and end “WALCompactionTime” values for an interval divided by the length of the interval provides an indication of how much of maximum cache throughput is being consumed. #### writeDropped The total number of writes dropped due to timeouts. #### writeErr The total number of writes that failed. #### writeOk The total number of successful writes. ### tsm1\_engine The `tsm1_engine` measurement statistics are related to the usage of a TSM storage engine with compressed blocks. #### cacheCompactionDuration The duration (wall time), in nanoseconds, spent in cache compactions. #### cacheCompactionErr The number of cache compactions that have failed due to errors. #### cacheCompactions The total number of cache compactions that have ever run. #### cacheCompactionsActive The number of cache compactions that are currently running. #### compactionPlannerFindGenerations The number of times the compaction planner scanned TSM files to find generations to compact. #### compactionPlannerPlan The number of times the compaction planner ran to generate a compaction plan. #### compactionPlannerPlanLevel1 The number of times the compaction planner generated a level 1 compaction plan. #### compactionPlannerPlanLevel2 The number of times the compaction planner generated a level 2 compaction plan. #### compactionPlannerPlanLevel3 The number of times the compaction planner generated a level 3 compaction plan. #### compactionPlannerPlanOptimize The number of times the compaction planner generated an optimize compaction plan. #### tsmFullCompactionDuration The duration (wall time), in nanoseconds, spent in full compactions. #### tsmFullCompactionErr The total number of TSM full compactions that have failed due to errors. #### tsmFullCompactionQueue The current number of pending TMS Full compactions. #### tsmFullCompactions The total number of TSM full compactions that have ever run. #### tsmFullCompactionsActive The number of TSM full compactions currently running. #### tsmLevel1CompactionDuration The duration (wall time), in nanoseconds, spent in TSM level 1 compactions. #### tsmLevel1CompactionErr The total number of TSM level 1 compactions that have failed due to errors. #### tsmLevel1CompactionQueue The current number of pending TSM level 1 compactions. #### tsmLevel1Compactions The total number of TSM level 1 compactions that have ever run. #### tsmLevel1CompactionsActive The number of TSM level 1 compactions that are currently running. #### tsmLevel2CompactionDuration The duration (wall time), in nanoseconds, spent in TSM level 2 compactions. #### tsmLevel2CompactionErr The number of TSM level 2 compactions that have failed due to errors. #### tsmLevel2CompactionQueue The current number of pending TSM level 2 compactions. #### tsmLevel2Compactions The total number of TSM level 2 compactions that have ever run. #### tsmLevel2CompactionsActive The number of TSM level 2 compactions that are currently running. #### tsmLevel3CompactionDuration The duration (wall time), in nanoseconds, spent in TSM level 3 compactions. #### tsmLevel3CompactionErr The number of TSM level 3 compactions that have failed due to errors. #### tsmLevel3CompactionQueue The current number of pending TSM level 3 compactions. #### tsmLevel3Compactions The total number of TSM level 3 compactions that have ever run. #### tsmLevel3CompactionsActive The number of TSM level 3 compactions that are currently running. #### tsmOptimizeCompactionDuration The duration (wall time), in nanoseconds, spent during TSM optimize compactions. #### tsmOptimizeCompactionErr The total number of TSM optimize compactions that have failed due to errors. #### tsmOptimizeCompactionQueue The current number of pending TSM optimize compactions. #### tsmOptimizeCompactions The total number of TSM optimize compactions that have ever run. #### tsmOptimizeCompactionsActive The number of TSM optimize compactions that are currently running. ### tsm1\_filestore The `tsm1_filestore` measurement statistics are related to the usage of the TSM file store. #### diskBytes The size, in bytes, of disk usage by the TSM file store. #### numFiles The total number of files in the TSM file store. ### tsm1\_wal The `tsm1_wal` measurement statistics are related to the usage of the TSM Write Ahead Log (WAL). #### currentSegmentDiskBytes The current size, in bytes, of the segment disk. #### oldSegmentsDiskBytes The size, in bytes, of the segment disk. #### writeErr The number of writes that failed due to errors. #### writeOk The number of writes that succeeded. ### userquerybytes The `userquerybytes` measurement tracks the number of bytes returned by queries for each user. It only appears when[`user-query-bytes-enabled`](/influxdb/v1/administration/config/#user-query-bytes-enabled) is set to `true` (available in InfluxDB OSS and Enterprise v1.12.3+). Each series carries a `user` tag (unauthenticated queries are attributed to`(anonymous)`), along with `bind` and `hostname` tags. On InfluxDB Enterprise, series also carry `clusterID` and `nodeID` tags. #### userQueryRespBytes The number of bytes returned by queries for this user. ### userwritebytes The `userwritebytes` measurement tracks the number of bytes received in write requests for each user. It only appears when[`user-write-bytes-enabled`](/influxdb/v1/administration/config/#user-write-bytes-enabled) is set to `true` (available in InfluxDB OSS v1.13.0+ and InfluxDB Enterprise v1.13.1+). Each series carries a `user` tag (unauthenticated writes are attributed to`(anonymous)`), along with `bind` and `hostname` tags. On InfluxDB Enterprise, series also carry `clusterID` and `nodeID` tags. #### userWriteReqBytes The number of bytes received in write requests for this user. ### write The `write` measurement statistics are about writes to the data node, regardless of the source of the write. #### pointReq The total number of every point requested to be written to this data node. Incoming writes have to make it through a couple of checks before reaching this point (points parse correctly, correct authentication provided, etc.). After these checks, this statistic should be incremented regardless of source (HTTP, UDP, `_internal` stats, OpenTSDB plugin, etc.). #### pointReqHH (Enterprise only) The total number of points received for write by this node and then enqueued into hinted handoff for the destination node. #### pointReqLocal The total number of point requests that have been attempted to be written into a shard on the same (local) node. #### pointReqRemote (Enterprise only) The total number of points received for write by this node but needed to be forwarded into a shard on a remote node. The `pointReqRemote` statistic is incremented immediately before the remote write attempt, which only happens if HH doesn’t exist for that node. Then if the write attempt fails, we check again if HH exists, and if so, add the point to HH instead. This statistic does not distinguish between requests that are directly written to the destination node versus enqueued into the hinted handoff queue for the destination node. #### pointsWrittenOK Number of points written to the HTTP `/write` endpoint and persisted successfully. #### req The total number of batches of points requested to be written to this node. #### subWriteDrop The total number of batches of points that failed to be sent to the subscription dispatcher. #### subWriteOk The total number of batches of points that were successfully sent to the subscription dispatcher. #### valuesWrittenOK Number of values (fields) written to the HTTP `/write` endpoint and persisted successfully. #### writeDrop The total number of write requests for points that have been dropped due to timestamps not matching any existing retention policies. #### writeError The total number of batches of points that were not successfully written, due to a failure to write to a local or remote shard. #### writeOk The total number of batches of points written at the requested consistency level. #### writePartial (Enterprise only) The total number of batches of points written to at least one node but did not meet the requested consistency level. #### writeTimeout The total number of write requests that failed to complete within the default write timeout duration. This could indicate severely reduced or contentious disk I/O or a congested network to a remote node. For a single write request that comes in over HTTP or another input method, `writeTimeout`will be incremented by 1 if the entire batch is not written within the timeout period, regardless of whether the points within the batch can be written locally or remotely. --- # Monitor Kubernetes using the InfluxData 1.x Platform Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/monitor-kubernetes/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## Monitor Kubernetes The TICK stack is an easy and performant way to monitor the services that make up a Kubernetes cluster, whether or not you’re running InfluxDB in a Kubernetes cluster or somewhere else. ### kube-influxdb Kubernetes monitoring project The [kube-influxdb](https://github.com/influxdata/kube-influxdb) project is a set of Helm charts to make collection and visualization of Kubernetes metrics easy. It uses Telegraf as the primary agent to collect metrics and events. [Read the kube-influxdb Getting Started guide.](https://github.com/influxdata/kube-influxdb/blob/master/docs/v1.0/getting_started.md) ### Collect Kubernetes metrics with Telegraf The [Telegraf metrics collection agent](/telegraf/v1/introduction/getting-started/) can collect many types of metrics in a Kubernetes cluster, like [Docker container metrics](/telegraf/v1/input-plugins/docker/) and [stats from kubelets](/telegraf/v1/input-plugins/kubernetes/). It can even scrape [Prometheus metrics API endpoints](/telegraf/v1/input-plugins/prometheus/). Telegraf is used in the [kube-influxdb project](#kube-influxdb-kubernetes-monitoring-project) to collect metrics. [Read about setting up a Kubernetes monitoring architecture using Telegraf](https://www.influxdata.com/blog/monitoring-kubernetes-architecture/) ### Prometheus remote read and write support InfluxDB supports the Prometheus remote read and write API for clusters already using Prometheus for metrics collection that need a more flexible time series data store. [Read about the Prometheus remote read and write API support in InfluxDB](/influxdb/v1/supported_protocols/prometheus/) --- # Set up 1.x monitoring dashboards Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/monitoring-dashboards/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The following dashboards provide visualizations of performance metrics for InfluxDB open source (OSS), InfluxDB Enterprise, and Kapacitor. ## Prebuilt dashboards Chronograf provides prebuilt monitoring dashboards that use data from specific Telegraf input plugins. To view prebuilt dashboards: 1. Open Chronograf and click **Host List** in the navigation bar. 2. Each link in the **Apps** column is a prebuilt dashboard generated using metrics from Telegraf input plugins. Click a link to view the associated dashboard. ## Import monitoring dashboards Use the dashboards below to visualize and monitor key TICK stack metrics. Download the dashboard file and import it into Chronograf. For detailed instructions, see [Import a dashboard](/chronograf/v1/administration/import-export-dashboards/#import-a-dashboard). * [Monitor InfluxDB OSS](#monitor-influxdb-oss) * [Monitor InfluxDB Enterprise](#monitor-influxdb-enterprise) * [Monitor Kapacitor](#monitor-kapacitor) ### Monitor InfluxDB OSS Use the InfluxDB OSS Monitor dashboard to monitor InfluxDB OSS in Chronograf. [Download InfluxDB OSS Monitor dashboard](/downloads/influxdb-oss-monitor-dashboard.json) The InfluxDB OSS Monitor dashboard uses data from the `_internal` database*([not recommended for production](/influxdb/v1/administration/monitor/internal-vs-external/#disable-the-_internal-database-in-production-clusters))*or collected by the [Telegraf `influxdb` input plugin](/telegraf/v1/input-plugins/influxdb/). This dashboard contains the following cells: * Queries Executed Per Minute * HTTP Requests Per Minute * Points Throughput Per Minute by Hostname * Series Cardinality & Measurements by Database * HTTP Request Duration (99th %) * Heap Size * Shard Write Errors * Continuous Queries Executed Per Minute ### Monitor InfluxDB Enterprise Use the InfluxDB Enterprise Monitor dashboard to monitor InfluxDB Enterprise in Chronograf. [Download InfluxDB Enterprise Monitor dashboard](/downloads/influxdb-enterprise-monitor-dashboard.json) The InfluxDB Enterprise Monitor dashboard uses data from the `_internal` database*([not recommended for production](/influxdb/v1/administration/monitor/internal-vs-external/#disable-the-_internal-database-in-production-clusters))*or collected by the [Telegraf `influxdb` input plugin](/telegraf/v1/input-plugins/influxdb/). This dashboard contains the following cells: * Queries Executed Per Minute * HTTP Requests Per Minute * Points Throughput Per Minute by Hostname * Series Cardinality & Measurements by Database * HTTP Request Duration (99th %) * Heap Size * Shard Write Errors * Continuous Queries Executed Per Minute * Hinted HandOff Queue Size * Anti-Entropy Errors & Jobs ### Monitor Kapacitor Use the Kapacitor Monitor dashboard to monitor Kapacitor in Chronograf. [Download Kapacitor Monitor dashboard](/downloads/kapacitor-monitor-dashboard.json) The Kapacitor Monitor dashboard requires the Telegraf[`mem`](/telegraf/v1/input-plugins/mem/),[`cpu`](/telegraf/v1/input-plugins/cpu/),[`system`](/telegraf/v1/input-plugins/system/), and [`kapacitor`](/telegraf/v1/input-plugins/kapacitor/)input plugins and contains the following cells: * Kapacitor Host RAM Usage * Kapacitor Host CPU Usage * Kapacitor Host Load (1, 5, 15) * Number of Subscriptions * Number of Running Tasks * Data Ingest Rate --- # Performance metrics commands Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/performance-metrics-commands/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Below are a collection of commands to find useful performance metrics about your InfluxDB instance. To find the number of points per second being written to the instance, the `monitor` service must be enabled. Then run: ```bash $ influx -execute 'select derivative(pointReq, 1s) from "write" where time > now() - 5m' -database '_internal' -precision 'rfc3339' ``` To find the number of writes separated by database since the beginning of the log file: ```bash grep 'POST' /var/log/influxdb/influxd.log | awk '{ print $10 }' | sort | uniq -c ``` Or, for systemd systems logging to journald: ```bash journalctl -u influxdb.service | awk '/POST/ { print $10 }' | sort | uniq -c ``` ## InfluxDB `/metrics` HTTP endpoint The InfluxDB `/metrics` endpoint is configured to produce the default Go metrics in Prometheus metrics format. ### Example using InfluxDB `/metrics` endpoint Below is an example of the output generated using the `/metrics` endpoint. Note that HELP is available to explain the Go statistics. ``` # HELP go_gc_duration_seconds A summary of the GC invocation durations. # TYPE go_gc_duration_seconds summary go_gc_duration_seconds{quantile="0"} 6.4134e-05 go_gc_duration_seconds{quantile="0.25"} 8.8391e-05 go_gc_duration_seconds{quantile="0.5"} 0.000131335 go_gc_duration_seconds{quantile="0.75"} 0.000169204 go_gc_duration_seconds{quantile="1"} 0.000544705 go_gc_duration_seconds_sum 0.004619405 go_gc_duration_seconds_count 27 # HELP go_goroutines Number of goroutines that currently exist. # TYPE go_goroutines gauge go_goroutines 29 # HELP go_info Information about the Go environment. # TYPE go_info gauge go_info{version="go1.10"} 1 # HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. # TYPE go_memstats_alloc_bytes gauge go_memstats_alloc_bytes 1.581062048e+09 # HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed. # TYPE go_memstats_alloc_bytes_total counter go_memstats_alloc_bytes_total 2.808293616e+09 # HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. # TYPE go_memstats_buck_hash_sys_bytes gauge go_memstats_buck_hash_sys_bytes 1.494326e+06 # HELP go_memstats_frees_total Total number of frees. # TYPE go_memstats_frees_total counter go_memstats_frees_total 1.1279913e+07 # HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started. # TYPE go_memstats_gc_cpu_fraction gauge go_memstats_gc_cpu_fraction -0.00014404354379774563 # HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. # TYPE go_memstats_gc_sys_bytes gauge go_memstats_gc_sys_bytes 6.0936192e+07 # HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use. # TYPE go_memstats_heap_alloc_bytes gauge go_memstats_heap_alloc_bytes 1.581062048e+09 # HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used. # TYPE go_memstats_heap_idle_bytes gauge go_memstats_heap_idle_bytes 3.8551552e+07 # HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use. # TYPE go_memstats_heap_inuse_bytes gauge go_memstats_heap_inuse_bytes 1.590673408e+09 # HELP go_memstats_heap_objects Number of allocated objects. # TYPE go_memstats_heap_objects gauge go_memstats_heap_objects 1.6924595e+07 # HELP go_memstats_heap_released_bytes Number of heap bytes released to OS. # TYPE go_memstats_heap_released_bytes gauge go_memstats_heap_released_bytes 0 # HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system. # TYPE go_memstats_heap_sys_bytes gauge go_memstats_heap_sys_bytes 1.62922496e+09 # HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection. # TYPE go_memstats_last_gc_time_seconds gauge go_memstats_last_gc_time_seconds 1.520291233297057e+09 # HELP go_memstats_lookups_total Total number of pointer lookups. # TYPE go_memstats_lookups_total counter go_memstats_lookups_total 397 # HELP go_memstats_mallocs_total Total number of mallocs. # TYPE go_memstats_mallocs_total counter go_memstats_mallocs_total 2.8204508e+07 # HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures. # TYPE go_memstats_mcache_inuse_bytes gauge go_memstats_mcache_inuse_bytes 13888 # HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system. # TYPE go_memstats_mcache_sys_bytes gauge go_memstats_mcache_sys_bytes 16384 # HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures. # TYPE go_memstats_mspan_inuse_bytes gauge go_memstats_mspan_inuse_bytes 1.4781696e+07 # HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system. # TYPE go_memstats_mspan_sys_bytes gauge go_memstats_mspan_sys_bytes 1.4893056e+07 # HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place. # TYPE go_memstats_next_gc_bytes gauge go_memstats_next_gc_bytes 2.38107752e+09 # HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations. # TYPE go_memstats_other_sys_bytes gauge go_memstats_other_sys_bytes 4.366786e+06 # HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator. # TYPE go_memstats_stack_inuse_bytes gauge go_memstats_stack_inuse_bytes 983040 # HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator. # TYPE go_memstats_stack_sys_bytes gauge go_memstats_stack_sys_bytes 983040 # HELP go_memstats_sys_bytes Number of bytes obtained from system. # TYPE go_memstats_sys_bytes gauge go_memstats_sys_bytes 1.711914744e+09 # HELP go_threads Number of OS threads created. # TYPE go_threads gauge go_threads 16 ``` --- # SHOW DIAGNOSTICS statement Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/show-diagnostics/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Diagnostic information includes mostly information about your InfluxDB server that is not necessarily numerical in format. This diagnostic information is not stored in the [`_internal`](/influxdb/v1/administration/monitor/measurements-internal/) database. To see InfluxDB server or node diagnostic information, you can use the [`SHOW DIAGNOSTICS`](/influxdb/v1/query_language/spec#show-diagnostics) statement. This statement returns InfluxDB instance information, including build details, uptime, hostname, server configuration, memory usage, and Go runtime diagnostics. This information is particularly useful to InfluxData Support, so be sure to include the output of this query anytime you file a support ticket or GitHub issue. ## `SHOW DIAGNOSTICS` measurement details The `SHOW DIAGNOSTICS` statement returns the following information. #### build * Branch * Build Time * Commit #### config * bind-address * reporting-disabled #### config-coordinator * log-queries-after * max-concurrent-queries * max-select-buckets * max-select-point * max-select-series * query-timeout * write-timeout #### config-cqs * enabled * query-stats-enabled * run-interval #### config-data * cache-max-memory-size * cache-snapshot-memory-size * cache-snapshot-write-cold-duration * compact-full-write-cold-duration * dir * max-concurrent-compactions * max-series-per-database * max-values-per-tag * wal-dir * wal-fsync-delay #### config-httpd * access-log-path * bind-address * enabled * https-enabled * max-connection-limit * max-row-limit #### config-meta * dir #### config-monitor * store-database * store-enabled * store-interval #### config-precreator * advance-period * check-interval * enabled #### config-retention * check-interval * enabled #### config-subscriber * enabled * http-timeout * write-buffer-size * write-concurrency #### network * hostname #### runtime * GOARCH * GOMAXPROCS * GOOS * version #### system * PID currentTime * started * uptime ## Example of `SHOW DIAGNOSTICS` output Here is an example of the output returned when running the `SHOW DIAGNOSTICS` statement on an InfluxDB OSS server. ``` > show diagnostics name: build Branch Build Time Commit Version ------ ---------- ------ ------- master 389de31c961831de0a9f4172173337d4a6193909 v1.6.3 name: config bind-address reporting-disabled ------------ ------------------ 127.0.0.1:8088 false name: config-coordinator log-queries-after max-concurrent-queries max-select-buckets max-select-point max-select-series query-timeout write-timeout ----------------- ---------------------- ------------------ ---------------- ----------------- ------------- ------------- 0s 0 0 0 0 0s 10s name: config-cqs enabled query-stats-enabled run-interval ------- ------------------- ------------ true false 1s name: config-data cache-max-memory-size cache-snapshot-memory-size cache-snapshot-write-cold-duration compact-full-write-cold-duration dir max-concurrent-compactions max-series-per-database max-values-per-tag wal-dir wal-fsync-delay --------------------- -------------------------- ---------------------------------- -------------------------------- --- -------------------------- ----------------------- ------------------ ------- --------------- 1073741824 26214400 10m0s 4h0m0s /usr/local/var/influxdb/data 0 1000000 100000 /usr/local/var/influxdb/wal 0s name: config-httpd access-log-path bind-address enabled https-enabled max-connection-limit max-row-limit --------------- ------------ ------- ------------- -------------------- ------------- :8086 true false 0 0 name: config-meta ## dir /usr/local/var/influxdb/meta name: config-monitor store-database store-enabled store-interval -------------- ------------- -------------- _internal true 10s name: config-precreator advance-period check-interval enabled -------------- -------------- ------- 30m0s 10m0s true name: config-retention check-interval enabled -------------- ------- 30m0s true name: config-subscriber enabled http-timeout write-buffer-size write-concurrency ------- ------------ ----------------- ----------------- true 30s 1000 40 name: network ## hostname influxdb-1.local name: runtime GOARCH GOMAXPROCS GOOS version ------ ---------- ---- ------- amd64 8 darwin go1.11 name: system PID currentTime started uptime --- ----------- ------- ------ 940 2018-10-15T15:07:47.435739Z 2018-10-15T06:03:34.002126Z 9h4m13.433613s ``` --- # SHOW STATS statement Source: https://docs.influxdata.com/influxdb/v1/administration/monitor/show-stats/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## Using the `SHOW STATS` statement The InfluxQL [`SHOW STATS`](/influxdb/v1/query_language/spec#show-stats)statement returns detailed measurement statistics on InfluxDB servers and available (enabled) components. Each component exporting statistics exports a measurement named after the module and various series that are associated with the measurement. To monitor InfluxDB system measurements over time, you can use the `_internal` database. For details on measurements available in the `_internal` database, see [Measurements for monitoring InfluxDB OSS and InfluxDB Enterprise (`_internal`)](/influxdb/v1/administration/monitor/measurements-internal/). > [!Note] > These statistics are all reset to zero when the InfluxDB service starts. ### The `SHOW STATS [FOR ]` option If you are interested in monitoring a specific InfluxDB component, use`SHOW STATS FOR ` to limit the results of the `SHOW STATS` statement to a specified module. The `SHOW STATS FOR ` statement displays detailed measurement statistics about a subsystem within a running `influxd` service. If a component is specified, it must be single-quoted. In the following example, the available statistics for the `httpd` module are returned. ```sql SHOW STATS FOR 'httpd' ``` ### The `SHOW STATS FOR 'indexes'` option The `SHOW STATS` statement does not list index memory usage unless you use the `SHOW STATS FOR 'indexes'` statement. This statement returns an estimate of memory use of all indexes. > [!Note] > Index memory usage is not reported with the default `SHOW STATS`statement because it is a potentially expensive operation. ## `SHOW STATS` measurement details The `SHOW STATS` statement returns the same statistics captured in the `internal` database, but only for the instant you run the statement. For details on the measurement statistics returned, see [InfluxDB `_internal` measurements](/influxdb/v1/administration/monitor/measurements-internal/). --- # InfluxDB ports Source: https://docs.influxdata.com/influxdb/v1/administration/ports/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## Enabled ports ### `8086` The default port that runs the InfluxDB HTTP service.[Configure this port](/influxdb/v1/administration/config#http-bind-address)in the configuration file. **Resources** [API Reference](/influxdb/v1/api/) ### 8088 The default port used by the RPC service for RPC calls made by the CLI for backup and restore operations (`influxdb backup` and `influxd restore`).[Configure this port](/influxdb/v1/administration/config#rpc-bind-address)in the configuration file. **Resources** [Backup and Restore](/influxdb/v1/administration/backup_and_restore/) ## Disabled ports ### 2003 The default port that runs the Graphite service.[Enable and configure this port](/influxdb/v1/administration/config#graphite-bind-address)in the configuration file. **Resources** [Graphite README](https://github.com/influxdata/influxdb/tree/1.8/services/graphite/README.md) ### 4242 The default port that runs the OpenTSDB service.[Enable and configure this port](/influxdb/v1/administration/config#opentsdb-bind-address)in the configuration file. **Resources** [OpenTSDB README](https://github.com/influxdata/influxdb/tree/1.8/services/opentsdb/README.md) ### 8089 The default port that runs the UDP service.[Enable and configure this port](/influxdb/v1/administration/config#udp-bind-address)in the configuration file. **Resources** [UDP README](https://github.com/influxdata/influxdb/tree/1.8/services/udp/README.md) ### 25826 The default port that runs the Collectd service.[Enable and configure this port](/influxdb/v1/administration/config#collectd-bind-address)in the configuration file. **Resources** [Collectd README](https://github.com/influxdata/influxdb/tree/1.8/services/collectd/README.md) --- # Rebuild the TSI index Source: https://docs.influxdata.com/influxdb/v1/administration/rebuild-tsi-index/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The InfluxDB [Time Series Index (TSI)](/influxdb/v1/concepts/tsi-details/)indexes or caches measurement and tag data to ensure queries are performant. In some cases, it may be necessary to flush and rebuild the TSI Index. Use the following steps to rebuild your InfluxDB TSI index: ## 1. Stop InfluxDB Stop InfluxDB by stopping the `influxd` process. ## 2. Remove all `_series` directories Remove all `_series` directories. By default, `_series` directories are stored at `/data//_series`, however you should check for and remove `_series` files throughout the `/data` directory. ## 3. Remove all index directories Remove all index directories. By default, index directories are stored at `/data///index`. ## 4. Rebuild the TSI index Use the [`influx_inspect` command line client (CLI)](/influxdb/v1/tools/influx_inspect)to rebuild the TSI index: ```sh # Syntax influx_inspect buildtsi -datadir -waldir # Example influx_inspect buildtsi -datadir /data -waldir /wal ``` ## 5. Restart InfluxDB Restart InfluxDB by starting the `influxd` process. > [!Note] > ## Rebuild the TSI index in an InfluxDB Enterprise cluster > > To rebuild the TSI index in an InfluxDB Enterprise cluster, perform the steps > above on each data node in the cluster one after the other. > After restarting the `influxd` process on a data node, allow the[hinted handoff queue (HHQ)](/enterprise_influxdb/v1/concepts/clustering/#hinted-handoff)to write all missed data to the updated node before moving on to the next node. --- # Manage InfluxDB security Source: https://docs.influxdata.com/influxdb/v1/administration/security/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Some customers may choose to install InfluxDB with public internet access, however doing so can inadvertently expose your data and invite unwelcome attacks on your database. Check out the sections below for how protect the data in your InfluxDB instance. ## Enable authentication Password protect your InfluxDB instance to keep any unauthorized individuals from accessing your data. Resources:[Set up Authentication](/influxdb/v1/administration/authentication_and_authorization/#set-up-authentication) ## Manage users and permissions Restrict access by creating individual users and assigning them relevant read and/or write permissions. Resources:[User Types and Privileges](/influxdb/v1/administration/authentication_and_authorization/#user-types-and-privileges),[User Management Commands](/influxdb/v1/administration/authentication_and_authorization/#user-management-commands) ## Enable HTTPS Enabling HTTPS encrypts the communication between clients and the InfluxDB server. HTTPS can also verify the authenticity of the InfluxDB server to connecting clients. Resources:[Enabling HTTPS](/influxdb/v1/administration/https_setup/) ## Configure security headers HTTP headers allow servers and clients to pass additional information along with requests. Certain headers help [enforce security](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers#Security) properties. Resources:[Configure HTTP headers](/influxdb/v1/administration/config/#http-headers) ## Secure your host ### Ports If you’re only running InfluxDB, close all ports on the host except for port `8086`. You can also use a proxy to port `8086`. InfluxDB uses port `8088` for remote [backups and restores](/influxdb/v1/administration/backup_and_restore/). We highly recommend closing that port and, if performing a remote backup, giving specific permission only to the remote machine. ### AWS recommendations We recommend implementing on-disk encryption; InfluxDB does not offer built-in support to encrypt the data. --- # Stability and compatibility Source: https://docs.influxdata.com/influxdb/v1/administration/stability_and_compatibility/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## 1.x API compatibility and stability One of the more important aspects of the 1.0 release is that this marks the stabilization of our API and storage format. Over the course of the last three years we’ve iterated aggressively, often breaking the API in the process. With the release of 1.0 and for the entire 1.x line of releases we’re committing to the following: ### No breaking InfluxDB API changes When it comes to the InfluxDB API, if a command works in 1.0 it will work unchanged in all 1.x releases…with one caveat. We will be adding [keywords](/influxdb/v1/query_language/spec/#keywords) to the query language. New keywords won’t break your queries if you wrap all [identifiers](/influxdb/v1/concepts/glossary/#identifier) in double quotes and all string literals in single quotes. This is generally considered best practice so it should be followed anyway. For users following that guideline, the query and ingestion APIs will have no breaking changes for all 1.x releases. Note that this does not include the Go code in the project. The underlying Go API in InfluxDB can and will change over the course of 1.x development. Users should be accessing InfluxDB through the [InfluxDB API](/influxdb/v1/api/). ### Storage engine stability The [TSM](/influxdb/v1/concepts/glossary/#tsm-time-structured-merge-tree) storage engine file format is now at version 1. While we may introduce new versions of the format in the 1.x releases, these new versions will run side-by-side with previous versions. What this means for users is there will be no lengthy migrations when upgrading from one 1.x release to another. ### Additive changes The query engine will have additive changes over the course of the new releases. We’ll introduce new query functions and new functionality into the language without breaking backwards compatibility. We may introduce new protocol endpoints (like a binary format) and versions of the line protocol and query API to improve performance and/or functionality, but they will have to run in parallel with the existing versions. Existing versions will be supported for the entirety of the 1.x release line. ### Ongoing support We’ll continue to fix bugs on the 1.x versions of the [line protocol](/influxdb/v1/concepts/glossary/#influxdb-line-protocol), query API, and TSM storage format. Users should expect to upgrade to the latest 1.x.x release for bug fixes, but those releases will all be compatible with the 1.0 API and won’t require data migrations. For instance, if a user is running 1.2 and there are bug fixes released in 1.3, they should upgrade to the 1.3 release. Until 1.4 is released, patch fixes will go into 1.3.x. Because all future 1.x releases are drop in replacements for previous 1.x releases, users should upgrade to the latest in the 1.x line to get all bug fixes. --- # Manage subscriptions in InfluxDB Source: https://docs.influxdata.com/influxdb/v1/administration/subscription-management/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxDB subscriptions are local or remote endpoints to which all data written to InfluxDB is copied. Subscriptions are primarily used with [Kapacitor](/kapacitor/), but any endpoint able to accept UDP, HTTP, or HTTPS connections can subscribe to InfluxDB and receive a copy of all data as it is written. ## How subscriptions work As data is written to InfluxDB, writes are duplicated to subscriber endpoints via HTTP, HTTPS, or UDP in [line protocol](/influxdb/v1/write_protocols/line_protocol_tutorial/). the InfluxDB subscriber service creates multiple “writers” ([goroutines](https://golangbot.com/goroutines/)) which send writes to the subscription endpoints. *The number of writer goroutines is defined by the [`write-concurrency`](/influxdb/v1/administration/config#write-concurrency) configuration.* As writes occur in InfluxDB, each subscription writer sends the written data to the specified subscription endpoints. However, with a high `write-concurrency` (multiple writers) and a high ingest rate, nanosecond differences in writer processes and the transport layer can result in writes being received out of order. > > > #### Important information about high write loads #### > > > > While setting the subscriber `write-concurrency` to greater than 1 does increase your > subscriber write throughput, it can result in out-of-order writes under high ingest rates. > Setting `write-concurrency` to 1 ensures writes are passed to subscriber endpoints sequentially, > but can create a bottleneck under high ingest rates. > > > > What `write-concurrency` should be set to depends on your specific workload > and need for in-order writes to your subscription endpoint. > > ## InfluxQL subscription statements Use the following InfluxQL statements to manage subscriptions: [`CREATE SUBSCRIPTION`](#create-subscriptions) [`SHOW SUBSCRIPTIONS`](#show-subscriptions) [`DROP SUBSCRIPTION`](#remove-subscriptions) ## Create subscriptions Create subscriptions using the `CREATE SUBSCRIPTION` InfluxQL statement. Specify the subscription name, the database name and retention policy to subscribe to, and the URL of the host to which data written to InfluxDB should be copied. ```sql -- Pattern: CREATE SUBSCRIPTION "" ON ""."" DESTINATIONS "" -- Examples: -- Create a SUBSCRIPTION on database 'mydb' and retention policy 'autogen' that sends data to 'example.com:9090' via HTTP. CREATE SUBSCRIPTION "sub0" ON "mydb"."autogen" DESTINATIONS ALL 'http://example.com:9090' -- Create a SUBSCRIPTION on database 'mydb' and retention policy 'autogen' that round-robins the data to 'h1.example.com:9090' and 'h2.example.com:9090' via UDP. CREATE SUBSCRIPTION "sub0" ON "mydb"."autogen" DESTINATIONS ANY 'udp://h1.example.com:9090', 'udp://h2.example.com:9090' ``` In case authentication is enabled on the subscriber host, adapt the URL to contain the credentials. ``` -- Create a SUBSCRIPTION on database 'mydb' and retention policy 'autogen' that sends data to another InfluxDB on 'example.com:8086' via HTTP. Authentication is enabled on the subscription host (user: subscriber, pass: secret). CREATE SUBSCRIPTION "sub0" ON "mydb"."autogen" DESTINATIONS ALL 'http://subscriber:secret@example.com:8086' ``` `SHOW SUBSCRIPTIONS` outputs all subscriber URL in plain text, including those with authentication credentials. Any user with the privileges to run `SHOW SUBSCRIPTIONS` is able to see these credentials. ### Sending subscription data to multiple hosts The `CREATE SUBSCRIPTION` statement allows you to specify multiple hosts as endpoints for the subscription. In your `DESTINATIONS` clause, you can pass multiple host strings separated by commas. Using `ALL` or `ANY` in the `DESTINATIONS` clause determines how InfluxDB writes data to each endpoint: `ALL`: Writes data to all specified hosts. `ANY`: Round-robins writes between specified hosts. ***Subscriptions with multiple hosts*** ```sql -- Write all data to multiple hosts CREATE SUBSCRIPTION "mysub" ON "mydb"."autogen" DESTINATIONS ALL 'http://host1.example.com:9090', 'http://host2.example.com:9090' -- Round-robin writes between multiple hosts CREATE SUBSCRIPTION "mysub" ON "mydb"."autogen" DESTINATIONS ANY 'http://host1.example.com:9090', 'http://host2.example.com:9090' ``` ### Subscription protocols Subscriptions can use HTTP, HTTPS, or UDP transport protocols. Which to use is determined by the protocol expected by the subscription endpoint. If creating a Kapacitor subscription, this is defined by the `subscription-protocol`option in the `[[influxdb]]` section of your [`kapacitor.conf`](/kapacitor/v1/administration/subscription-management/#subscription-protocol). ***kapacitor.conf*** ```toml [[influxdb]] # ... subscription-protocol = "http" # ... ``` *For information regarding HTTPS connections and secure communication between InfluxDB and Kapacitor, view the [Kapacitor security](/kapacitor/v1/administration/security/#secure-influxdb-and-kapacitor) documentation.* ## Show subscriptions The `SHOW SUBSCRIPTIONS` InfluxQL statement returns a list of all subscriptions registered in InfluxDB. ```sql SHOW SUBSCRIPTIONS ``` ***Example output:*** ```bash name: _internal retention_policy name mode destinations ---------------- ---- ---- ------------ monitor kapacitor-39545771-7b64-4692-ab8f-1796c07f3314 ANY [http://localhost:9092] ``` ## Remove subscriptions Remove or drop subscriptions using the `DROP SUBSCRIPTION` InfluxQL statement. ```sql -- Pattern: DROP SUBSCRIPTION "" ON ""."" -- Example: DROP SUBSCRIPTION "sub0" ON "mydb"."autogen" ``` ### Drop all subscriptions In some cases, it may be necessary to remove all subscriptions. Run the following bash script that utilizes the `influx` CLI, loops through all subscriptions, and removes them. This script depends on the `$INFLUXUSER` and `$INFLUXPASS` environment variables. If these are not set, export them as part of the script. ```bash # Environment variable exports: # Uncomment these if INFLUXUSER and INFLUXPASS are not already globally set. # export INFLUXUSER=influxdb-username # export INFLUXPASS=influxdb-password IFS=$'\n'; for i in $(influx -format csv -username $INFLUXUSER -password $INFLUXPASS -database _internal -execute 'show subscriptions' | tail -n +2 | grep -v name); do influx -format csv -username $INFLUXUSER -password $INFLUXPASS -database _internal -execute "drop subscription \"$(echo "$i" | cut -f 3 -d ',')\" ON \"$(echo "$i" | cut -f 1 -d ',')\".\"$(echo "$i" | cut -f 2 -d ',')\""; done ``` ## Configure InfluxDB subscriptions InfluxDB subscription configuration options are available in the `[subscriber]`section of the `influxdb.conf`. In order to use subscriptions, the `enabled` option in the `[subscriber]` section must be set to `true`. Below is an example `influxdb.conf` subscriber configuration: ```toml [subscriber] enabled = true http-timeout = "30s" insecure-skip-verify = false ca-certs = "" write-concurrency = 40 write-buffer-size = 1000 ``` ***Descriptions of `[subscriber]` configuration options are available in the [Configuring InfluxDB](/influxdb/v1/administration/config#subscription-settings) documentation.*** ## Troubleshooting ### Inaccessible or decommissioned subscription endpoints Unless a subscription is [dropped](#remove-subscriptions), InfluxDB assumes the endpoint should always receive data and will continue to attempt to send data. If an endpoint host is inaccessible or has been decommissioned, you will see errors similar to the following: ```bash # Some message content omitted (...) for the sake of brevity "Post http://x.y.z.a:9092/write?consistency=...: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" ... service=subscriber "Post http://x.y.z.a:9092/write?consistency=...: dial tcp x.y.z.a:9092: getsockopt: connection refused" ... service=subscriber "Post http://x.y.z.a:9092/write?consistency=...: dial tcp 172.31.36.5:9092: getsockopt: no route to host" ... service=subscriber ``` In some cases, this may be caused by a networking error or something similar preventing a successful connection to the subscription endpoint. In other cases, it’s because the subscription endpoint no longer exists and the subscription hasn’t been dropped from InfluxDB. > > > Because InfluxDB does not know if a subscription endpoint will or will not become accessible again, > subscriptions are not automatically dropped when an endpoint becomes inaccessible. > If a subscription endpoint is removed, you must manually [drop the subscription](#remove-subscriptions) from InfluxDB. > > --- # Upgrade to InfluxDB 1.11.x Source: https://docs.influxdata.com/influxdb/v1/administration/upgrading/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Upgrade to the latest version of InfluxDB OSS v1. ## Upgrade to InfluxDB 1.11.x 1. [Download](https://www.influxdata.com/downloads/) InfluxDB version 1.11.x and [install the upgrade](/influxdb/v1/introduction/installation/). 2. Migrate configuration file customizations from your existing configuration file to the InfluxDB 1.11.x [configuration file](/influxdb/v1/administration/config/). Add or modify your environment variables as needed. > [!Important] > #### Choose your index type > > InfluxDB 1.11.x supports two index types: > > * **Time Series Index (TSI)** - Recommended for most users. Removes RAM-based limits on series cardinality and provides better performance for high-cardinality datasets. > * **In-memory index (inmem)** - Default option that maintains compatibility with earlier versions but is limited by available system RAM (series cardinality is limited by available RAM). > > **When to use TSI:** > > * General purpose production instances. > * Especially recommended for: > * High-cardinality datasets (many unique tag combinations) > * Experiencing high memory usage or out-of-memory errors > * Large production deployments > > **When to use inmem:** > > * Small datasets when memory is not a constraint > * Ephemeral deployments such as development or testing environments > > To learn more about TSI, see [Time Series Index overview](/influxdb/v1/concepts/time-series-index/) and [TSI details](/influxdb/v1/concepts/tsi-details/). 1. **Optional:** To enable TSI in InfluxDB 1.11.x, complete the following steps: 1. If using the InfluxDB configuration file, find the `[data]` section, uncomment `index-version = "inmem"` and change the value to `tsi1`. 2. If using environment variables, set `INFLUXDB_DATA_INDEX_VERSION` to `tsi1`. 3. Delete shard `index` directories in your [InfluxDB `data/` directory](/influxdb/v1/concepts/file-system-layout). For example, in a Linux environment: ``` /var/lib/influxdb/data////index ``` 4. Build TSI by running the [influx\_inspect buildtsi](/influxdb/v1/tools/influx_inspect/#buildtsi) command. > [!Important] > Run the `buildtsi` command using the user account that you are going to run the database as, or ensure that the permissions match afterward. 2. Restart the `influxdb` service. > [!Tip] > #### Switch index types anytime > > The default configuration continues to use TSM-based shards with in-memory indexes (as in earlier versions). You can [switch between TSI and inmem index types](#switch-index-types) at any time. ## Switch index types You can switch between index types at any time after upgrading: **Switch from inmem to TSI:** * Complete steps 3 and 4 in [Upgrade to InfluxDB 1.11.x](#upgrade-to-influxdb-111x) * Recommended when experiencing high memory usage or out-of-memory errors with high-cardinality data **Switch from TSI to inmem:** * Change `tsi1` to `inmem` by completing steps 3a-3c and 4 in [Upgrade to InfluxDB 1.11.x](#upgrade-to-influxdb-111x) * Suitable for small datasets where memory is not a constraint ## Downgrade InfluxDB To downgrade to an earlier version, complete the procedures above in [Upgrade to InfluxDB 1.11.x](#upgrade-to-influxdb-111x), replacing the version numbers with the version that you want to downgrade to. After downloading the release, migrating your configuration settings, and enabling TSI or TSM, make sure to [rebuild your index](/influxdb/v1/administration/rebuild-tsi-index/). > [!Warning] > Some versions of InfluxDB may have breaking changes that impact your ability to upgrade and downgrade. For example, you cannot downgrade from InfluxDB 1.3 or later to an earlier version. Please review the applicable version of release notes to check for compatibility issues between releases. ## Upgrade to InfluxDB Enterprise To upgrade from InfluxDB OSS to InfluxDB Enterprise, [contact InfluxData Sales](https://www.influxdata.com/contact-sales/). #### Related * [Migrate InfluxDB OSS instances to InfluxDB Enterprise v1 clusters](/enterprise_influxdb/v1/guides/migration/) * [Upgrade InfluxDB Enterprise v1 clusters](/enterprise_influxdb/v1/administration/upgrading/) --- # InfluxDB HTTP API Source: https://docs.influxdata.com/influxdb/v1/api/ [Download InfluxDB API Spec](/openapi/influxdb-oss-v1-openapi.yml) Use the InfluxDB HTTP API to write data, query data, and check server status. ### [Authentication](/influxdb/v1/api/authentication/) Authentication is **disabled by default** in InfluxDB v1. When disabled, the database accepts any request without checking credentials—supplied credentials are silently ignored. When enabled, every… ### [Buckets (v2 compatible)](/influxdb/v1/api/buckets-v2-compatible/) Manage databases as buckets using InfluxDB v2-compatible endpoints. Provides forward compatibility with InfluxDB 2.x client libraries and InfluxDB 1.8+. ### [Debug](/influxdb/v1/api/debug/) Access debugging and profiling endpoints for troubleshooting and performance analysis of an InfluxDB v1 instance. ### [Delete (v2 compatible)](/influxdb/v1/api/delete-v2-compatible/) Delete data using InfluxDB v2-compatible predicate expressions. Provides forward compatibility with InfluxDB 2.x client libraries and InfluxDB 1.8+. ### [Query](/influxdb/v1/api/query/) Query data using InfluxQL via the `/query` endpoint, supporting both read queries (SELECT, SHOW) and management queries (CREATE, DROP, ALTER). ### [System Information](/influxdb/v1/api/system-information/) Check server status, health, and version information for an InfluxDB v1 instance. ### [Write](/influxdb/v1/api/write/) Write time series data to InfluxDB v1 in line protocol format using the `/write` endpoint. --- # All endpoints Source: https://docs.influxdata.com/influxdb/v1/api/all-endpoints/ [Download InfluxDB API Spec](/openapi/influxdb-oss-v1-openapi.yml) All InfluxDB API endpoints, sorted by path. ## v2-compatible API [GET`/api/v2/buckets`List buckets (v2 compatible)v2](/influxdb/v1/api/buckets-v2-compatible/#operation/GetApiV2Buckets)[POST`/api/v2/buckets`Create bucket (v2 compatible)v2](/influxdb/v1/api/buckets-v2-compatible/#operation/PostApiV2Buckets)[DELETE`/api/v2/buckets/{bucketID}`Delete bucket (v2 compatible)v2](/influxdb/v1/api/buckets-v2-compatible/#operation/DeleteApiV2BucketsBucketID)[POST`/api/v2/delete`Delete data (v2 compatible)v2](/influxdb/v1/api/delete-v2-compatible/#operation/PostApiV2Delete)[POST`/api/v2/query`Query with Flux (v2 compatible)v2](/influxdb/v1/api/query/#operation/PostApiV2Query)[POST`/api/v2/write`Write data (v2 compatible)v2](/influxdb/v1/api/write/#operation/PostApiV2Write) ## v1-compatible API [GET`/health`Check server healthv2](/influxdb/v1/api/system-information/#operation/GetHealth)[GET`/ping`Check server status](/influxdb/v1/api/system-information/#operation/GetPing)[HEAD`/ping`Check server status (HEAD)](/influxdb/v1/api/system-information/#operation/HeadPing)[GET`/query`Query data (GET)](/influxdb/v1/api/query/#operation/GetQuery)[POST`/query`Query data (POST)](/influxdb/v1/api/query/#operation/PostQuery)[POST`/write`Write data](/influxdb/v1/api/write/#operation/PostWrite) ## Management API [GET`/debug/pprof`Get profiling index](/influxdb/v1/api/debug/#operation/GetDebugPprof)[GET`/debug/pprof/{profile}`Get profile data](/influxdb/v1/api/debug/#operation/GetDebugPprofProfile)[GET`/debug/pprof/all`Get all profiles archive](/influxdb/v1/api/debug/#operation/GetDebugPprofAll)[GET`/debug/requests`Track HTTP requests](/influxdb/v1/api/debug/#operation/GetDebugRequests)[GET`/debug/vars`Get server statistics](/influxdb/v1/api/debug/#operation/GetDebugVars) --- # Authentication Source: https://docs.influxdata.com/influxdb/v1/api/authentication/ [Download InfluxDB API Spec](/openapi/influxdb-oss-v1-openapi.yml) Authentication is **disabled by default** in InfluxDB v1. When disabled, the database accepts any request without checking credentials—supplied credentials are silently ignored. When enabled, every HTTP request must include valid credentials. Use one of the following schemes to authenticate to the InfluxDB v1 API: * Basic authentication * Query string authentication * Token authentication ## Basic Authentication Use HTTP Basic Authentication by including your username and password in the request. ```bash curl -u username:password "http://localhost:8086/query?q=SHOW+DATABASES" ``` Or encode credentials in the URL (not recommended for production): ```bash curl "http://username:password@localhost:8086/query?q=SHOW+DATABASES" ``` ## Query String Authentication Pass your credentials as query parameters. Use `u` for username and `p` for password. ```bash curl "http://localhost:8086/query?u=username&p=password&q=SHOW+DATABASES" ``` > [!Note] > Query string authentication exposes credentials in URLs and server logs. > Use Basic Authentication or Token Authentication for production environments. ## Token Authentication For v2-compatible endpoints (`/api/v2/*`), use the `Authorization` header with the `Token` scheme. Include your InfluxDB 1.x username and password separated by a colon: ```bash curl -H "Authorization: Token username:password" \ "http://localhost:8086/api/v2/query" ``` This format is compatible with InfluxDB 2.x client libraries, allowing you to use the same code with both InfluxDB 1.8+ and InfluxDB 2.x. #### Related * [Manage authentication and authorization](/influxdb/v1/administration/authentication_and_authorization/) --- # Buckets (v2 compatible) Source: https://docs.influxdata.com/influxdb/v1/api/buckets-v2-compatible/ [Download InfluxDB API Spec](/openapi/influxdb-oss-v1-openapi.yml) Manage databases as buckets using InfluxDB v2-compatible endpoints. Provides forward compatibility with InfluxDB 2.x client libraries and InfluxDB 1.8+. GET`/api/v2/buckets`v2 ### List buckets (v2 compatible) List all databases as buckets. Provides forward compatibility with InfluxDB 2.x client libraries. Example request[Ask AI about this](#) ```sh curl --request GET \ "http://localhost:8086/api/v2/buckets" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 200List of buckets `buckets`object[] `id`string Bucket ID (database/retention-policy) `name`string Bucket name `retentionRules`object[] `everySeconds`integer `type`string Allowed:`expire` 401Unauthorized `error`requiredstring Error message POST`/api/v2/buckets`v2 ### Create bucket (v2 compatible) Create a new database as a bucket. Provides forward compatibility with InfluxDB 2.x client libraries. #### Request bodyrequired Content-Type:`application/json` `name`requiredstring Database name `retentionRules`object[] `everySeconds`integer Retention period in seconds `type`string Allowed:`expire` Example request[Ask AI about this](#) ```sh curl --request POST \ "http://localhost:8086/api/v2/buckets" \ --header "Authorization: Bearer INFLUX_TOKEN" \ --header "Content-Type: application/json" \ --data-raw '{ "name": "NAME", "retentionRules": [ { "everySeconds": 0, "type": "expire" } ] }' ``` #### Responses 201Bucket created `id`string Bucket ID (database/retention-policy) `name`string Bucket name `retentionRules`object[] `everySeconds`integer `type`string Allowed:`expire` 401Unauthorized `error`requiredstring Error message DELETE`/api/v2/buckets/{bucketID}`v2 ### Delete bucket (v2 compatible) Delete a database/retention policy combination. The bucketID format is `database/retention-policy`. #### Parameters ##### Path parameters `bucketID`requiredstring Bucket ID in format `database/retention-policy` Example request[Ask AI about this](#) ```sh curl --request DELETE \ "http://localhost:8086/api/v2/buckets/{bucketID}" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 204Bucket deleted 401Unauthorized `error`requiredstring Error message 404Bucket not found `error`requiredstring Error message --- # Debug Source: https://docs.influxdata.com/influxdb/v1/api/debug/ [Download InfluxDB API Spec](/openapi/influxdb-oss-v1-openapi.yml) Access debugging and profiling endpoints for troubleshooting and performance analysis of an InfluxDB v1 instance. GET`/debug/pprof` ### Get profiling index Returns an HTML page listing available Go pprof profiles. Use the individual profile endpoints to retrieve specific profile data. Example request[Ask AI about this](#) ```sh curl --request GET \ "http://localhost:8086/debug/pprof" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 200HTML page with profile links GET`/debug/pprof/{profile}` ### Get profile data Retrieve a specific Go pprof profile. Available profiles: * `block`: Stack traces that led to blocking on synchronization primitives * `goroutine`: Stack traces of all current goroutines * `heap`: Sampling of stack traces for heap allocations * `mutex`: Stack traces of holders of contended mutexes * `threadcreate`: Stack traces that led to creation of new OS threads * `profile`: CPU profile (use `seconds` parameter to specify duration) * `trace`: Execution trace (use `seconds` parameter to specify duration) #### Parameters ##### Path parameters `profile`requiredstring Profile name Allowed values:`block`, `goroutine`, `heap`, `mutex`, `threadcreate`, `profile`, `trace`, `allocs`, `cmdline` ##### Query parameters `seconds`integer Duration in seconds for CPU profile or trace Default:`30` `debug`integer Return human-readable text output instead of binary Allowed values:`0`, `1` Example request[Ask AI about this](#) ```sh curl --request GET \ "http://localhost:8086/debug/pprof/{profile}" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 200Profile data GET`/debug/pprof/all` ### Get all profiles archive Generate a `profiles.tar.gz` archive containing all standard Go profiling information and additional debugging data. Intended primarily for use by InfluxData support. Use the `cpu` parameter to include a CPU profile of the specified duration. #### Parameters ##### Query parameters `cpu`string Duration for CPU profile. Specify as a duration string (e.g., `30s`). For InfluxDB 1.8.3 and earlier, use `cpu=true`. Example request[Ask AI about this](#) ```sh curl --request GET \ "http://localhost:8086/debug/pprof/all" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 200Compressed archive containing all profiles GET`/debug/requests` ### Track HTTP requests Track HTTP client requests to the `/write` and `/query` endpoints. Returns the number of writes and queries per username and IP address over the specified time interval. #### Parameters ##### Query parameters `seconds`integer Duration in seconds to collect request data Default:`10` Example request[Ask AI about this](#) ```sh curl --request GET \ "http://localhost:8086/debug/requests" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 200Request statistics by user and IP GET`/debug/vars` ### Get server statistics Retrieve runtime statistics and information about the InfluxDB instance. Returns detailed metrics in JSON format including memory usage, goroutine counts, and database statistics. The [InfluxDB Telegraf input plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/influxdb)can collect these metrics automatically. Example request[Ask AI about this](#) ```sh curl --request GET \ "http://localhost:8086/debug/vars" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 200Server statistics in JSON format --- # Delete (v2 compatible) Source: https://docs.influxdata.com/influxdb/v1/api/delete-v2-compatible/ [Download InfluxDB API Spec](/openapi/influxdb-oss-v1-openapi.yml) Delete data using InfluxDB v2-compatible predicate expressions. Provides forward compatibility with InfluxDB 2.x client libraries and InfluxDB 1.8+. POST`/api/v2/delete`v2 ### Delete data (v2 compatible) Delete data from InfluxDB using predicate expressions. Supports deletion by tag value, timestamp, and measurement. **Predicate Syntax:** ``` _measurement="example" AND tagKey="tagValue" ``` See [delete predicate syntax](/influxdb/v2/reference/syntax/delete-predicate/)for detailed syntax and [limitations](/influxdb/v2/reference/syntax/delete-predicate/#limitations). #### Parameters ##### Query parameters `bucket`requiredstring Database and retention policy in format `database/retention-policy` `precision`string Timestamp precision Allowed values:`ns`, `us`, `ms`, `s` #### Request bodyrequired Content-Type:`application/json` `predicate`string InfluxQL-like predicate expression to filter data to delete. Example: `_measurement="example" AND tagKey="tagValue"` `start`requiredstring \ Start time (inclusive) `stop`requiredstring \ Stop time (exclusive) Example request[Ask AI about this](#) ```sh curl --request POST \ "http://localhost:8086/api/v2/delete?bucket=BUCKET" \ --header "Authorization: Bearer INFLUX_TOKEN" \ --header "Content-Type: application/json" \ --data-raw '{ "predicate": "PREDICATE", "start": "START", "stop": "STOP" }' ``` #### Responses 204Delete successful 400Bad request `error`requiredstring Error message 401Unauthorized `error`requiredstring Error message --- # Query Source: https://docs.influxdata.com/influxdb/v1/api/query/ [Download InfluxDB API Spec](/openapi/influxdb-oss-v1-openapi.yml) Query data using InfluxQL via the `/query` endpoint, supporting both read queries (SELECT, SHOW) and management queries (CREATE, DROP, ALTER). GET`/query` ### Query data (GET) Query data using InfluxQL. Use GET for read-only queries that start with: * `SELECT` (except queries with `INTO` clause) * `SHOW` For write operations (CREATE, DROP, ALTER, etc.), use POST. #### Parameters ##### Query parameters `db`string Target database for the query `q`requiredstring InfluxQL query string `epoch`string Return timestamps as Unix epoch values with specified precision. By default, timestamps are returned in RFC3339 format. Allowed values:`ns`, `u`, `µ`, `ms`, `s`, `m`, `h` `pretty`boolean Enable pretty-printed JSON output (not recommended for production) `chunked`string Return results in streamed batches. Set to `true` to chunk by series or every 10,000 points. Set to a number to chunk by that many points. `u`string Username for authentication `p`string Password for authentication Example request[Ask AI about this](#) ```sh curl --request GET \ "http://localhost:8086/query?q=Q" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 200Query executed successfully `results`object[] `error`string `series`object[] `columns`string[] `name`string Measurement name `tags`object `values`array[] `statement_id`integer 400Bad request (syntax error in query) `error`requiredstring Error message 401Unauthorized `error`requiredstring Error message POST`/query` ### Query data (POST) Query data or execute database management commands using InfluxQL. Use POST for queries that start with: * `SELECT` with `INTO` clause * `ALTER` * `CREATE` * `DELETE` * `DROP` * `GRANT` * `KILL` * `REVOKE` #### Parameters ##### Query parameters `db`string Target database for the query `epoch`string Return timestamps as Unix epoch values with specified precision. By default, timestamps are returned in RFC3339 format. Allowed values:`ns`, `u`, `µ`, `ms`, `s`, `m`, `h` `pretty`boolean Enable pretty-printed JSON output (not recommended for production) `chunked`string Return results in streamed batches. Set to `true` to chunk by series or every 10,000 points. Set to a number to chunk by that many points. `u`string Username for authentication `p`string Password for authentication #### Request bodyrequired Content-Type:`application/json` Example request[Ask AI about this](#) ```sh curl --request POST \ "http://localhost:8086/query" \ --header "Authorization: Bearer INFLUX_TOKEN" \ --header "Content-Type: application/x-www-form-urlencoded" ``` #### Responses 200Query executed successfully `results`object[] `error`string `series`object[] `columns`string[] `name`string Measurement name `tags`object `values`array[] `statement_id`integer 400Bad request `error`requiredstring Error message 401Unauthorized `error`requiredstring Error message POST`/api/v2/query`v2 ### Query with Flux (v2 compatible) Query data using [Flux](/influxdb/v1/flux/) language. This endpoint provides forward compatibility with InfluxDB 2.x client libraries. **Required Headers:** * `Accept: application/csv` * `Content-type: application/vnd.flux` * `Authorization: Token username:password` (if authentication is enabled) #### Request bodyrequired Content-Type:`application/json` Example request[Ask AI about this](#) ```sh curl --request POST \ "http://localhost:8086/api/v2/query" \ --header "Authorization: Bearer INFLUX_TOKEN" \ --header "Content-Type: application/vnd.flux" ``` #### Responses 200Query results in CSV format 401Unauthorized `error`requiredstring Error message --- # System Information Source: https://docs.influxdata.com/influxdb/v1/api/system-information/ [Download InfluxDB API Spec](/openapi/influxdb-oss-v1-openapi.yml) Check server status, health, and version information for an InfluxDB v1 instance. GET`/ping` ### Check server status Check the status of your InfluxDB instance and retrieve version information. The `/ping` endpoint returns a `204 No Content` response by default. Use the `verbose=true` query parameter to return a `200 OK` response, which is required for [Google Cloud Load Balancing](https://cloud.google.com/load-balancing/docs/health-check-concepts) health checks. #### Parameters ##### Query parameters `verbose`boolean When `true`, returns HTTP 200 instead of 204. Required for Google Cloud Load Balancing health checks. Example request[Ask AI about this](#) ```sh curl --request GET \ "http://localhost:8086/ping" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 200Server is running (verbose mode) 204Server is running HEAD`/ping` ### Check server status (HEAD) Check the status of your InfluxDB instance using a HEAD request. Returns the same headers as GET but without a response body. Example request[Ask AI about this](#) ```sh curl --request HEAD \ "http://localhost:8086/ping" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 204Server is running GET`/health`v2 ### Check server health Check the health of your InfluxDB instance. Returns health status as JSON with a `pass` or `fail` status. Example request[Ask AI about this](#) ```sh curl --request GET \ "http://localhost:8086/health" \ --header "Authorization: Bearer INFLUX_TOKEN" ``` #### Responses 200Server is healthy `message`string Example:`"ready for queries and writes"` `name`string Example:`"influxdb"` `status`string Allowed:`pass`, `fail` `version`string Example:`"1.8.10"` 503Server is unhealthy `message`string Example:`"ready for queries and writes"` `name`string Example:`"influxdb"` `status`string Allowed:`pass`, `fail` `version`string Example:`"1.8.10"` --- # Write Source: https://docs.influxdata.com/influxdb/v1/api/write/ [Download InfluxDB API Spec](/openapi/influxdb-oss-v1-openapi.yml) Write time series data to InfluxDB v1 in line protocol format using the `/write` endpoint. POST`/write` ### Write data Write time series data to InfluxDB using line protocol format. Data must be sent as binary encoded line protocol in the request body. Use the `--data-binary` flag with curl to preserve newlines. **Best Practices:** * Write points in batches of 5,000 to 10,000 for optimal performance * Use the least precise timestamp precision possible for better compression #### Parameters ##### Query parameters `db`requiredstring Target database for the write `rp`string Target retention policy. If not specified, writes to the default retention policy. `precision`string Timestamp precision. InfluxDB assumes nanoseconds if not specified. Allowed values:`n`, `u`, `ms`, `s`, `m`, `h` Default:`n` `u`string Username for authentication `p`string Password for authentication #### Request bodyrequired Line protocol data. Multiple points should be separated by newlines. Use `@filename` to write from a file. Content-Type:`application/json` Example request[Ask AI about this](#) ```sh curl --request POST \ "http://localhost:8086/write?db=DB" \ --header "Authorization: Bearer INFLUX_TOKEN" \ --header "Content-Type: text/plain" \ --data-raw 'measurement,tag=value field=1.0' ``` #### Responses 204Write successful 400Bad request (line protocol syntax error or type conflict) `error`requiredstring Error message 401Unauthorized `error`requiredstring Error message 404Database not found `error`requiredstring Error message 413Request entity too large `error`requiredstring Error message 500Internal server error (e.g., retention policy not found) `error`requiredstring Error message POST`/api/v2/write`v2 ### Write data (v2 compatible) Write data using the InfluxDB 2.x API format. This endpoint provides forward compatibility with InfluxDB 2.x client libraries. **Bucket Mapping:**The `bucket` parameter maps to InfluxDB 1.x database and retention policy: * `database/retention-policy` - specific retention policy * `database/` or `database` - default retention policy The `org` parameter is ignored in InfluxDB 1.x. #### Parameters ##### Query parameters `bucket`requiredstring Database and retention policy in format `database/retention-policy`. Use `database/` or `database` for the default retention policy. `org`string Organization (ignored in InfluxDB 1.x) `precision`string Timestamp precision Allowed values:`ns`, `us`, `ms`, `s` Default:`ns` #### Request bodyrequired Content-Type:`application/json` Example request[Ask AI about this](#) ```sh curl --request POST \ "http://localhost:8086/api/v2/write?bucket=BUCKET" \ --header "Authorization: Bearer INFLUX_TOKEN" \ --header "Content-Type: text/plain" \ --data-raw 'measurement,tag=value field=1.0' ``` #### Responses 204Write successful 400Bad request `error`requiredstring Error message 401Unauthorized `error`requiredstring Error message --- # InfluxDB concepts Source: https://docs.influxdata.com/influxdb/v1/concepts/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Understanding the following concepts will help you get the most out of InfluxDB. ## [InfluxDB file system layout](/influxdb/v1/concepts/file-system-layout/) The InfluxDB file system layout depends on the operating system, package manager, or containerization platform used to install InfluxDB. ## [Compare InfluxDB to SQL databases](/influxdb/v1/concepts/crosswalk/) Differences between InfluxDB and SQL databases. ## [In-memory indexing and the Time-Structured Merge Tree (TSM)](/influxdb/v1/concepts/storage_engine/) InfluxDB storage engine, in-memory indexing, and the Time-Structured Merge Tree (TSM) in InfluxDB OSS. ## [InfluxDB design insights and tradeoffs](/influxdb/v1/concepts/insights_tradeoffs/) Optimizing for time series use case entails some tradeoffs, primarily to increase performance at the cost of functionality. ## [InfluxDB glossary](/influxdb/v1/concepts/glossary/) Terms related to InfluxDB OSS. ## [InfluxDB key concepts](/influxdb/v1/concepts/key_concepts/) Covers key concepts to learn about InfluxDB. ## [InfluxDB schema design and data layout](/influxdb/v1/concepts/schema_and_data_layout/) Improve InfluxDB schema design and data layout to reduce high cardinality and make your data more performant. ## [Time Series Index (TSI) details](/influxdb/v1/concepts/tsi-details/) Enable and understand the Time Series Index (TSI). ## [Time Series Index (TSI) overview](/influxdb/v1/concepts/time-series-index/) The Time Series Index (TSI) storage engine supports high cardinality in time series data. --- # Compare InfluxDB to SQL databases Source: https://docs.influxdata.com/influxdb/v1/concepts/crosswalk/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxDB is similar to a SQL database, but different in many ways. InfluxDB is purpose-built for time series data. Relational databases *can* handle time series data, but are not optimized for common time series workloads. InfluxDB is designed to store large volumes of time series data and quickly perform real-time analysis on that data. ### Timing is everything In InfluxDB, a timestamp identifies a single point in any given data series. This is like an SQL database table where the primary key is pre-set by the system and is always time. InfluxDB also recognizes that your [schema](/influxdb/v1/concepts/glossary/#schema) preferences may change over time. In InfluxDB you don’t have to define schemas up front. Data points can have one of the fields on a measurement, all of the fields on a measurement, or any number in-between. You can add new fields to a measurement simply by writing a point for that new field. If you need an explanation of the terms measurements, tags, and fields check out the next section for an SQL database to InfluxDB terminology crosswalk. ## Terminology The table below is a (very) simple example of a table called `foodships` in an SQL database with the unindexed column `#_foodships` and the indexed columns `park_id`, `planet`, and `time`. ```sql +---------+---------+---------------------+--------------+ | park_id | planet | time | #_foodships | +---------+---------+---------------------+--------------+ | 1 | Earth | 1429185600000000000 | 0 | | 1 | Earth | 1429185601000000000 | 3 | | 1 | Earth | 1429185602000000000 | 15 | | 1 | Earth | 1429185603000000000 | 15 | | 2 | Saturn | 1429185600000000000 | 5 | | 2 | Saturn | 1429185601000000000 | 9 | | 2 | Saturn | 1429185602000000000 | 10 | | 2 | Saturn | 1429185603000000000 | 14 | | 3 | Jupiter | 1429185600000000000 | 20 | | 3 | Jupiter | 1429185601000000000 | 21 | | 3 | Jupiter | 1429185602000000000 | 21 | | 3 | Jupiter | 1429185603000000000 | 20 | | 4 | Saturn | 1429185600000000000 | 5 | | 4 | Saturn | 1429185601000000000 | 5 | | 4 | Saturn | 1429185602000000000 | 6 | | 4 | Saturn | 1429185603000000000 | 5 | +---------+---------+---------------------+--------------+ ``` Those same data look like this in InfluxDB: ```sql name: foodships tags: park_id=1, planet=Earth time #_foodships ---- ------------ 2015-04-16T12:00:00Z 0 2015-04-16T12:00:01Z 3 2015-04-16T12:00:02Z 15 2015-04-16T12:00:03Z 15 name: foodships tags: park_id=2, planet=Saturn time #_foodships ---- ------------ 2015-04-16T12:00:00Z 5 2015-04-16T12:00:01Z 9 2015-04-16T12:00:02Z 10 2015-04-16T12:00:03Z 14 name: foodships tags: park_id=3, planet=Jupiter time #_foodships ---- ------------ 2015-04-16T12:00:00Z 20 2015-04-16T12:00:01Z 21 2015-04-16T12:00:02Z 21 2015-04-16T12:00:03Z 20 name: foodships tags: park_id=4, planet=Saturn time #_foodships ---- ------------ 2015-04-16T12:00:00Z 5 2015-04-16T12:00:01Z 5 2015-04-16T12:00:02Z 6 2015-04-16T12:00:03Z 5 ``` Referencing the example above, in general: * An InfluxDB measurement (`foodships`) is similar to an SQL database table. * InfluxDB tags ( `park_id` and `planet`) are like indexed columns in an SQL database. * InfluxDB fields (`#_foodships`) are like unindexed columns in an SQL database. * InfluxDB points (for example, `2015-04-16T12:00:00Z 5`) are similar to SQL rows. Building on this comparison of database terminology, InfluxDB [continuous queries](/influxdb/v1/concepts/glossary/#continuous-query-cq)and [retention policies](/influxdb/v1/concepts/glossary/#retention-policy-rp) are similar to stored procedures in an SQL database. They’re specified once and then performed regularly and automatically. Of course, there are some major disparities between SQL databases and InfluxDB. SQL `JOIN`s aren’t available for InfluxDB measurements; your schema design should reflect that difference. And, as we mentioned above, a measurement is like an SQL table where the primary index is always pre-set to time. InfluxDB timestamps must be in UNIX epoch (GMT) or formatted as a date-time string valid under RFC3339. For more detailed descriptions of the InfluxDB terms mentioned in this section see our [Glossary of Terms](/influxdb/v1/concepts/glossary/). ## Query languages InfluxDB supports multiple query languages: * [Flux](#flux) * [InfluxQL](#influxql) ### Flux [Flux](/influxdb/v1/flux/) is a data scripting language designed for querying, analyzing, and acting on time series data. Beginning with **InfluxDB 1.8.0**, Flux is available for production use along side InfluxQL. For those familiar with [InfluxQL](#influxql), Flux is intended to address many of the outstanding feature requests that we’ve received since introducing InfluxDB 1.0. For a comparison between Flux and InfluxQL, see [Flux vs InfluxQL](/influxdb/v1/flux/flux-vs-influxql/). Flux is the primary language for working with data in [InfluxDB OSS 2.0](/influxdb/v2/get-started)and [InfluxDB Cloud](/influxdb/cloud/get-started/), a generally available Platform as a Service (PaaS) available across multiple Cloud Service Providers. Using Flux with InfluxDB 1.8+ lets you get familiar with Flux concepts and syntax and ease the transition to InfluxDB 2.0. ### InfluxQL InfluxQL is an SQL-like query language for interacting with InfluxDB. It has been crafted to feel familiar to those coming from other SQL or SQL-like environments while also providing features specific to storing and analyzing time series data. However **InfluxQL is not SQL** and lacks support for more advanced operations like `UNION`, `JOIN` and `HAVING` that SQL power-users are accustomed to. This functionality is available with [Flux](/flux/latest/introduction). InfluxQL’s `SELECT` statement follows the form of an SQL `SELECT` statement: ```sql SELECT FROM WHERE ``` where `WHERE` is optional. To get the InfluxDB output in the section above, you’d enter: ```sql SELECT * FROM "foodships" ``` If you only wanted to see data for the planet `Saturn`, you’d enter: ```sql SELECT * FROM "foodships" WHERE "planet" = 'Saturn' ``` If you wanted to see data for the planet `Saturn` after 12:00:01 UTC on April 16, 2015, you’d enter: ```sql SELECT * FROM "foodships" WHERE "planet" = 'Saturn' AND time > '2015-04-16 12:00:01' ``` As shown in the example above, InfluxQL allows you to specify the time range of your query in the `WHERE` clause. You can use date-time strings wrapped in single quotes that have the format `YYYY-MM-DD HH:MM:SS.mmm`(`mmm` is milliseconds and is optional, and you can also specify microseconds or nanoseconds). You can also use relative time with `now()` which refers to the server’s current timestamp: ```sql SELECT * FROM "foodships" WHERE time > now() - 1h ``` That query outputs the data in the `foodships` measure where the timestamp is newer than the server’s current time minus one hour. The options for specifying time durations with `now()` are: |Letter| Meaning | |------|------------| | ns |nanoseconds | |u or µ|microseconds| | ms |milliseconds| | s | seconds | | m | minutes | | h | hours | | d | days | | w | weeks | InfluxQL also supports regular expressions, arithmetic in expressions, `SHOW` statements, and `GROUP BY` statements. See our [data exploration](/influxdb/v1/query_language/explore-data/) page for an in-depth discussion of those topics. InfluxQL functions include `COUNT`, `MIN`, `MAX`, `MEDIAN`, `DERIVATIVE` and more. For a full list check out the [functions](/influxdb/v1/query_language/functions/) page. Now that you have the general idea, check out our [Getting Started Guide](/influxdb/v1/introduction/getting-started/). ## InfluxDB is not CRUD InfluxDB is a database that has been optimized for time series data. This data commonly comes from sources like distributed sensor groups, click data from large websites, or lists of financial transactions. One thing this data has in common is that it is more useful in the aggregate. One reading saying that your computer’s CPU is at 12% utilization at 12:38:35 UTC on a Tuesday is hard to draw conclusions from. It becomes more useful when combined with the rest of the series and visualized. This is where trends over time begin to show, and actionable insight can be drawn from the data. In addition, time series data is generally written once and rarely updated. The result is that InfluxDB is not a full CRUD database but more like a CR-ud, prioritizing the performance of creating and reading data over update and destroy, and [preventing some update and destroy behaviors](/influxdb/v1/concepts/insights_tradeoffs/) to make create and read more performant: * To update a point, insert one with [the same measurement, tag set, and timestamp](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-does-influxdb-handle-duplicate-points). * You can [drop or delete a series](/influxdb/v1/query_language/manage-database/#drop-series-from-the-index-with-drop-series), but not individual points based on field values. As a workaround, you can search for the field value, retrieve the time, then [DELETE based on the `time` field](/influxdb/v1/query_language/manage-database/#delete-series-with-delete). * You can’t update or rename tags yet - see GitHub issue [#4157](https://github.com/influxdata/influxdb/issues/4157) for more information. To modify the tag of a series of points, find the points with the offending tag value, change the value to the desired one, write the points back, then drop the series with the old tag value. * You can’t delete tags by tag key (as opposed to value) - see GitHub issue [#8604](https://github.com/influxdata/influxdb/issues/8604). | Letter | Meaning | | --- | --- | | Letter | Meaning | | ns | nanoseconds | | u or µ | microseconds | | ms | milliseconds | | s | seconds | | m | minutes | | h | hours | | d | days | | w | weeks | --- # InfluxDB file system layout Source: https://docs.influxdata.com/influxdb/v1/concepts/file-system-layout/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The InfluxDB file system layout depends on the operating system, installation method, or containerization platform used to install InfluxDB. * [InfluxDB file structure](#influxdb-file-structure) * [File system layout](#file-system-layout) ## InfluxDB file structure The InfluxDB file structure includes of the following: ### Data directory Directory path where InfluxDB stores time series data (TSM files). To customize this path, use the [`[data].dir`](/influxdb/v1/administration/config/#dir-1)configuration option. ### WAL directory Directory path where InfluxDB stores Write Ahead Log (WAL) files. To customize this path, use the [`[data].wal-dir`](/influxdb/v1/administration/config/#wal-dir)configuration option. ### Metastore directory Directory path of the InfluxDB metastore, which stores information about users, databases, retention policies, shards, and continuous queries. To customize this path, use the [`[meta].dir`](/influxdb/v1/administration/config/#dir)configuration option. ## InfluxDB configuration files Some operating systems and package managers store a default InfluxDB configuration file on disk. For more information about using InfluxDB configuration files, see[Configure InfluxDB](/influxdb/v1/administration/config/). ## File system layout [macOS](#macos)[Linux](#linux)[Windows](#windows)[Docker](#docker)[Kubernetes](#kubernetes) #### macOS default directories | Path | Default | |-------------------------------------------|-------------------| | [Data directory](#data-directory) |`~/.influxdb/data/`| | [WAL directory](#wal-directory) |`~/.influxdb/wal/` | |[Metastore directory](#metastore-directory)|`~/.influxdb/meta/`| #### Homebrew default directories | Path | Default | |-------------------------------------------|----------------------------------| | [Data directory](#data-directory) |`/opt/homebrew/var/influxdb/data/`| | [WAL directory](#wal-directory) |`/opt/homebrew/var/influxdb/wal/` | |[Metastore directory](#metastore-directory)| `/opt/homebrew/var/meta/` | #### macOS file system overview * \~/.influxdb/ * data/ * *TSM directories and files* * wal/ * *WAL directories and files* * meta/ * meta.db When installing InfluxDB on Linux, you can download and install the `influxd` binary, or you can use a package manager. Which installation method you use determines the file system layout. * [Installed as a standalone binary](#installed-as-a-standalone-binary) * [Installed as a package](#installed-as-a-package) ### Installed as a standalone binary #### Linux default directories (standalone binary) | Path | Default | |-------------------------------------------|-------------------| | [Data directory](#data-directory) |`~/.influxdb/data/`| | [WAL directory](#wal-directory) |`~/.influxdb/wal/` | |[Metastore directory](#metastore-directory)|`~/.influxdb/meta/`| #### Linux file system overview (standalone binary) * \~/.influxdb/ * data/ * *TSM directories and files* * wal/ * *WAL directories and files* * meta/ * meta.db ### Installed as a package InfluxDB 2.0 supports **.deb-** and **.rpm-based** Linux package managers. The file system layout is the same with each. #### Linux default directories (package) | Path | Default | |---------------------------------------------------------|-----------------------------| | [Data directory](#data-directory) | `/var/lib/influxdb/data/` | | [WAL directory](#wal-directory) | `/var/lib/influxdb/wal/` | | [Metastore directory](#metastore-directory) | `/var/lib/influxdb/meta/` | |[Default config file path](#influxdb-configuration-files)|`/etc/influxdb/influxdb.conf`| #### Linux file system overview (package) * /var/lib/influxdb/ * data/ * *TSM directories and files* * wal/ * *WAL directories and files* * meta/ * meta.db * /etc/influxdb/ * influxdb.conf *(influxd configuration file)* #### Windows default paths | Path | Default | |-------------------------------------------|-------------------------------| | [Data directory](#data-directory) |`%USERPROFILE%\.influxdb\data\`| | [WAL directory](#wal-directory) |`%USERPROFILE%\.influxdb\wal\` | |[Metastore directory](#metastore-directory)|`%USERPROFILE%\.influxdb\meta\`| #### Windows file system overview * %USERPROFILE%\\.influxdb\\ * data/ * *TSM directories and files* * wal/ * *WAL directories and files* * meta/ * meta.db #### Docker default directories | Path | Default | |-------------------------------------------|-------------------------| | [Data directory](#data-directory) |`/var/lib/influxdb/data/`| | [WAL directory](#wal-directory) |`/var/lib/influxdb/wal/` | |[Metastore directory](#metastore-directory)|`/var/lib/influxdb/meta/`| #### Docker Hub file system overview * /var/lib/influxdb/ * data/ * *TSM directories and files* * wal/ * *WAL directories and files* * meta/ * meta.db #### Kubernetes default paths | Path | Default | |-------------------------------------------|-------------------------| | [Data directory](#data-directory) |`/var/lib/influxdb/data/`| | [WAL directory](#wal-directory) |`/var/lib/influxdb/wal/` | |[Metastore directory](#metastore-directory)|`/var/lib/influxdb/meta/`| #### Kubernetes file system overview * /var/lib/influxdb/ * data/ * *TSM directories and files* * wal/ * *WAL directories and files* * meta/ * meta.db ## Directory permissions The user running the `influxd` process should have the following permissions for directories in the [InfluxDB file system](#file-system-layout): | Directory path |Permission| |--------------------|----------| | `.../influxdb/` | 755 | |`.../influxdb/data/`| 755 | |`.../influxdb/meta/`| 755 | |`.../influxdb/wal/` | 700 | | Path | Default | | --- | --- | | Path | Default | | Data directory | ~/.influxdb/data/ | | WAL directory | ~/.influxdb/wal/ | | Metastore directory | ~/.influxdb/meta/ | | Path | Default | | --- | --- | | Path | Default | | Data directory | /opt/homebrew/var/influxdb/data/ | | WAL directory | /opt/homebrew/var/influxdb/wal/ | | Metastore directory | /opt/homebrew/var/meta/ | | Path | Default | | --- | --- | | Path | Default | | Data directory | ~/.influxdb/data/ | | WAL directory | ~/.influxdb/wal/ | | Metastore directory | ~/.influxdb/meta/ | | Path | Default | | --- | --- | | Path | Default | | Data directory | /var/lib/influxdb/data/ | | WAL directory | /var/lib/influxdb/wal/ | | Metastore directory | /var/lib/influxdb/meta/ | | Default config file path | /etc/influxdb/influxdb.conf | | Path | Default | | --- | --- | | Path | Default | | Data directory | %USERPROFILE%\.influxdb\data\ | | WAL directory | %USERPROFILE%\.influxdb\wal\ | | Metastore directory | %USERPROFILE%\.influxdb\meta\ | | Path | Default | | --- | --- | | Path | Default | | Data directory | /var/lib/influxdb/data/ | | WAL directory | /var/lib/influxdb/wal/ | | Metastore directory | /var/lib/influxdb/meta/ | | Path | Default | | --- | --- | | Path | Default | | Data directory | /var/lib/influxdb/data/ | | WAL directory | /var/lib/influxdb/wal/ | | Metastore directory | /var/lib/influxdb/meta/ | | Directory path | Permission | | --- | --- | | Directory path | Permission | | .../influxdb/ | 755 | | .../influxdb/data/ | 755 | | .../influxdb/meta/ | 755 | | .../influxdb/wal/ | 700 | --- # InfluxDB glossary Source: https://docs.influxdata.com/influxdb/v1/concepts/glossary/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## aggregation An InfluxQL function that returns an aggregated value across a set of points. For a complete list of the available and upcoming aggregations, see [InfluxQL functions](/influxdb/v1/query_language/functions/#aggregations). Related entries: [function](#function), [selector](#selector), [transformation](#transformation) ## batch A collection of data points in InfluxDB line protocol format, separated by newlines (`0x0A`). A batch of points may be submitted to the database using a single HTTP request to the write endpoint. This makes writes using the InfluxDB API much more performant by drastically reducing the HTTP overhead. InfluxData recommends batch sizes of 5,000-10,000 points, although different use cases may be better served by significantly smaller or larger batches. Related entries: [InfluxDB line protocol](#influxdb-line-protocol), [point](#point) ## bucket A bucket is a named location where time series data is stored in **InfluxDB 2.0**. In InfluxDB 1.8+, each combination of a database and a retention policy (database/retention-policy) represents a bucket. Use the [InfluxDB 2.x API compatibility endpoints](/influxdb/v1/tools/api#influxdb-2-0-api-compatibility-endpoints) included with InfluxDB 1.8+ to interact with buckets. ## continuous query (CQ) An InfluxQL query that runs automatically and periodically within a database. Continuous queries require a function in the `SELECT` clause and must include a `GROUP BY time()` clause. See [Continuous Queries](/influxdb/v1/query_language/continuous_queries/). Related entries: [function](#function) ## database A logical container for users, retention policies, continuous queries, and time series data. Related entries: [continuous query](#continuous-query-cq), [retention policy](#retention-policy-rp), [user](#user) ## duration The attribute of the retention policy that determines how long InfluxDB stores data. Data older than the duration are automatically dropped from the database. See [Database Management](/influxdb/v1/query_language/manage-database/#create-retention-policies-with-create-retention-policy) for how to set duration. Related entries: [retention policy](#retention-policy-rp) ## field The key-value pair in an InfluxDB data structure that records metadata and the actual data value. Fields are required in InfluxDB data structures and they are not indexed - queries on field values scan all points that match the specified time range and, as a result, are not performant relative to tags. *Query tip:* Compare fields to tags; tags are indexed. Related entries: [field key](#field-key), [field set](#field-set), [field value](#field-value), [tag](#tag) ## field key The key part of the key-value pair that makes up a field. Field keys are strings and they store metadata. Related entries: [field](#field), [field set](#field-set), [field value](#field-value), [tag key](#tag-key) ## field set The collection of field keys and field values on a point. Related entries: [field](#field), [field key](#field-key), [field value](#field-value), [point](#point) ## field value The value part of the key-value pair that makes up a field. Field values are the actual data; they can be strings, floats, integers, or booleans. A field value is always associated with a timestamp. Field values are not indexed - queries on field values scan all points that match the specified time range and, as a result, are not performant. *Query tip:* Compare field values to tag values; tag values are indexed. Related entries: [field](#field), [field key](#field-key), [field set](#field-set), [tag value](#tag-value), [timestamp](#timestamp) ## function InfluxQL aggregations, selectors, and transformations. See [InfluxQL Functions](/influxdb/v1/query_language/functions/) for a complete list of InfluxQL functions. Related entries: [aggregation](#aggregation), [selector](#selector), [transformation](#transformation) ## identifier Tokens that refer to continuous query names, database names, field keys, measurement names, retention policy names, subscription names, tag keys, and user names. See [Query Language Specification](/influxdb/v1/query_language/spec/#identifiers). Related entries:[database](#database),[field key](#field-key),[measurement](#measurement),[retention policy](#retention-policy-rp),[tag key](#tag-key),[user](#user) ## InfluxDB line protocol The text based format for writing points to InfluxDB. See [InfluxDB line protocol](/influxdb/v1/write_protocols/). ## measurement The part of the InfluxDB data structure that describes the data stored in the associated fields. Measurements are strings. Related entries: [field](#field), [series](#series) ## metastore Contains internal information about the status of the system. The metastore contains the user information, databases, retention policies, shard metadata, continuous queries, and subscriptions. Related entries: [database](#database), [retention policy](#retention-policy-rp), [user](#user) ## node An independent `influxd` process. Related entries: [server](#server) ## now() The local server’s nanosecond timestamp. ## point In InfluxDB, a point represents a single data record, similar to a row in a SQL database table. Each point: * has a measurement, a tag set, a field key, a field value, and a timestamp; * is uniquely identified by its series and timestamp. You cannot store more than one point with the same timestamp in a series. If you write a point to a series with a timestamp that matches an existing point, the field set becomes a union of the old and new field set, and any ties go to the new field set. For more information about duplicate points, see [How does InfluxDB handle duplicate points?](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-does-influxdb-handle-duplicate-points) Related entries: [field set](#field-set), [series](#series), [timestamp](#timestamp) ## points per second A deprecated measurement of the rate at which data is persisted to InfluxDB. The schema allows and even encourages the recording of multiple metric values per point, rendering points per second ambiguous. Write speeds are generally quoted in values per second, a more precise metric. Related entries: [point](#point), [schema](#schema), [values per second](#values-per-second) ## query An operation that retrieves data from InfluxDB. See [Data Exploration](/influxdb/v1/query_language/explore-data/), [Schema Exploration](/influxdb/v1/query_language/explore-schema/), [Database Management](/influxdb/v1/query_language/manage-database/). ## replication factor (RF) The attribute of the retention policy that determines how many copies of data to concurrently store (or retain) in a cluster. Replicating copies ensures that data is available when one or more data nodes are unavailable. Replication requires multiple data nodes, which are available only in[InfluxDB Enterprise](/enterprise_influxdb/v1/) clusters. InfluxDB OSS runs as a single node, so the replication factor has no effect – there is only ever one copy of your data. For default replication factor behavior in a cluster, see the InfluxDB Enterprise[`CREATE RETENTION POLICY` reference](/enterprise_influxdb/v1/query_language/manage-database/#replication). Related entries: [duration](#duration), [node](#node),[retention policy](#retention-policy-rp) ## retention policy (RP) Describes how long InfluxDB keeps data (duration), how many copies of the data to store in the cluster (replication factor), and the time range covered by shard groups (shard group duration). RPs are unique per database and along with the measurement and tag set define a series. When you create a database, InfluxDB creates a retention policy called `autogen` with an infinite duration, a replication factor set to one, and a shard group duration set to seven days. For more information, see [Retention policy management](/influxdb/v1/query_language/manage-database/#retention-policy-management). Related entries: [duration](#duration), [measurement](#measurement), [replication factor](#replication-factor-rf), [series](#series), [shard duration](#shard-duration), [tag set](#tag-set) ## schema How the data is organized in InfluxDB. The fundamentals of the InfluxDB schema are databases, retention policies, series, measurements, tag keys, tag values, and field keys. See [Schema Design](/influxdb/v1/concepts/schema_and_data_layout/) for more information. Related entries: [database](#database), [field key](#field-key), [measurement](#measurement), [retention policy](#retention-policy-rp), [series](#series), [tag key](#tag-key), [tag value](#tag-value) ## selector An InfluxQL function that returns a single point from the range of specified points. See [InfluxQL Functions](/influxdb/v1/query_language/functions/#selectors) for a complete list of the available and upcoming selectors. Related entries: [aggregation](#aggregation), [function](#function), [transformation](#transformation) ## series A logical grouping of data defined by shared measurement, tag set, and field key. Related entries: [field set](#field-set), [measurement](#measurement), [tag set](#tag-set) ## series cardinality The number of unique database, measurement, tag set, and field key combinations in an InfluxDB instance. For example, assume that an InfluxDB instance has a single database and one measurement. The single measurement has two tag keys: `email` and `status`. If there are three different `email`s, and each email address is associated with two different `status`es then the series cardinality for the measurement is 6 (3 \* 2 = 6): | email |status| |--------------------|------| |lorr@influxdata.com |start | |lorr@influxdata.com |finish| |marv@influxdata.com |start | |marv@influxdata.com |finish| |cliff@influxdata.com|start | |cliff@influxdata.com|finish| Note that, in some cases, simply performing that multiplication may overestimate series cardinality because of the presence of dependent tags. Dependent tags are tags that are scoped by another tag and do not increase series cardinality. If we add the tag `firstname` to the example above, the series cardinality would not be 18 (3 \* 2 \* 3 = 18). It would remain unchanged at 6, as `firstname` is already scoped by the `email` tag: | email |status|firstname| |--------------------|------|---------| |lorr@influxdata.com |start |lorraine | |lorr@influxdata.com |finish|lorraine | |marv@influxdata.com |start | marvin | |marv@influxdata.com |finish| marvin | |cliff@influxdata.com|start |clifford | |cliff@influxdata.com|finish|clifford | See [SHOW CARDINALITY](/influxdb/v1/query_language/spec/#show-cardinality) to learn about the InfluxQL commands for series cardinality. Related entries: [field key](#field-key),[measurement](#measurement), [tag key](#tag-key), [tag set](#tag-set) ## series key A series key identifies a particular series by measurement, tag set, and field key. For example: ``` # measurement, tag set, field key h2o_level, location=santa_monica, h2o_feet ``` Related entries: [series](#series) ## server A machine, virtual or physical, that is running InfluxDB. There should only be one InfluxDB process per server. Related entries: [node](#node) ## shard A shard contains the actual encoded and compressed data, and is represented by a TSM file on disk. Every shard belongs to one and only one shard group. Multiple shards may exist in a single shard group. Each shard contains a specific set of series. All points falling on a given series in a given shard group will be stored in the same shard (TSM file) on disk. Related entries: [series](#series), [shard duration](#shard-duration), [shard group](#shard-group), [tsm](#tsm-time-structured-merge-tree) ## shard duration The shard duration determines how much time each shard group spans. The specific interval is determined by the `SHARD DURATION` of the retention policy. See [Retention Policy management](/influxdb/v1/query_language/manage-database/#retention-policy-management) for more information. For example, given a retention policy with `SHARD DURATION` set to `1w`, each shard group will span a single week and contain all points with timestamps in that week. Related entries: [database](#database), [retention policy](#retention-policy-rp), [series](#series), [shard](#shard), [shard group](#shard-group) ## shard group Shard groups are logical containers for shards. Shard groups are organized by time and retention policy. Every retention policy that contains data has at least one associated shard group. A given shard group contains all shards with data for the interval covered by the shard group. The interval spanned by each shard group is the shard duration. Related entries: [database](#database), [retention policy](#retention-policy-rp), [series](#series), [shard](#shard), [shard duration](#shard-duration) ## subscription Subscriptions allow [Kapacitor](/kapacitor/v1/) to receive data from InfluxDB in a push model rather than the pull model based on querying data. When Kapacitor is configured to work with InfluxDB, the subscription will automatically push every write for the subscribed database from InfluxDB to Kapacitor. Subscriptions can use TCP or UDP for transmitting the writes. ## tag The key-value pair in the InfluxDB data structure that records metadata. Tags are an optional part of the data structure, but they are useful for storing commonly queried metadata; tags are indexed so queries on tags are performant.*Query tip:* Compare tags to fields; fields are not indexed. Related entries: [field](#field), [tag key](#tag-key), [tag set](#tag-set), [tag value](#tag-value) ## tag key The key part of the key-value pair that makes up a tag. Tag keys are strings and they store metadata. Tag keys are indexed so queries on tag keys are performant. *Query tip:* Compare tag keys to field keys; field keys are not indexed. Related entries: [field key](#field-key), [tag](#tag), [tag set](#tag-set), [tag value](#tag-value) ## tag set The collection of tag keys and tag values on a point. Related entries: [point](#point), [series](#series), [tag](#tag), [tag key](#tag-key), [tag value](#tag-value) ## tag value The value part of the key-value pair that makes up a tag. Tag values are strings and they store metadata. Tag values are indexed so queries on tag values are performant. Related entries: [tag](#tag), [tag key](#tag-key), [tag set](#tag-set) ## timestamp The date and time associated with a point. All time in InfluxDB is UTC. For how to specify time when writing data, see [Write Syntax](/influxdb/v1/write_protocols/write_syntax/). For how to specify time when querying data, see [Data Exploration](/influxdb/v1/query_language/explore-data/#time-syntax). Related entries: [point](#point) ## transformation An InfluxQL function that returns a value or a set of values calculated from specified points, but does not return an aggregated value across those points. See [InfluxQL Functions](/influxdb/v1/query_language/functions/#transformations) for a complete list of the available and upcoming aggregations. Related entries: [aggregation](#aggregation), [function](#function), [selector](#selector) ## TSM (Time Structured Merge tree) The purpose-built data storage format for InfluxDB. TSM allows for greater compaction and higher write and read throughput than existing B+ or LSM tree implementations. See [Storage Engine](/influxdb/v1/concepts/storage_engine/) for more. ## user There are two kinds of users in InfluxDB: * *Admin users* have `READ` and `WRITE` access to all databases and full access to administrative queries and user management commands. * *Non-admin users* have `READ`, `WRITE`, or `ALL` (both `READ` and `WRITE`) access per database. When authentication is enabled, InfluxDB only executes HTTP requests that are sent with a valid username and password. See [Authentication and Authorization](/influxdb/v1/administration/authentication_and_authorization/). ## values per second The preferred measurement of the rate at which data is persisted to InfluxDB. Write speeds are generally quoted in values per second. To calculate the values per second rate, multiply the number of points written per second by the number of values stored per point. For example, if the points have four fields each, and a batch of 5000 points is written 10 times per second, then the values per second rate is `4 field values per point * 5000 points per batch * 10 batches per second = 200,000 values per second`. Related entries: [batch](#batch), [field](#field), [point](#point), [points per second](#points-per-second) ## WAL (Write Ahead Log) The temporary cache for recently written points. To reduce the frequency with which the permanent storage files are accessed, InfluxDB caches new points in the WAL until their total size or age triggers a flush to more permanent storage. This allows for efficient batching of the writes into the TSM. Points in the WAL can be queried, and they persist through a system reboot. On process start, all points in the WAL must be flushed before the system accepts new writes. Related entries: [tsm](#tsm-time-structured-merge-tree) | email | status | | --- | --- | | email | status | | lorr@influxdata.com | start | | lorr@influxdata.com | finish | | marv@influxdata.com | start | | marv@influxdata.com | finish | | cliff@influxdata.com | start | | cliff@influxdata.com | finish | | email | status | firstname | | --- | --- | --- | | email | status | firstname | | lorr@influxdata.com | start | lorraine | | lorr@influxdata.com | finish | lorraine | | marv@influxdata.com | start | marvin | | marv@influxdata.com | finish | marvin | | cliff@influxdata.com | start | clifford | | cliff@influxdata.com | finish | clifford | --- # InfluxDB design insights and tradeoffs Source: https://docs.influxdata.com/influxdb/v1/concepts/insights_tradeoffs/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxDB is a time series database. Optimizing for this use case entails some tradeoffs, primarily to increase performance at the cost of functionality. Below is a list of some of those design insights that lead to tradeoffs: 1. For the time series use case, we assume that if the same data is sent multiple times, it is the exact same data that a client just sent several times. ***Pro:*** Simplified [conflict resolution](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-does-influxdb-handle-duplicate-points) increases write performance. ***Con:*** Cannot store duplicate data; may overwrite data in rare circumstances. 2. Deletes are a rare occurrence. When they do occur it is almost always against large ranges of old data that are cold for writes. ***Pro:*** Restricting access to deletes allows for increased query and write performance. ***Con:*** Delete functionality is significantly restricted. 3. Updates to existing data are a rare occurrence and contentious updates never happen. Time series data is predominantly new data that is never updated. ***Pro:*** Restricting access to updates allows for increased query and write performance. ***Con:*** Update functionality is significantly restricted. 4. The vast majority of writes are for data with very recent timestamps and the data is added in time ascending order. ***Pro:*** Adding data in time ascending order is significantly more performant. ***Con:*** Writing points with random times or with time not in ascending order is significantly less performant. 5. Scale is critical. The database must be able to handle a *high* volume of reads and writes. ***Pro:*** The database can handle a *high* volume of reads and writes. ***Con:*** The InfluxDB development team was forced to make tradeoffs to increase performance. 6. Being able to write and query the data is more important than having a strongly consistent view. ***Pro:*** Writing and querying the database can be done by multiple clients and at high loads. ***Con:*** Query returns may not include the most recent points if database is under heavy load. 7. Many time [series](/influxdb/v1/concepts/glossary/#series) are ephemeral. There are often time series that appear only for a few hours and then go away, for example. a new host that gets started and reports for a while and then gets shut down. ***Pro:*** InfluxDB is good at managing discontinuous data. ***Con:*** Schema-less design means that some database functions are not supported, for example, there are no cross table joins. 8. No one point is too important. ***Pro:*** InfluxDB has very powerful tools to deal with aggregate data and large data sets. ***Con:*** Points don’t have IDs in the traditional sense, they are differentiated by timestamp and series. --- # InfluxDB key concepts Source: https://docs.influxdata.com/influxdb/v1/concepts/key_concepts/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Before diving into InfluxDB, it’s good to get acquainted with some key concepts of the database. This document introduces key InfluxDB concepts and elements. To introduce the key concepts, we’ll cover how the following elements work together in InfluxDB: * [database](/influxdb/v1/concepts/glossary/#database) * [field key](/influxdb/v1/concepts/glossary/#field-key) * [field set](/influxdb/v1/concepts/glossary/#field-set) * [field value](/influxdb/v1/concepts/glossary/#field-value) * [measurement](/influxdb/v1/concepts/glossary/#measurement) * [point](/influxdb/v1/concepts/glossary/#point) * [retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp) * [series](/influxdb/v1/concepts/glossary/#series) * [tag key](/influxdb/v1/concepts/glossary/#tag-key) * [tag set](/influxdb/v1/concepts/glossary/#tag-set) * [tag value](/influxdb/v1/concepts/glossary/#tag-value) * [timestamp](/influxdb/v1/concepts/glossary/#timestamp) ### Sample data The next section references the data printed out below. The data is fictional, but represents a believable setup in InfluxDB. They show the number of butterflies and honeybees counted by two scientists (`langstroth` and `perpetua`) in two locations (location `1` and location `2`) over the time period from August 18, 2015 at midnight through August 18, 2015 at 6:12 AM. Assume that the data lives in a database called `my_database` and are subject to the `autogen` retention policy (more on databases and retention policies to come). *Hint:* Hover over the links for tooltips to get acquainted with InfluxDB terminology and the layout. **name:** census | time |butterflies|honeybees|location|scientist | |--------------------|-----------|---------|--------|----------| |2015-08-18T00:00:00Z| 12 | 23 | 1 |langstroth| |2015-08-18T00:00:00Z| 1 | 30 | 1 | perpetua | |2015-08-18T00:06:00Z| 11 | 28 | 1 |langstroth| |2015-08-18T00:06:00Z| 3 | 28 | 1 | perpetua | |2015-08-18T05:54:00Z| 2 | 11 | 2 |langstroth| |2015-08-18T06:00:00Z| 1 | 10 | 2 |langstroth| |2015-08-18T06:06:00Z| 8 | 23 | 2 | perpetua | |2015-08-18T06:12:00Z| 7 | 22 | 2 | perpetua | ### Discussion Now that you’ve seen some sample data in InfluxDB this section covers what it all means. InfluxDB is a time series database so it makes sense to start with what is at the root of everything we do: time. In the data above there’s a column called `time` - all data in InfluxDB have that column.`time` stores timestamps, and the ***timestamp*** shows the date and time, in [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) UTC, associated with particular data. The next two columns, called `butterflies` and `honeybees`, are fields. Fields are made up of field keys and field values.***Field keys*** (`butterflies` and `honeybees`) are strings; the field key `butterflies` tells us that the field values `12`-`7` refer to butterflies and the field key `honeybees` tells us that the field values `23`-`22` refer to, well, honeybees. ***Field values*** are your data; they can be strings, floats, integers, or Booleans, and, because InfluxDB is a time series database, a field value is always associated with a timestamp. The field values in the sample data are: ``` 12 23 1 30 11 28 3 28 2 11 1 10 8 23 7 22 ``` In the data above, the collection of field-key and field-value pairs make up a ***field set***. Here are all eight field sets in the sample data: * `butterflies = 12 honeybees = 23` * `butterflies = 1 honeybees = 30` * `butterflies = 11 honeybees = 28` * `butterflies = 3 honeybees = 28` * `butterflies = 2 honeybees = 11` * `butterflies = 1 honeybees = 10` * `butterflies = 8 honeybees = 23` * `butterflies = 7 honeybees = 22` Fields are a required piece of the InfluxDB data structure - you cannot have data in InfluxDB without fields. It’s also important to note that fields are not indexed.[Queries](/influxdb/v1/concepts/glossary/#query) that use field values as filters must scan all values that match the other conditions in the query. As a result, those queries are not performant relative to queries on tags (more on tags below). In general, fields should not contain commonly queried metadata. The last two columns in the sample data, called `location` and `scientist`, are tags. Tags are made up of tag keys and tag values. Both ***tag keys*** and ***tag values*** are stored as strings and record metadata. The tag keys in the sample data are `location` and `scientist`. The tag key `location` has two tag values: `1` and `2`. The tag key `scientist` also has two tag values: `langstroth` and `perpetua`. In the data above, the ***tag set*** is the different combinations of all the tag key-value pairs. The four tag sets in the sample data are: * `location = 1`, `scientist = langstroth` * `location = 2`, `scientist = langstroth` * `location = 1`, `scientist = perpetua` * `location = 2`, `scientist = perpetua` Tags are optional. You don’t need to have tags in your data structure, but it’s generally a good idea to make use of them because, unlike fields, tags are indexed. This means that queries on tags are faster and that tags are ideal for storing commonly queried metadata. Avoid using the following reserved keys: * `_field` * `_measurement` * `time` If reserved keys are included as a tag or field key, the associated point is discarded. > [!Note] > #### Why indexing matters: The schema case study > > Say you notice that most of your queries focus on the values of the field keys `honeybees` and `butterflies`: > > ``` > SELECT * FROM "census" WHERE "butterflies" = 1 > SELECT * FROM "census" WHERE "honeybees" = 23 > ``` > > Because fields aren’t indexed, InfluxDB scans every value of `butterflies` in the first query and every value of `honeybees` in the second query before it provides a response. > That behavior can hurt query response times - especially on a much larger scale. > To optimize your queries, it may be beneficial to rearrange your [schema](/influxdb/v1/concepts/glossary/#schema) such that the fields (`butterflies` and `honeybees`) become the tags and the tags (`location` and `scientist`) become the fields: > > **name:** census > > | time |location|scientist |butterflies|honeybees| > |--------------------|--------|----------|-----------|---------| > |2015-08-18T00:00:00Z| 1 |langstroth| 12 | 23 | > |2015-08-18T00:00:00Z| 1 | perpetua | 1 | 30 | > |2015-08-18T00:06:00Z| 1 |langstroth| 11 | 28 | > |2015-08-18T00:06:00Z| 1 | perpetua | 3 | 28 | > |2015-08-18T05:54:00Z| 2 |langstroth| 2 | 11 | > |2015-08-18T06:00:00Z| 2 |langstroth| 1 | 10 | > |2015-08-18T06:06:00Z| 2 | perpetua | 8 | 23 | > |2015-08-18T06:12:00Z| 2 | perpetua | 7 | 22 | > > Now that `butterflies` and `honeybees` are tags, InfluxDB won’t have to scan every one of their values when it performs the queries above - this means that your queries are even faster. The ***measurement*** acts as a container for tags, fields, and the `time` column, and the measurement name is the description of the data that are stored in the associated fields. Measurement names are strings, and, for any SQL users out there, a measurement is conceptually similar to a table. The only measurement in the sample data is `census`. The name `census` tells us that the field values record the number of `butterflies` and `honeybees` - not their size, direction, or some sort of happiness index. A single measurement can belong to different retention policies. A ***retention policy*** describes how long InfluxDB keeps data (`DURATION`) and how many copies of this data is stored in the cluster (`REPLICATION`). If you’re interested in reading more about retention policies, check out [Database Management](/influxdb/v1/query_language/manage-database/#retention-policy-management). Replication factors do not serve a purpose with single node instances. In the sample data, everything in the `census` measurement belongs to the `autogen` retention policy. InfluxDB automatically creates that retention policy; it has an infinite duration and a replication factor set to one. Now that you’re familiar with measurements, tag sets, and retention policies, let’s discuss series. In InfluxDB, a ***series*** is a collection of points that share a measurement, tag set, and field key. The data above consist of eight series: |Series number|Measurement| Tag set | Field key | |-------------|-----------|---------------------------------------|-------------| | series 1 | `census` |`location = 1`,`scientist = langstroth`|`butterflies`| | series 2 | `census` |`location = 2`,`scientist = langstroth`|`butterflies`| | series 3 | `census` | `location = 1`,`scientist = perpetua` |`butterflies`| | series 4 | `census` | `location = 2`,`scientist = perpetua` |`butterflies`| | series 5 | `census` |`location = 1`,`scientist = langstroth`| `honeybees` | | series 6 | `census` |`location = 2`,`scientist = langstroth`| `honeybees` | | series 7 | `census` | `location = 1`,`scientist = perpetua` | `honeybees` | | series 8 | `census` | `location = 2`,`scientist = perpetua` | `honeybees` | Understanding the concept of a series is essential when designing your [schema](/influxdb/v1/concepts/glossary/#schema) and when working with your data in InfluxDB. A ***point*** represents a single data record that has four components: a measurement, tag set, field set, and a timestamp. A point is uniquely identified by its series and timestamp. For example, here’s a single point: ``` ## name: census time butterflies honeybees location scientist 2015-08-18T00:00:00Z 1 30 1 perpetua ``` The point in this example is part of series 3 and 7 and defined by the measurement (`census`), the tag set (`location = 1`, `scientist = perpetua`), the field set (`butterflies = 1`, `honeybees = 30`), and the timestamp `2015-08-18T00:00:00Z`. All of the stuff we’ve just covered is stored in a database - the sample data are in the database `my_database`. An InfluxDB ***database*** is similar to traditional relational databases and serves as a logical container for users, retention policies, continuous queries, and, of course, your time series data. See [Authentication and Authorization](/influxdb/v1/administration/authentication_and_authorization/) and [Continuous Queries](/influxdb/v1/query_language/continuous_queries/) for more on those topics. Databases can have several users, continuous queries, retention policies, and measurements. InfluxDB is a schemaless database which means it’s easy to add new measurements, tags, and fields at any time. It’s designed to make working with time series data awesome. You made it! You’ve covered the fundamental concepts and terminology in InfluxDB. If you’re just starting out, we recommend taking a look at [Getting Started](/influxdb/v1/introduction/getting_started/) and the [Writing Data](/influxdb/v1/guides/writing_data/) and [Querying Data](/influxdb/v1/guides/querying_data/) guides. May our time series database serve you well 🕔. | time | butterflies | honeybees | location | scientist | | --- | --- | --- | --- | --- | | time | butterflies | honeybees | location | scientist | | 2015-08-18T00:00:00Z | 12 | 23 | 1 | langstroth | | 2015-08-18T00:00:00Z | 1 | 30 | 1 | perpetua | | 2015-08-18T00:06:00Z | 11 | 28 | 1 | langstroth | | 2015-08-18T00:06:00Z | 3 | 28 | 1 | perpetua | | 2015-08-18T05:54:00Z | 2 | 11 | 2 | langstroth | | 2015-08-18T06:00:00Z | 1 | 10 | 2 | langstroth | | 2015-08-18T06:06:00Z | 8 | 23 | 2 | perpetua | | 2015-08-18T06:12:00Z | 7 | 22 | 2 | perpetua | | time | location | scientist | butterflies | honeybees | | --- | --- | --- | --- | --- | | time | location | scientist | butterflies | honeybees | | 2015-08-18T00:00:00Z | 1 | langstroth | 12 | 23 | | 2015-08-18T00:00:00Z | 1 | perpetua | 1 | 30 | | 2015-08-18T00:06:00Z | 1 | langstroth | 11 | 28 | | 2015-08-18T00:06:00Z | 1 | perpetua | 3 | 28 | | 2015-08-18T05:54:00Z | 2 | langstroth | 2 | 11 | | 2015-08-18T06:00:00Z | 2 | langstroth | 1 | 10 | | 2015-08-18T06:06:00Z | 2 | perpetua | 8 | 23 | | 2015-08-18T06:12:00Z | 2 | perpetua | 7 | 22 | | Series number | Measurement | Tag set | Field key | | --- | --- | --- | --- | | Series number | Measurement | Tag set | Field key | | series 1 | census | location = 1 , scientist = langstroth | butterflies | | series 2 | census | location = 2 , scientist = langstroth | butterflies | | series 3 | census | location = 1 , scientist = perpetua | butterflies | | series 4 | census | location = 2 , scientist = perpetua | butterflies | | series 5 | census | location = 1 , scientist = langstroth | honeybees | | series 6 | census | location = 2 , scientist = langstroth | honeybees | | series 7 | census | location = 1 , scientist = perpetua | honeybees | | series 8 | census | location = 2 , scientist = perpetua | honeybees | --- # InfluxDB schema design and data layout Source: https://docs.influxdata.com/influxdb/v1/concepts/schema_and_data_layout/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Each InfluxDB use case is unique and your [schema](/influxdb/v1/concepts/glossary/#schema) reflects that uniqueness. In general, a schema designed for querying leads to simpler and more performant queries. We recommend the following design guidelines for most use cases: * [Where to store data (tag or field)](#where-to-store-data-tag-or-field) * [Avoid too many series](#avoid-too-many-series) * [Use recommended naming conventions](#use-recommended-naming-conventions) * [Shard Group Duration Management](#shard-group-duration-management) ## Where to store data (tag or field) Your queries should guide what data you store in [tags](/influxdb/v1/concepts/glossary/#tag) and what you store in [fields](/influxdb/v1/concepts/glossary/#field) : * Store commonly queried and grouping ([`group()`](/flux/v0/stdlib/universe/group) or [`GROUP BY`](/influxdb/v1/query_language/explore-data/#group-by-tags)) metadata in tags. * Store data in fields if each data point contains a different value. * Store numeric values as fields ([tag values](/influxdb/v1/concepts/glossary/#tag-value) only support string values). ## Avoid too many series InfluxDB indexes the following data elements to speed up reads: * [measurement](/influxdb/v1/concepts/glossary/#measurement) * [tags](/influxdb/v1/concepts/glossary/#tag) [Tag values](/influxdb/v1/concepts/glossary/#tag-value) are indexed and [field values](/influxdb/v1/concepts/glossary/#field-value) are not. This means that querying by tags is more performant than querying by fields. However, when too many indexes are created, both writes and reads may start to slow down. Each unique set of indexed data elements forms a [series key](/influxdb/v1/concepts/glossary/#series-key).[Tags](/influxdb/v1/concepts/glossary/#tag) containing highly variable information like unique IDs, hashes, and random strings lead to a large number of [series](/influxdb/v1/concepts/glossary/#series), also known as high [series cardinality](/influxdb/v1/concepts/glossary/#series-cardinality). High series cardinality is a primary driver of high memory usage for many database workloads. Therefore, to reduce memory consumption, consider storing high-cardinality values in field values rather than in tags or field keys. > [!Note] > If reads and writes to InfluxDB start to slow down, you may have high series cardinality (too many series). > See [how to find and reduce series high cardinality](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-does-series-cardinality-matter). ## Use recommended naming conventions Use the following conventions when naming your tag and field keys: * [Avoid reserved keywords in tag and field keys](#avoid-reserved-keywords-in-tag-and-field-keys) * [Avoid the same tag and field name](#avoid-the-same-name-for-a-tag-and-a-field) * [Avoid encoding data in measurements and keys](#avoid-encoding-data-in-measurements-and-keys) * [Avoid more than one piece of information in one tag](#avoid-putting-more-than-one-piece-of-information-in-one-tag) ### Avoid reserved keywords in tag and field keys Not required, but avoiding the use of reserved keywords in your tag and field keys simplifies writing queries because you won’t have to wrap your keys in double quotes. See [InfluxQL](https://github.com/influxdata/influxql/blob/master/README.md#keywords) and [Flux keywords](/flux/v0/spec/lexical-elements/#keywords) to avoid. Also, if a tag or field key contains characters other than `[A-z,_]`, you must wrap it in double quotes in InfluxQL or use [bracket notation](/flux/v0/data-types/composite/record/#bracket-notation) in Flux. ### Avoid the same name for a tag and a field Avoid using the same name for a tag and field key. This often results in unexpected behavior when querying data. If you inadvertently add the same name for a tag and a field, see[Frequently asked questions](/influxdb/v1/troubleshooting/frequently-asked-questions/#tag-and-field-key-with-the-same-name)for information about how to query the data predictably and how to fix the issue. ### Avoid encoding data in measurements and keys Store data in [tag values](/influxdb/v1/concepts/glossary/#tag-value) or [field values](/influxdb/v1/concepts/glossary/#field-value), not in [tag keys](/influxdb/v1/concepts/glossary/#tag-key), [field keys](/influxdb/v1/concepts/glossary/#field-key), or [measurements](/influxdb/v1/concepts/glossary/#measurement). If you design your schema to store data in tag and field values, your queries will be easier to write and more efficient. In addition, you’ll keep cardinality low by not creating measurements and keys as you write data. To learn more about the performance impact of high series cardinality, see [how to find and reduce high series cardinality](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-does-series-cardinality-matter). #### Compare schemas Compare the following valid schemas represented by line protocol. **Recommended**: the following schema stores metadata in separate `crop`, `plot`, and `region` tags. The `temp` field contains variable numeric data. ##### ``` ## Good Measurements schema - Data encoded in tags (recommended) weather_sensor,crop=blueberries,plot=1,region=north temp=50.1 1472515200000000000 weather_sensor,crop=blueberries,plot=2,region=midwest temp=49.8 1472515200000000000 ``` **Not recommended**: the following schema stores multiple attributes (`crop`, `plot` and `region`) concatenated (`blueberries.plot-1.north`) within the measurement, similar to Graphite metrics. ##### ``` ## Bad Measurements schema - Data encoded in the measurement (not recommended) blueberries.plot-1.north temp=50.1 1472515200000000000 blueberries.plot-2.midwest temp=49.8 1472515200000000000 ``` **Not recommended**: the following schema stores multiple attributes (`crop`, `plot` and `region`) concatenated (`blueberries.plot-1.north`) within the field key. ##### ``` ## Bad Keys schema - Data encoded in field keys (not recommended) weather_sensor blueberries.plot-1.north.temp=50.1 1472515200000000000 weather_sensor blueberries.plot-2.midwest.temp=49.8 1472515200000000000 ``` #### Compare queries Compare the following queries of the [*Good Measurements*](#good-measurements-schema) and [*Bad Measurements*](#bad-measurements-schema) schemas. The [Flux](/flux/v0/) queries calculate the average `temp` for blueberries in the `north` region **Easy to query**: [*Good Measurements*](#good-measurements-schema) data is easily filtered by `region` tag values, as in the following example. ```js // Query *Good Measurements*, data stored in separate tag values (recommended) from(bucket: "/") |> range(start:2016-08-30T00:00:00Z) |> filter(fn: (r) => r._measurement == "weather_sensor" and r.region == "north" and r._field == "temp") |> mean() ``` **Difficult to query**: [*Bad Measurements*](#bad-measurements-schema) requires regular expressions to extract `plot` and `region` from the measurement, as in the following example. ```js // Query *Bad Measurements*, data encoded in the measurement (not recommended) from(bucket: "/") |> range(start:2016-08-30T00:00:00Z) |> filter(fn: (r) => r._measurement =~ /\.north$/ and r._field == "temp") |> mean() ``` Complex measurements make some queries impossible. For example, calculating the average temperature of both plots is not possible with the [*Bad Measurements*](#bad-measurements-schema) schema. ##### InfluxQL example to query schemas ``` # Query *Bad Measurements*, data encoded in the measurement (not recommended) > SELECT mean("temp") FROM /\.north$/ # Query *Good Measurements*, data stored in separate tag values (recommended) > SELECT mean("temp") FROM "weather_sensor" WHERE "region" = 'north' ``` ### Avoid putting more than one piece of information in one tag Splitting a single tag with multiple pieces into separate tags simplifies your queries and improves performance by reducing the need for regular expressions. Consider the following schema represented by line protocol. #### Example line protocol schemas ``` ## Schema 1 - Multiple data encoded in a single tag weather_sensor,crop=blueberries,location=plot-1.north temp=50.1 1472515200000000000 weather_sensor,crop=blueberries,location=plot-2.midwest temp=49.8 1472515200000000000 ``` The Schema 1 data encodes multiple separate parameters, the `plot` and `region` into a long tag value (`plot-1.north`). Compare this to the following schema represented in line protocol. ``` ## Schema 2 - Data encoded in multiple tags weather_sensor,crop=blueberries,plot=1,region=north temp=50.1 1472515200000000000 weather_sensor,crop=blueberries,plot=2,region=midwest temp=49.8 1472515200000000000 ``` Use Flux or InfluxQL to calculate the average `temp` for blueberries in the `north` region. Schema 2 is preferable because using multiple tags, you don’t need a regular expression. #### Flux example to query schemas ```js // Schema 1 - Query for multiple data encoded in a single tag from(bucket:"/") |> range(start:2016-08-30T00:00:00Z) |> filter(fn: (r) => r._measurement == "weather_sensor" and r.location =~ /\.north$/ and r._field == "temp") |> mean() // Schema 2 - Query for data encoded in multiple tags from(bucket:"/") |> range(start:2016-08-30T00:00:00Z) |> filter(fn: (r) => r._measurement == "weather_sensor" and r.region == "north" and r._field == "temp") |> mean() ``` #### InfluxQL example to query schemas ``` # Schema 1 - Query for multiple data encoded in a single tag > SELECT mean("temp") FROM "weather_sensor" WHERE location =~ /\.north$/ # Schema 2 - Query for data encoded in multiple tags > SELECT mean("temp") FROM "weather_sensor" WHERE region = 'north' ``` ## Shard group duration management ### Shard group duration overview InfluxDB stores data in shard groups. Shard groups are organized by [retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp) (RP) and store data with timestamps that fall within a specific time interval called the [shard duration](/influxdb/v1/concepts/glossary/#shard-duration). If no shard group duration is provided, the shard group duration is determined by the RP [duration](/influxdb/v1/concepts/glossary/#duration) at the time the RP is created. The default values are: | RP Duration |Shard Group Duration| |---------------------------|--------------------| | \< 2 days | 1 hour | |\>= 2 days and \<= 6 months| 1 day | | \> 6 months | 7 days | The shard group duration is also configurable per RP. To configure the shard group duration, see [Retention Policy Management](/influxdb/v1/query_language/manage-database/#retention-policy-management). ### Shard group duration tradeoffs Determining the optimal shard group duration requires finding the balance between: * Better overall performance with longer shards * Flexibility provided by shorter shards #### Long shard group duration Longer shard group durations let InfluxDB store more data in the same logical location. This reduces data duplication, improves compression efficiency, and improves query speed in some cases. #### Short shard group duration Shorter shard group durations allow the system to more efficiently drop data and record incremental backups. When InfluxDB enforces an RP it drops entire shard groups, not individual data points, even if the points are older than the RP duration. A shard group will only be removed once a shard group’s duration *end time* is older than the RP duration. For example, if your RP has a duration of one day, InfluxDB will drop an hour’s worth of data every hour and will always have 25 shard groups. One for each hour in the day and an extra shard group that is partially expiring, but isn’t removed until the whole shard group is older than 24 hours. > > > **Note:** A special use case to consider: filtering queries on schema data (such as tags, series, measurements) by time. For example, if you want to filter schema data within a one hour interval, you must set the shard group duration to 1h. For more information, see [filter schema data by time](/influxdb/v1/query_language/explore-schema/#filter-meta-queries-by-time). > > ### Shard group duration recommendations The default shard group durations work well for most cases. However, high-throughput or long-running instances will benefit from using longer shard group durations. Here are some recommendations for longer shard group durations: | RP Duration |Shard Group Duration| |--------------------------|--------------------| | \<= 1 day | 6 hours | | \> 1 day and \<= 7 days | 1 day | |\> 7 days and \<= 3 months| 7 days | | \> 3 months | 30 days | | infinite | 52 weeks or longer | > > > **Note:** Note that `INF` (infinite) is not a [valid shard group duration](/influxdb/v1/query_language/manage-database/#retention-policy-management). > In extreme cases where data covers decades and will never be deleted, a long shard group duration like `1040w` (20 years) is perfectly valid. > > Other factors to consider before setting shard group duration: * Shard groups should be twice as long as the longest time range of the most frequent queries * Shard groups should each contain more than 100,000 [points](/influxdb/v1/concepts/glossary/#point) per shard group * Shard groups should each contain more than 1,000 points per [series](/influxdb/v1/concepts/glossary/#series) #### Shard group duration for backfilling Bulk insertion of historical data covering a large time range in the past will trigger the creation of a large number of shards at once. The concurrent access and overhead of writing to hundreds or thousands of shards can quickly lead to slow performance and memory exhaustion. When writing historical data, we highly recommend temporarily setting a longer shard group duration so fewer shards are created. Typically, a shard group duration of 52 weeks works well for backfilling. | RP Duration | Shard Group Duration | | --- | --- | | RP Duration | Shard Group Duration | | < 2 days | 1 hour | | >= 2 days and <= 6 months | 1 day | | > 6 months | 7 days | | RP Duration | Shard Group Duration | | --- | --- | | RP Duration | Shard Group Duration | | <= 1 day | 6 hours | | > 1 day and <= 7 days | 1 day | | > 7 days and <= 3 months | 7 days | | > 3 months | 30 days | | infinite | 52 weeks or longer | --- # In-memory indexing and the Time-Structured Merge Tree (TSM) Source: https://docs.influxdata.com/influxdb/v1/concepts/storage_engine/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## The InfluxDB storage engine and the Time-Structured Merge Tree (TSM) The InfluxDB storage engine looks very similar to a LSM Tree. It has a write ahead log and a collection of read-only data files which are similar in concept to SSTables in an LSM Tree. TSM files contain sorted, compressed series data. InfluxDB will create a [shard](/influxdb/v1/concepts/glossary/#shard) for each block of time. For example, if you have a [retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp) with an unlimited duration, shards will be created for each 7 day block of time. Each of these shards maps to an underlying storage engine database. Each of these databases has its own [WAL](/influxdb/v1/concepts/glossary/#wal-write-ahead-log) and TSM files. We’ll dig into each of these parts of the storage engine. ## Storage engine The storage engine ties a number of components together and provides the external interface for storing and querying series data. It is composed of a number of components that each serve a particular role: * In-Memory Index - The in-memory index is a shared index across shards that provides the quick access to [measurements](/influxdb/v1/concepts/glossary/#measurement), [tags](/influxdb/v1/concepts/glossary/#tag), and [series](/influxdb/v1/concepts/glossary/#series). The index is used by the engine, but is not specific to the storage engine itself. * WAL - The WAL is a write-optimized storage format that allows for writes to be durable, but not easily queryable. Writes to the WAL are appended to segments of a fixed size. * Cache - The Cache is an in-memory representation of the data stored in the WAL. It is queried at runtime and merged with the data stored in TSM files. * TSM Files - TSM files store compressed series data in a columnar format. * FileStore - The FileStore mediates access to all TSM files on disk. It ensures that TSM files are installed atomically when existing ones are replaced as well as removing TSM files that are no longer used. * Compactor - The Compactor is responsible for converting less optimized Cache and TSM data into more read-optimized formats. It does this by compressing series, removing deleted data, optimizing indices and combining smaller files into larger ones. * Compaction Planner - The Compaction Planner determines which TSM files are ready for a compaction and ensures that multiple concurrent compactions do not interfere with each other. * Compression - Compression is handled by various Encoders and Decoders for specific data types. Some encoders are fairly static and always encode the same type the same way; others switch their compression strategy based on the shape of the data. * Writers/Readers - Each file type (WAL segment, TSM files, tombstones, etc..) has Writers and Readers for working with the formats. ### Write Ahead Log (WAL) The WAL is organized as a bunch of files that look like `_000001.wal`. The file numbers are monotonically increasing and referred to as WAL segments. When a segment reaches 10MB in size, it is closed and a new one is opened. Each WAL segment stores multiple compressed blocks of writes and deletes. When a write comes in the new points are serialized, compressed using Snappy, and written to a WAL file. The file is `fsync`’d and the data is added to an in-memory index before a success is returned. This means that batching points together is required to achieve high throughput performance. (Optimal batch size seems to be 5,000-10,000 points per batch for many use cases.) Each entry in the WAL follows a [TLV standard](https://en.wikipedia.org/wiki/Type-length-value) with a single byte representing the type of entry (write or delete), a 4 byte `uint32` for the length of the compressed block, and then the compressed block. ### Cache The Cache is an in-memory copy of all data points current stored in the WAL. The points are organized by the key, which is the measurement, [tag set](/influxdb/v1/concepts/glossary/#tag-set), and unique [field](/influxdb/v1/concepts/glossary/#field). Each field is kept as its own time-ordered range. The Cache data is not compressed while in memory. Queries to the storage engine will merge data from the Cache with data from the TSM files. Queries execute on a copy of the data that is made from the cache at query processing time. This way writes that come in while a query is running won’t affect the result. Deletes sent to the Cache will clear out the given key or the specific time range for the given key. The Cache exposes a few controls for snapshotting behavior. The two most important controls are the memory limits. There is a lower bound, [`cache-snapshot-memory-size`](/influxdb/v1/administration/config#cache-snapshot-memory-size), which when exceeded will trigger a snapshot to TSM files and remove the corresponding WAL segments. There is also an upper bound, [`cache-max-memory-size`](/influxdb/v1/administration/config#cache-max-memory-size), which when exceeded will cause the Cache to reject new writes. These configurations are useful to prevent out of memory situations and to apply back pressure to clients writing data faster than the instance can persist it. The checks for memory thresholds occur on every write. The other snapshot controls are time based. The idle threshold, [`cache-snapshot-write-cold-duration`](/influxdb/v1/administration/config#cache-snapshot-write-cold-duration), forces the Cache to snapshot to TSM files if it hasn’t received a write within the specified interval. The in-memory Cache is recreated on restart by re-reading the WAL files on disk. ### TSM files TSM files are a collection of read-only files that are memory mapped. The structure of these files looks very similar to an SSTable in LevelDB or other LSM Tree variants. A TSM file is composed of four sections: header, blocks, index, and footer. ``` +--------+------------------------------------+-------------+--------------+ | Header | Blocks | Index | Footer | |5 bytes | N bytes | N bytes | 4 bytes | +--------+------------------------------------+-------------+--------------+ ``` The Header is a magic number to identify the file type and a version number. ``` +-------------------+ | Header | +-------------------+ | Magic │ Version | | 4 bytes │ 1 byte | +-------------------+ ``` Blocks are sequences of pairs of CRC32 checksums and data. The block data is opaque to the file. The CRC32 is used for block level error detection. The length of the blocks is stored in the index. ``` +--------------------------------------------------------------------+ │ Blocks │ +---------------------+-----------------------+----------------------+ | Block 1 | Block 2 | Block N | +---------------------+-----------------------+----------------------+ | CRC | Data | CRC | Data | CRC | Data | | 4 bytes | N bytes | 4 bytes | N bytes | 4 bytes | N bytes | +---------------------+-----------------------+----------------------+ ``` Following the blocks is the index for the blocks in the file. The index is composed of a sequence of index entries ordered lexicographically by key and then by time. The key includes the measurement name, tag set, and one field. Multiple fields per point creates multiple index entries in the TSM file. Each index entry starts with a key length and the key, followed by the block type (float, int, bool, string) and a count of the number of index block entries that follow for that key. Each index block entry is composed of the min and max time for the block, the offset into the file where the block is located and the size of the block. There is one index block entry for each block in the TSM file that contains the key. The index structure can provide efficient access to all blocks as well as the ability to determine the cost associated with accessing a given key. Given a key and timestamp, we can determine whether a file contains the block for that timestamp. We can also determine where that block resides and how much data must be read to retrieve the block. Knowing the size of the block, we can efficiently provision our IO statements. ``` +-----------------------------------------------------------------------------+ │ Index │ +-----------------------------------------------------------------------------+ │ Key Len │ Key │ Type │ Count │Min Time │Max Time │ Offset │ Size │...│ │ 2 bytes │ N bytes │1 byte│2 bytes│ 8 bytes │ 8 bytes │8 bytes │4 bytes │ │ +-----------------------------------------------------------------------------+ ``` The last section is the footer that stores the offset of the start of the index. ``` +---------+ │ Footer │ +---------+ │Index Ofs│ │ 8 bytes │ +---------+ ``` ### Compression Each block is compressed to reduce storage space and disk IO when querying. A block contains the timestamps and values for a given series and field. Each block has one byte header, followed by the compressed timestamps and then the compressed values. ``` +--------------------------------------------------+ | Type | Len | Timestamps | Values | |1 Byte | VByte | N Bytes | N Bytes │ +--------------------------------------------------+ ``` The timestamps and values are compressed and stored separately using encodings dependent on the data type and its shape. Storing them independently allows timestamp encoding to be used for all timestamps, while allowing different encodings for different field types. For example, some points may be able to use run-length encoding whereas other may not. Each value type also contains a 1 byte header indicating the type of compression for the remaining bytes. The four high bits store the compression type and the four low bits are used by the encoder if needed. #### Timestamps Timestamp encoding is adaptive and based on the structure of the timestamps that are encoded. It uses a combination of delta encoding, scaling, and compression using simple8b run-length encoding, as well as falling back to no compression if needed. Timestamp resolution is variable but can be as granular as a nanosecond, requiring up to 8 bytes to store uncompressed. During encoding, the values are first delta-encoded. The first value is the starting timestamp and subsequent values are the differences from the prior value. This usually converts the values into much smaller integers that are easier to compress. Many timestamps are also monotonically increasing and fall on even boundaries of time such as every 10s. When timestamps have this structure, they are scaled by the largest common divisor that is also a factor of 10. This has the effect of converting very large integer deltas into smaller ones that compress even better. Using these adjusted values, if all the deltas are the same, the time range is stored using run-length encoding. If run-length encoding is not possible and all values are less than (1 « 60) - 1 ([\~36.5 years](https://www.wolframalpha.com/input/?i=(1+%3C%3C+60)+-+1+nanoseconds+to+years) at nanosecond resolution), then the timestamps are encoded using [simple8b encoding](https://github.com/jwilder/encoding/tree/master/simple8b). Simple8b encoding is a 64bit word-aligned integer encoding that packs multiple integers into a single 64bit word. If any value exceeds the maximum the deltas are stored uncompressed using 8 bytes each for the block. Future encodings may use a patched scheme such as Patched Frame-Of-Reference (PFOR) to handle outliers more effectively. #### Floats Floats are encoded using an implementation of the [Facebook Gorilla paper](http://www.vldb.org/pvldb/vol8/p1816-teller.pdf). The encoding XORs consecutive values together to produce a small result when the values are close together. The delta is then stored using control bits to indicate how many leading and trailing zeroes are in the XOR value. Our implementation removes the timestamp encoding described in paper and only encodes the float values. #### Integers Integer encoding uses two different strategies depending on the range of values in the uncompressed data. Encoded values are first encoded using [ZigZag encoding](https://developers.google.com/protocol-buffers/docs/encoding#signed-integers). This interleaves positive and negative integers across a range of positive integers. For example, [-2,-1,0,1] becomes [3,1,0,2]. See Google’s [Protocol Buffers documentation](https://developers.google.com/protocol-buffers/docs/encoding#signed-integers) for more information. If all ZigZag encoded values are less than (1 « 60) - 1, they are compressed using simple8b encoding. If any values are larger than the maximum then all values are stored uncompressed in the block. If all values are identical, run-length encoding is used. This works very well for values that are frequently constant. #### Booleans Booleans are encoded using a simple bit packing strategy where each Boolean uses 1 bit. The number of Booleans encoded is stored using variable-byte encoding at the beginning of the block. #### Strings Strings are encoding using [Snappy](http://google.github.io/snappy/) compression. Each string is packed consecutively and they are compressed as one larger block. ### Compactions Compactions are recurring processes that migrate data stored in a write-optimized format into a more read-optimized format. There are a number of stages of compaction that take place while a shard is hot for writes: * **Snapshots** - Values in the Cache and WAL must be converted to TSM files to free memory and disk space used by the WAL segments. These compactions occur based on the cache memory and time thresholds. * **Level Compactions** - Level compactions (levels 1-4) occur as the TSM files grow. TSM files are compacted from snapshots to level 1 files. Multiple level 1 files are compacted to produce level 2 files. The process continues until files reach level 4 (full compaction) and the max size for a TSM file. They will not be compacted further unless deletes, index optimization compactions, or full compactions need to run. Lower level compactions use strategies that avoid CPU-intensive activities like decompressing and combining blocks. Higher level (and thus less frequent) compactions will re-combine blocks to fully compact them and increase the compression ratio. * **Index Optimization** - When many level 4 TSM files accumulate, the internal indexes become larger and more costly to access. An index optimization compaction splits the series and indices across a new set of TSM files, sorting all points for a given series into one TSM file. Before an index optimization, each TSM file contained points for most or all series, and thus each contains the same series index. After an index optimization, each TSM file contains points from a minimum of series and there is little series overlap between files. Each TSM file thus has a smaller unique series index, instead of a duplicate of the full series list. In addition, all points from a particular series are contiguous in a TSM file rather than spread across multiple TSM files. * **Full Compactions** - Full compactions (level 4 compactions) run when a shard has become cold for writes for long time, or when deletes have occurred on the shard. Full compactions produce an optimal set of TSM files and include all optimizations from Level and Index Optimization compactions. Once a shard is fully compacted, no other compactions will run on it unless new writes or deletes are stored. ### Writes Writes are appended to the current WAL segment and are also added to the Cache. Each WAL segment has a maximum size. Writes roll over to a new file once the current file fills up. The cache is also size bounded; snapshots are taken and WAL compactions are initiated when the cache becomes too full. If the inbound write rate exceeds the WAL compaction rate for a sustained period, the cache may become too full, in which case new writes will fail until the snapshot process catches up. When WAL segments fill up and are closed, the Compactor snapshots the Cache and writes the data to a new TSM file. When the TSM file is successfully written and `fsync`’d, it is loaded and referenced by the FileStore. ### Updates Updates (writing a newer value for a point that already exists) occur as normal writes. Since cached values overwrite existing values, newer writes take precedence. If a write would overwrite a point in a prior TSM file, the points are merged at query runtime and the newer write takes precedence. ### Deletes Deletes occur by writing a delete entry to the WAL for the measurement or series and then updating the Cache and FileStore. The Cache evicts all relevant entries. The FileStore writes a tombstone file for each TSM file that contains relevant data. These tombstone files are used at startup time to ignore blocks as well as during compactions to remove deleted entries. Queries against partially deleted series are handled at query time until a compaction removes the data fully from the TSM files. ### Queries When a query is executed by the storage engine, it is essentially a seek to a given time associated with a specific series key and field. First, we do a search on the data files to find the files that contain a time range matching the query as well containing matching series. Once we have the data files selected, we next need to find the position in the file of the series key index entries. We run a binary search against each TSM index to find the location of its index blocks. In common cases the blocks will not overlap across multiple TSM files and we can search the index entries linearly to find the start block from which to read. If there are overlapping blocks of time, the index entries are sorted to ensure newer writes will take precedence and that blocks can be processed in order during query execution. When iterating over the index entries the blocks are read sequentially from the blocks section. The block is decompressed and we seek to the specific point. # The new InfluxDB storage engine: from LSM Tree to B+Tree and back again to create the Time Structured Merge Tree Writing a new storage format should be a last resort. So how did InfluxData end up writing our own engine? InfluxData has experimented with many storage formats and found each lacking in some fundamental way. The performance requirements for InfluxDB are significant, and eventually overwhelm other storage systems. The 0.8 line of InfluxDB allowed multiple storage engines, including LevelDB, RocksDB, HyperLevelDB, and LMDB. The 0.9 line of InfluxDB used BoltDB as the underlying storage engine. This writeup is about the Time Structured Merge Tree storage engine that was released in 0.9.5 and is the only storage engine supported in InfluxDB 0.11+, including the entire 1.x family. The properties of the time series data use case make it challenging for many existing storage engines. Over the course of InfluxDB development, InfluxData tried a few of the more popular options. We started with LevelDB, an engine based on LSM Trees, which are optimized for write throughput. After that we tried BoltDB, an engine based on a memory mapped B+Tree, which is optimized for reads. Finally, we ended up building our own storage engine that is similar in many ways to LSM Trees. With our new storage engine we were able to achieve up to a 45x reduction in disk space usage from our B+Tree setup with even greater write throughput and compression than what we saw with LevelDB and its variants. This post will cover the details of that evolution and end with an in-depth look at our new storage engine and its inner workings. ## Properties of time series data The workload of time series data is quite different from normal database workloads. There are a number of factors that conspire to make it very difficult to scale and remain performant: * Billions of individual data points * High write throughput * High read throughput * Large deletes (data expiration) * Mostly an insert/append workload, very few updates The first and most obvious problem is one of scale. In DevOps, IoT, or APM it is easy to collect hundreds of millions or billions of unique data points every day. For example, let’s say we have 200 VMs or servers running, with each server collecting an average of 100 measurements every 10 seconds. Given there are 86,400 seconds in a day, a single measurement will generate 8,640 points in a day per server. That gives us a total of 172,800,000 (`200 * 100 * 8,640`) individual data points per day. We find similar or larger numbers in sensor data use cases. The volume of data means that the write throughput can be very high. We regularly get requests for setups than can handle hundreds of thousands of writes per second. Some larger companies will only consider systems that can handle millions of writes per second. At the same time, time series data can be a high read throughput use case. It’s true that if you’re tracking 700,000 unique metrics or time series you can’t hope to visualize all of them. That leads many people to think that you don’t actually read most of the data that goes into the database. However, other than dashboards that people have up on their screens, there are automated systems for monitoring or combining the large volume of time series data with other types of data. Inside InfluxDB, aggregate functions calculated on the fly may combine tens of thousands of distinct time series into a single view. Each one of those queries must read each aggregated data point, so for InfluxDB the read throughput is often many times higher than the write throughput. Given that time series is mostly an append-only workload, you might think that it’s possible to get great performance on a B+Tree. Appends in the keyspace are efficient and you can achieve greater than 100,000 per second. However, we have those appends happening in individual time series. So the inserts end up looking more like random inserts than append only inserts. One of the biggest problems we found with time series data is that it’s very common to delete all data after it gets past a certain age. The common pattern here is that users have high precision data that is kept for a short period of time like a few days or months. Users then downsample and aggregate that data into lower precision rollups that are kept around much longer. The naive implementation would be to simply delete each record once it passes its expiration time. However, that means that once the first points written reach their expiration date, the system is processing just as many deletes as writes, which is something most storage engines aren’t designed for. Let’s dig into the details of the two types of storage engines we tried and how these properties had a significant impact on our performance. ## LevelDB and log structured merge trees When the InfluxDB project began, we picked LevelDB as the storage engine because we had used it for time series data storage in the product that was the precursor to InfluxDB. We knew that it had great properties for write throughput and everything seemed to “just work”. LevelDB is an implementation of a log structured merge tree (LSM tree) that was built as an open source project at Google. It exposes an API for a key-value store where the key space is sorted. This last part is important for time series data as it allowed us to quickly scan ranges of time as long as the timestamp was in the key. LSM Trees are based on a log that takes writes and two structures known as Mem Tables and SSTables. These tables represent the sorted keyspace. SSTables are read only files that are continuously replaced by other SSTables that merge inserts and updates into the keyspace. The two biggest advantages that LevelDB had for us were high write throughput and built in compression. However, as we learned more about what people needed with time series data, we encountered a few insurmountable challenges. The first problem we had was that LevelDB doesn’t support hot backups. If you want to do a safe backup of the database, you have to close it and then copy it. The LevelDB variants RocksDB and HyperLevelDB fix this problem, but there was another more pressing problem that we didn’t think they could solve. Our users needed a way to automatically manage data retention. That meant we needed deletes on a very large scale. In LSM Trees, a delete is as expensive, if not more so, than a write. A delete writes a new record known as a tombstone. After that queries merge the result set with any tombstones to purge the deleted data from the query return. Later, a compaction runs that removes the tombstone record and the underlying deleted record in the SSTable file. To get around doing deletes, we split data across what we call shards, which are contiguous blocks of time. Shards would typically hold either one day or seven days worth of data. Each shard mapped to an underlying LevelDB. This meant that we could drop an entire day of data by just closing out the database and removing the underlying files. Users of RocksDB may at this point bring up a feature called ColumnFamilies. When putting time series data into Rocks, it’s common to split blocks of time into column families and then drop those when their time is up. It’s the same general idea: create a separate area where you can just drop files instead of updating indexes when you delete a large block of data. Dropping a column family is a very efficient operation. However, column families are a fairly new feature and we had another use case for shards. Organizing data into shards meant that it could be moved within a cluster without having to examine billions of keys. At the time of this writing, it was not possible to move a column family in one RocksDB to another. Old shards are typically cold for writes so moving them around would be cheap and easy. We would have the added benefit of having a spot in the keyspace that is cold for writes so it would be easier to do consistency checks later. The organization of data into shards worked great for a while, until a large amount of data went into InfluxDB. LevelDB splits the data out over many small files. Having dozens or hundreds of these databases open in a single process ended up creating a big problem. Users that had six months or a year of data would run out of file handles. It’s not something we found with the majority of users, but anyone pushing the database to its limits would hit this problem and we had no fix for it. There were simply too many file handles open. ## BoltDB and mmap B+Trees After struggling with LevelDB and its variants for a year we decided to move over to BoltDB, a pure Golang database heavily inspired by LMDB, a mmap B+Tree database written in C. It has the same API semantics as LevelDB: a key value store where the keyspace is ordered. Many of our users were surprised. Our own posted tests of the LevelDB variants vs. LMDB (a mmap B+Tree) showed RocksDB as the best performer. However, there were other considerations that went into this decision outside of the pure write performance. At this point our most important goal was to get to something stable that could be run in production and backed up. BoltDB also had the advantage of being written in pure Go, which simplified our build chain immensely and made it easy to build for other OSes and platforms. The biggest win for us was that BoltDB used a single file as the database. At this point our most common source of bug reports were from people running out of file handles. Bolt solved the hot backup problem and the file limit problems all at the same time. We were willing to take a hit on write throughput if it meant that we’d have a system that was more reliable and stable that we could build on. Our reasoning was that for anyone pushing really big write loads, they’d be running a cluster anyway. We released versions 0.9.0 to 0.9.2 based on BoltDB. From a development perspective it was delightful. Clean API, fast and easy to build in our Go project, and reliable. However, after running for a while we found a big problem with write throughput. After the database got over a few GB, writes would start spiking IOPS. Some users were able to get past this by putting InfluxDB on big hardware with near unlimited IOPS. However, most users are on VMs with limited resources in the cloud. We had to figure out a way to reduce the impact of writing a bunch of points into hundreds of thousands of series at a time. With the 0.9.3 and 0.9.4 releases our plan was to put a write ahead log (WAL) in front of Bolt. That way we could reduce the number of random insertions into the keyspace. Instead, we’d buffer up multiple writes that were next to each other and then flush them at once. However, that only served to delay the problem. High IOPS still became an issue and it showed up very quickly for anyone operating at even moderate work loads. However, our experience building the first WAL implementation in front of Bolt gave us the confidence we needed that the write problem could be solved. The performance of the WAL itself was fantastic, the index simply could not keep up. At this point we started thinking again about how we could create something similar to an LSM Tree that could keep up with our write load. Thus was born the Time Structured Merge Tree. --- # Time Series Index (TSI) overview Source: https://docs.influxdata.com/influxdb/v1/concepts/time-series-index/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Find overview and background information on Time Series Index (TSI) in this topic. For detail, including how to enable and configure TSI, see [Time Series Index (TSI) details](/influxdb/v1/concepts/tsi-details/). ## Overview To support a large number of time series, that is, a very high cardinality in the number of unique time series that the database stores, InfluxData has added the new Time Series Index (TSI). InfluxData supports customers using InfluxDB with tens of millions of time series. InfluxData’s goal, however, is to expand to hundreds of millions, and eventually billions. Using InfluxData’s TSI storage engine, users should be able to have millions of unique time series. The goal is that the number of series should be unbounded by the amount of memory on the server hardware. Importantly, the number of series that exist in the database will have a negligible impact on database startup time. This work represents the most significant technical advancement in the database since InfluxData released the Time Series Merge Tree (TSM) storage engine in 2016. ## Background information InfluxDB actually looks like two databases in one, a time series data store and an inverted index for the measurement, tag, and field metadata. ### Time-Structured Merge Tree (TSM) The Time-Structured Merge Tree (TSM) engine solves the problem of getting maximum throughput, compression, and query speed for raw time series data. Up until TSI, the inverted index was an in-memory data structure that was built during startup of the database based on the data in TSM. This meant that for every measurement, tag key-value pair, and field name, there was a lookup table in-memory to map those bits of metadata to an underlying time series. For users with a high number of ephemeral series, memory utilization continued increasing as new time series were created. And, startup times increased since all of that data would have to be loaded onto the heap at start time. > > > For details, see [TSM-based data storage and in-memory indexing](/influxdb/v1/concepts/storage_engine/). > > ### Time Series Index (TSI) The new time series index (TSI) moves the index to files on disk that we memory map. This means that we let the operating system handle being the Least Recently Used (LRU) memory. Much like the TSM engine for raw time series data we have a write-ahead log with an in-memory structure that gets merged at query time with the memory-mapped index. Background routines run constantly to compact the index into larger and larger files to avoid having to do too many index merges at query time. Under the covers, we’re using techniques like Robin Hood Hashing to do fast index lookups and HyperLogLog++ to keep sketches of cardinality estimates. The latter will give us the ability to add things to the query languages like the [SHOW CARDINALITY](/influxdb/v1/query_language/spec#show-cardinality) queries. ### Issues solved by TSI and remaining to be solved The primary issue that Time Series Index (TSI) addresses is ephemeral time series. Most frequently, this occurs in use cases that want to track per process metrics or per container metrics by putting identifiers in tags. For example, the [Heapster project for Kubernetes](https://github.com/kubernetes/heapster) does this. For series that are no longer hot for writes or queries, they won’t take up space in memory. The issue that the Heapster project and similar use cases did not address is limiting the scope of data returned by the SHOW queries. We’ll have updates to the query language in the future to limit those results by time. We also don’t solve the problem of having all these series hot for reads and writes. For that problem, scale-out clustering is the solution. We’ll have to continue to optimize the query language and engine to work with large sets of series. We’ll need to add guard rails and limits into the language and eventually, add spill-to-disk query processing. That work will be on-going in every release of InfluxDB. --- # Time Series Index (TSI) details Source: https://docs.influxdata.com/influxdb/v1/concepts/tsi-details/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. When InfluxDB ingests data, we store not only the value but we also index the measurement and tag information so that it can be queried quickly. In earlier versions, index data could only be stored in-memory, however, that requires a lot of RAM and places an upper bound on the number of series a machine can hold. This upper bound is usually somewhere between 1 - 4 million series depending on the machine used. The Time Series Index (TSI) was developed to allow us to go past that upper bound. TSI stores index data on disk so that we are no longer restricted by RAM. TSI uses the operating system’s page cache to pull hot data into memory and let cold data rest on disk. ## Enable TSI To enable TSI, set the following line in the InfluxDB configuration file (`influxdb.conf`): ``` index-version = "tsi1" ``` (Be sure to include the double quotes.) ### InfluxDB Enterprise * To convert your data nodes to support TSI, see [Upgrade InfluxDB Enterprise clusters](/enterprise_influxdb/v1/administration/upgrading/). * For detail on configuration, see [Configure InfluxDB Enterprise clusters](/enterprise_influxdb/v1/administration/configuration/). ### InfluxDB OSS * For detail on configuration, see [Configuring InfluxDB OSS](/influxdb/v1/administration/config/). ## Tooling ### `influx_inspect dumptsi` If you are troubleshooting an issue with an index, you can use the `influx_inspect dumptsi` command. This command allows you to print summary statistics on an index, file, or a set of files. This command only works on one index at a time. For details on this command, see [influx\_inspect dumptsi](/influxdb/v1/tools/influx_inspect/#dumptsi). ### `influx_inspect buildtsi` If you want to convert an existing shard from an in-memory index to a TSI index, or if you have an existing TSI index which has become corrupt, you can use the `buildtsi` command to create the index from the underlying TSM data. If you have an existing TSI index that you want to rebuild, first delete the `index` directory within your shard. This command works at the server-level but you can optionally add database, retention policy and shard filters to only apply to a subset of shards. For details on this command, see [influx inspect buildtsi](/influxdb/v1/tools/influx_inspect/#buildtsi). ## Understanding TSI ### File organization TSI (Time Series Index) is a log-structured merge tree-based database for InfluxDB series data. TSI is composed of several parts: * **Index**: Contains the entire index dataset for a single shard. * **Partition**: Contains a sharded partition of the data for a shard. * **LogFile**: Contains newly written series as an in-memory index and is persisted as a WAL. * **IndexFile**: Contains an immutable, memory-mapped index built from a LogFile or merged from two contiguous index files. There is also a **SeriesFile** which contains a set of all series keys across the entire database. Each shard within the database shares the same series file. ### Writes The following occurs when a write comes into the system: 1. Series is added to the series file or is looked up if it already exists. This returns an auto-incrementing series ID. 2. The series is sent to the Index. The index maintains a roaring bitmap of existing series IDs and ignores series that have already been created. 3. The series is hashed and sent to the appropriate Partition. 4. The Partition writes the series as an entry to the LogFile. 5. The LogFile writes the series to a write-ahead log file on disk and adds the series to a set of in-memory indexes. ### Compaction When compactions are enabled, every second, InfluxDB checks to see whether compactions are needed. If there haven’t been writes during the `compact-full-write-cold-duration` period (by default, `4h`), InfluxDB compacts all TSM files. Otherwise, InfluxDB groups TSM files into compaction levels (determined by the number of times the file have been compacted), and attempts to combine files and compress them more efficiently. Once the `LogFile` exceeds a threshold (`5MB`), InfluxDB creates a new active log file, and the previous one begins compacting into an `IndexFile`. This first index file is at level 1 (L1). The log file is considered level 0 (L0). Index files can also be created by merging two smaller index files together. For example, if two contiguous L1 index files exist, InfluxDB merges them into an L2 index file. InfluxDB schedules compactions preferentially, using the following guidelines: * The lower the level (the fewer times a file has been compacted), the more weight is given to compacting it. * The more compactible files in a level, the higher the priority given to that level. If the number of files in each level is equal, lower levels are compacted first. * If a higher level has more candidates for compaction, it may be compacted before a lower level. InfluxDB multiplies the number of collection groups (collections of files to compact into a single next-generation file) by a specified weight (0.4, 0.3, 0.2, and 0.1) per level, to determine the compaction priority. #### Important compaction configuration settings Compaction workloads are driven by the ingestion rate of the database and the following throttling configuration settings: * `cache-snapshot-memory-size`: Specifies the `write-cache` size kept in memory before data is written to a TSM file. * `cache-snapshot-write-cold-duration`: If a cache does not exceed the `cache-snapshot-memory-size` size, this specifies the length of time to keep data in memory without any incoming data before data is written to a TSM file. * `max-concurrent-compactions`: The number compactions can run at once. * `compact-throughput`: Controls the average disk IO by the compaction engine. * `compact-throughput-burst`: Controls maximum disk IO by the compaction engine. * `compact-full-write-cold-duration`: How long a shard must receive no writes or deletes before it is scheduled for full compaction. These configuration settings are especially beneficial for systems with irregular loads, limiting compactions during periods of high usage, and letting compactions catch up during periods of lower load. In systems with stable loads, if compactions interfere with other operations, typically, the system is undersized for its load, and configuration changes won’t help much. ### Reads The index provides several API calls for retrieving sets of data such as: * `MeasurementIterator()`: Returns a sorted list of measurement names. * `TagKeyIterator()`: Returns a sorted list of tag keys in a measurement. * `TagValueIterator()`: Returns a sorted list of tag values for a tag key. * `MeasurementSeriesIDIterator()`: Returns a sorted list of all series IDs for a measurement. * `TagKeySeriesIDIterator()`: Returns a sorted list of all series IDs for a tag key. * `TagValueSeriesIDIterator()`: Returns a sorted list of all series IDs for a tag value. These iterators are all composable using several merge iterators. For each type of iterator (measurement, tag key, tag value, series id), there are multiple merge iterator types: * **Merge**: Deduplicates items from two iterators. * **Intersect**: Returns only items that exist in two iterators. * **Difference**: Only returns items from first iterator that don’t exist in the second iterator. For example, a query with a WHERE clause of `region != 'us-west'` that operates across two shards will construct a set of iterators like this: ``` DifferenceSeriesIDIterators( MergeSeriesIDIterators( Shard1.MeasurementSeriesIDIterator("m"), Shard2.MeasurementSeriesIDIterator("m"), ), MergeSeriesIDIterators( Shard1.TagValueSeriesIDIterator("m", "region", "us-west"), Shard2.TagValueSeriesIDIterator("m", "region", "us-west"), ), ) ``` ### Log File Structure The log file is simply structured as a list of LogEntry objects written to disk in sequential order. Log files are written until they reach 5MB and then they are compacted into index files. The entry objects in the log can be of any of the following types: * AddSeries * DeleteSeries * DeleteMeasurement * DeleteTagKey * DeleteTagValue The in-memory index on the log file tracks the following: * Measurements by name * Tag keys by measurement * Tag values by tag key * Series by measurement * Series by tag value * Tombstones for series, measurements, tag keys, and tag values. The log file also maintains bitsets for series ID existence and tombstones. These bitsets are merged with other log files and index files to regenerate the full index bitset on startup. ### Index File Structure The index file is an immutable file that tracks similar information to the log file, but all data is indexed and written to disk so that it can be directly accessed from a memory-map. The index file has the following sections: * **TagBlocks:** Maintains an index of tag values for a single tag key. * **MeasurementBlock:** Maintains an index of measurements and their tag keys. * **Trailer:** Stores offset information for the file as well as HyperLogLog sketches for cardinality estimation. ### Manifest The MANIFEST file is stored in the index directory and lists all the files that belong to the index and the order in which they should be accessed. This file is updated every time a compaction occurs. Any files that are in the directory that are not in the index file are index files that are in the process of being compacted. ### FileSet A file set is an in-memory snapshot of the manifest that is obtained while the InfluxDB process is running. This is required to provide a consistent view of the index at a point-in-time. The file set also facilitates reference counting for all of its files so that no file will be deleted via compaction until all readers of the file are done with it. --- # (untitled) Source: https://docs.influxdata.com/influxdb/v1/data_sources/carbon/ ========== This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. --- # (untitled) Source: https://docs.influxdata.com/influxdb/v1/data_sources/collectd/ ========== This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. --- # Diamond Source: https://docs.influxdata.com/influxdb/v1/data_sources/diamond/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## Saving Diamond Metrics into InfluxDB Diamond is a metrics collection and delivery daemon written in Python. It is capable of collecting cpu, memory, network, i/o, load and disk metrics. Additionally, it features an API for implementing custom collectors for gathering metrics from almost any source. [Diamond homepage](https://github.com/python-diamond) Diamond started supporting InfluxDB at version 3.5. ## Configuring Diamond to send metrics to InfluxDB Prerequisites: Diamond depends on the [influxdb python client](https://github.com/influxdb/influxdb-python). InfluxDB-version-specific installation instructions for the influxdb python client can be found on their [github page](https://github.com/influxdb/influxdb-python). [Diamond InfluxdbHandler configuration page](https://github.com/python-diamond/Diamond/wiki/handler-InfluxdbHandler) --- # OpenTSDB Source: https://docs.influxdata.com/influxdb/v1/data_sources/opentsdb/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxDB supports the OpenTSDB [“telnet” protocol](http://opentsdb.net/docs/build/html/user_guide/writing/index.html#telnet). When OpenTSDB support is enabled, InfluxDB can act as a drop-in replacement for your OpenTSDB system. An example input point, and how it is processed, is shown below. ``` put sys.cpu.user 1356998400 42.5 host=webserver01 cpu=0 ``` When InfluxDB receives this data, a point is written to the database. The point’s Measurement is `sys.cpu.user`, the timestamp is `1356998400`, and the value is `42.5`. The point is also tagged with `host=webserver01` and `cpu=0`. Tags allow fast and efficient queries to be performed on your data. To learn more about enabling OpenTSDB support, check the example [configuration file](https://github.com/influxdb/influxdb/blob/1.8/etc/config.sample.toml). --- # Flux data scripting language Source: https://docs.influxdata.com/influxdb/v1/flux/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Flux is a functional data scripting language designed for querying, analyzing, and acting on time series data. It takes the power of [InfluxQL](/influxdb/v1/query_language/spec/) and the functionality of [TICKscript](/kapacitor/v1/reference/tick/introduction/) and combines them into a single, unified syntax. > > > Flux is production-ready and included with [InfluxDB v1.8+](/influxdb/v1/). > > ## Flux design principles Flux is designed to be usable, readable, flexible, composable, testable, contributable, and shareable. Its syntax is largely inspired by [2018’s most popular scripting language](https://insights.stackoverflow.com/survey/2018#technology), JavaScript, and takes a functional approach to data exploration and processing. The following example illustrates pulling data from a bucket (similar to an InfluxQL database) for the last five minutes, filtering that data by the `cpu` measurement and the `cpu=cpu-total` tag, windowing the data in 1 minute intervals, and calculating the average of each window: ```js from(bucket:"telegraf/autogen") |> range(start:-1h) |> filter(fn:(r) => r._measurement == "cpu" and r.cpu == "cpu-total" ) |> aggregateWindow(every: 1m, fn: mean) ``` ### [Query data with Flux](/influxdb/v1/flux/guides/) Guides that walk through both common and complex queries and use cases for Flux. ### [Enable Flux](/influxdb/v1/flux/installation/) Instructions for enabling Flux in your InfluxDB configuration. ### [Flux vs InfluxQL](/influxdb/v1/flux/flux-vs-influxql/) ### [Get started with Flux](/influxdb/v1/flux/get-started/) Get started with Flux, InfluxData’s new functional data scripting language. This step-by-step guide will walk you through the basics and get you on your way. --- # Flux vs InfluxQL Source: https://docs.influxdata.com/influxdb/v1/flux/flux-vs-influxql/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Flux is an alternative to [InfluxQL](/influxdb/v1/query_language/) and other SQL-like query languages for querying and analyzing data. Flux uses functional language patterns making it incredibly powerful, flexible, and able to overcome many of the limitations of InfluxQL. This article outlines many of the tasks possible with Flux but not InfluxQL and provides information about Flux and InfluxQL parity. * [Possible with Flux](#possible-with-flux) * [InfluxQL and Flux parity](#influxql-and-flux-parity) ## Possible with Flux * [Joins](#joins) * [Math across measurements](#math-across-measurements) * [Sort by tags](#sort-by-tags) * [Group by any column](#group-by-any-column) * [Window by calendar months and years](#window-by-calendar-months-and-years) * [Work with multiple data sources](#work-with-multiple-data-sources) * [DatePart-like queries](#datepart-like-queries) * [Pivot](#pivot) * [Histograms](#histograms) * [Covariance](#covariance) * [Cast booleans to integers](#cast-booleans-to-integers) * [String manipulation and data shaping](#string-manipulation-and-data-shaping) * [Work with geo-temporal data](#work-with-geo-temporal-data) ### Joins InfluxQL has never supported joins. They can be accomplished using [TICKscript](/kapacitor/v1/reference/tick/introduction/), but even TICKscript’s join capabilities are limited. Flux’s [`join()` function](/flux/v0/stdlib/universe/join/) lets you to join data **from any bucket, any measurement, and on any columns** as long as each data set includes the columns on which they are to be joined. This opens the door for really powerful and useful operations. ```js dataStream1 = from(bucket: "bucket1") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "network" and r._field == "bytes-transferred") dataStream2 = from(bucket: "bucket1") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "httpd" and r._field == "requests-per-sec") join(tables: {d1: dataStream1, d2: dataStream2}, on: ["_time", "_stop", "_start", "host"]) ``` *For an in-depth walkthrough of using the `join()` function, see [How to join data with Flux](/influxdb/v1/flux/guides/join).* ### Math across measurements Being able to perform cross-measurement joins also allows you to run calculations using data from separate measurements – a highly requested feature from the InfluxData community. The example below takes two data streams from separate measurements, `mem` and `processes`, joins them, then calculates the average amount of memory used per running process: ```js // Memory used (in bytes) memUsed = from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used") // Total processes running procTotal = from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "processes" and r._field == "total") // Join memory used with total processes and calculate // the average memory (in MB) used for running processes. join(tables: {mem: memUsed, proc: procTotal}, on: ["_time", "_stop", "_start", "host"]) |> map(fn: (r) => ({_time: r._time, _value: r._value_mem / r._value_proc / 1000000})) ``` ### Sort by tags InfluxQL’s sorting capabilities are very limited, allowing you only to control the sort order of `time` using the `ORDER BY time` clause. Flux’s [`sort()` function](/flux/v0/stdlib/universe/sort) sorts records based on list of columns. Depending on the column type, records are sorted lexicographically, numerically, or chronologically. ```js from(bucket: "telegraf/autogen") |> range(start: -12h) |> filter(fn: (r) => r._measurement == "system" and r._field == "uptime") |> sort(columns: ["region", "host", "_value"]) ``` ### Group by any column InfluxQL lets you group by tags or by time intervals, but nothing else. Flux lets you group by any column in the dataset, including `_value`. Use the Flux [`group()` function](/flux/v0/stdlib/universe/group/)to define which columns to group data by. ```js from(bucket:"telegraf/autogen") |> range(start:-12h) |> filter(fn: (r) => r._measurement == "system" and r._field == "uptime" ) |> group(columns:["host", "_value"]) ``` ### Window by calendar months and years InfluxQL does not support windowing data by calendar months and years due to their varied lengths. Flux supports calendar month and year duration units (`1mo`, `1y`) and lets you window and aggregate data by calendar month and year. ```js from(bucket:"telegraf/autogen") |> range(start:-1y) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent" ) |> aggregateWindow(every: 1mo, fn: mean) ``` ### Work with multiple data sources InfluxQL can only query data stored in InfluxDB. Flux can query data from other data sources such as CSV, PostgreSQL, MySQL, Google BigTable, and more. Join that data with data in InfluxDB to enrich query results. * [Flux CSV package](/flux/v0/stdlib/csv/) * [Flux SQL package](/flux/v0/stdlib/sql/) * [Flux BigTable package](/flux/v0/stdlib/experimental/bigtable/) ```js import "csv" import "sql" csvData = csv.from(csv: rawCSV) sqlData = sql.from( driverName: "postgres", dataSourceName: "postgresql://user:password@localhost", query: "SELECT * FROM example_table", ) data = from(bucket: "telegraf/autogen") |> range(start: -24h) |> filter(fn: (r) => r._measurement == "sensor") auxData = join(tables: {csv: csvData, sql: sqlData}, on: ["sensor_id"]) enrichedData = join(tables: {data: data, aux: auxData}, on: ["sensor_id"]) enrichedData |> yield(name: "enriched_data") ``` *For an in-depth walkthrough of querying SQL data, see [Query SQL data sources](/influxdb/v1/flux/guides/sql).* ### DatePart-like queries InfluxQL doesn’t support DatePart-like queries that only return results during specified hours of the day. The Flux [`hourSelection` function](/flux/v0/stdlib/universe/hourselection/)returns only data with time values in a specified hour range. ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu" and r.cpu == "cpu-total") |> hourSelection(start: 9, stop: 17) ``` ### Pivot Pivoting data tables has never been supported in InfluxQL. The Flux [`pivot()` function](/flux/v0/stdlib/universe/pivot) provides the ability to pivot data tables by specifying `rowKey`, `columnKey`, and `valueColumn` parameters. ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu" and r.cpu == "cpu-total") |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") ``` ### Histograms The ability to generate histograms has been a highly requested feature for InfluxQL, but has never been supported. Flux’s [`histogram()` function](/flux/v0/stdlib/universe/histogram) uses input data to generate a cumulative histogram with support for other histogram types coming in the future. ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") |> histogram(bins: [10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0]) ``` *For an example of using Flux to create a cumulative histogram, see [Create histograms](/influxdb/v1/flux/guides/histograms).* ### Covariance Flux provides functions for simple covariance calculation. The [`covariance()` function](/flux/v0/stdlib/universe/covariance)calculates the covariance between two columns and the [`cov()` function](/flux/v0/stdlib/universe/cov)calculates the covariance between two data streams. ###### Covariance between two columns ```js from(bucket: "telegraf/autogen") |> range(start: -5m) |> covariance(columns: ["x", "y"]) ``` ###### Covariance between two streams of data ```js table1 = from(bucket: "telegraf/autogen") |> range(start: -15m) |> filter(fn: (r) => r._measurement == "measurement_1") table2 = from(bucket: "telegraf/autogen") |> range(start: -15m) |> filter(fn: (r) => r._measurement == "measurement_2") cov(x: table1, y: table2, on: ["_time", "_field"]) ``` ### Cast booleans to integers InfluxQL supports type casting, but only for numeric data types (floats to integers and vice versa).[Flux type conversion functions](/flux/v0/function-types#type-conversions)provide much broader support for type conversions and let you perform some long-requested operations like casting a boolean values to integers. ##### Cast boolean field values to integers ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "m" and r._field == "bool_field") |> toInt() ``` ### String manipulation and data shaping InfluxQL doesn’t support string manipulation when querying data. The [Flux Strings package](/flux/v0/stdlib/strings/) is a collection of functions that operate on string data. When combined with the [`map()` function](/flux/v0/stdlib/universe/map/), functions in the string package allow for operations like string sanitization and normalization. ```js import "strings" from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "weather" and r._field == "temp") |> map( fn: (r) => ({ r with location: strings.toTitle(v: r.location), sensor: strings.replaceAll(v: r.sensor, t: " ", u: "-"), status: strings.substring(v: r.status, start: 0, end: 8), }) ) ``` ### Work with geo-temporal data InfluxQL doesn’t provide functionality for working with geo-temporal data. The [Flux Geo package](/flux/v0/stdlib/experimental/geo/) is a collection of functions that let you shape, filter, and group geo-temporal data. ```js import "experimental/geo" from(bucket: "geo/autogen") |> range(start: -1w) |> filter(fn: (r) => r._measurement == "taxi") |> geo.shapeData(latField: "latitude", lonField: "longitude", level: 20) |> geo.filterRows(region: {lat: 40.69335938, lon: -73.30078125, radius: 20.0}, strict: true) |> geo.asTracks(groupBy: ["fare-id"]) ``` ## InfluxQL and Flux parity The table below shows InfluxQL statements, clauses, and functions along with their equivalent Flux functions. *For a complete list of Flux functions, [view all Flux functions](/flux/v0/stdlib/all-functions).* | InfluxQL | Flux Functions | |------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------| | [SELECT](/influxdb/v1/query_language/explore-data/#the-basic-select-statement) | [filter()](/flux/v0/stdlib/universe/filter/) | | [WHERE](/influxdb/v1/query_language/explore-data/#the-where-clause) | [filter()](/flux/v0/stdlib/universe/filter/), [range()](/flux/v0/stdlib/universe/range/) | | [GROUP BY](/influxdb/v1/query_language/explore-data/#the-group-by-clause) | [group()](/flux/v0/stdlib/universe/group/) | | [INTO](/influxdb/v1/query_language/explore-data/#the-into-clause) | [to()](/flux/v0/stdlib/influxdata/influxdb/to/) [\*](#footnote) | | [ORDER BY](/influxdb/v1/query_language/explore-data/#order-by-time-desc) | [sort()](/flux/v0/stdlib/universe/sort/) | | [LIMIT](/influxdb/v1/query_language/explore-data/#the-limit-clause) | [limit()](/flux/v0/stdlib/universe/limit/) | | [SLIMIT](/influxdb/v1/query_language/explore-data/#the-slimit-clause) | – | | [OFFSET](/influxdb/v1/query_language/explore-data/#the-offset-clause) | – | | [SOFFSET](/influxdb/v1/query_language/explore-data/#the-soffset-clause) | – | | [SHOW DATABASES](/influxdb/v1/query_language/explore-schema/#show-databases) | [buckets()](/flux/v0/stdlib/influxdata/influxdb/buckets/) | | [SHOW MEASUREMENTS](/influxdb/v1/query_language/explore-schema/#show-measurements) | [v1.measurements](/flux/v0/stdlib/influxdata/influxdb/v1/measurements) | | [SHOW FIELD KEYS](/influxdb/v1/query_language/explore-schema/#show-field-keys) | [keys()](/flux/v0/stdlib/universe/keys/) | | [SHOW RETENTION POLICIES](/influxdb/v1/query_language/explore-schema/#show-retention-policies) | [buckets()](/flux/v0/stdlib/influxdata/influxdb/buckets/) | | [SHOW TAG KEYS](/influxdb/v1/query_language/explore-schema/#show-tag-keys) | [v1.tagKeys()](/flux/v0/stdlib/influxdata/influxdb/v1/tagkeys), [v1.measurementTagKeys()](/flux/v0/stdlib/influxdata/influxdb/v1/measurementtagkeys) | | [SHOW TAG VALUES](/influxdb/v1/query_language/explore-schema/#show-tag-values) |[v1.tagValues()](/flux/v0/stdlib/influxdata/influxdb/v1/tagvalues), [v1.measurementTagValues()](/flux/v0/stdlib/influxdata/influxdb/v1/measurementtagvalues)| | [SHOW SERIES](/influxdb/v1/query_language/explore-schema/#show-series) | – | | [CREATE DATABASE](/influxdb/v1/query_language/manage-database/#create-database) | – | | [DROP DATABASE](/influxdb/v1/query_language/manage-database/#delete-a-database-with-drop-database) | – | | [DROP SERIES](/influxdb/v1/query_language/manage-database/#drop-series-from-the-index-with-drop-series) | – | | [DELETE](/influxdb/v1/query_language/manage-database/#delete-series-with-delete) | – | | [DROP MEASUREMENT](/influxdb/v1/query_language/manage-database/#delete-measurements-with-drop-measurement) | – | | [DROP SHARD](/influxdb/v1/query_language/manage-database/#delete-a-shard-with-drop-shard) | – | |[CREATE RETENTION POLICY](/influxdb/v1/query_language/manage-database/#create-retention-policies-with-create-retention-policy)| – | | [ALTER RETENTION POLICY](/influxdb/v1/query_language/manage-database/#modify-retention-policies-with-alter-retention-policy) | – | | [DROP RETENTION POLICY](/influxdb/v1/query_language/manage-database/#delete-retention-policies-with-drop-retention-policy) | – | | [COUNT](/influxdb/v1/query_language/functions#count) | [count()](/flux/v0/stdlib/universe/count/) | | [DISTINCT](/influxdb/v1/query_language/functions#distinct) | [distinct()](/flux/v0/stdlib/universe/distinct/) | | [INTEGRAL](/influxdb/v1/query_language/functions#integral) | [integral()](/flux/v0/stdlib/universe/integral/) | | [MEAN](/influxdb/v1/query_language/functions#mean) | [mean()](/flux/v0/stdlib/universe/mean/) | | [MEDIAN](/influxdb/v1/query_language/functions#median) | [median()](/flux/v0/stdlib/universe/median/) | | [MODE](/influxdb/v1/query_language/functions#mode) | [mode()](/flux/v0/stdlib/universe/mode/) | | [SPREAD](/influxdb/v1/query_language/functions#spread) | [spread()](/flux/v0/stdlib/universe/spread/) | | [STDDEV](/influxdb/v1/query_language/functions#stddev) | [stddev()](/flux/v0/stdlib/universe/stddev/) | | [SUM](/influxdb/v1/query_language/functions#sum) | [sum()](/flux/v0/stdlib/universe/sum/) | | [BOTTOM](/influxdb/v1/query_language/functions#bottom) | [bottom()](/flux/v0/stdlib/universe/bottom/) | | [FIRST](/influxdb/v1/query_language/functions#first) | [first()](/flux/v0/stdlib/universe/first/) | | [LAST](/influxdb/v1/query_language/functions#last) | [last()](/flux/v0/stdlib/universe/last/) | | [MAX](/influxdb/v1/query_language/functions#max) | [max()](/flux/v0/stdlib/universe/max/) | | [MIN](/influxdb/v1/query_language/functions#min) | [min()](/flux/v0/stdlib/universe/min/) | | [PERCENTILE](/influxdb/v1/query_language/functions#percentile) | [quantile()](/flux/v0/stdlib/universe/quantile/) | | [SAMPLE](/influxdb/v1/query_language/functions#sample) | [sample()](/flux/v0/stdlib/universe/sample/) | | [TOP](/influxdb/v1/query_language/functions#top) | [top()](/flux/v0/stdlib/universe/top/) | | [ABS](/influxdb/v1/query_language/functions#abs) | [math.abs()](/flux/v0/stdlib/math/abs/) | | [ACOS](/influxdb/v1/query_language/functions#acos) | [math.acos()](/flux/v0/stdlib/math/acos/) | | [ASIN](/influxdb/v1/query_language/functions#asin) | [math.asin()](/flux/v0/stdlib/math/asin/) | | [ATAN](/influxdb/v1/query_language/functions#atan) | [math.atan()](/flux/v0/stdlib/math/atan/) | | [ATAN2](/influxdb/v1/query_language/functions#atan2) | [math.atan2()](/flux/v0/stdlib/math/atan2/) | | [CEIL](/influxdb/v1/query_language/functions#ceil) | [math.ceil()](/flux/v0/stdlib/math/ceil/) | | [COS](/influxdb/v1/query_language/functions#cos) | [math.cos()](/flux/v0/stdlib/math/cos/) | | [CUMULATIVE\_SUM](/influxdb/v1/query_language/functions#cumulative_sum) | [cumulativeSum()](/flux/v0/stdlib/universe/cumulativesum/) | | [DERIVATIVE](/influxdb/v1/query_language/functions#derivative) | [derivative()](/flux/v0/stdlib/universe/derivative/) | | [DIFFERENCE](/influxdb/v1/query_language/functions#difference) | [difference()](/flux/v0/stdlib/universe/difference/) | | [ELAPSED](/influxdb/v1/query_language/functions#elapsed) | [elapsed()](/flux/v0/stdlib/universe/elapsed/) | | [EXP](/influxdb/v1/query_language/functions#exp) | [math.exp()](/flux/v0/stdlib/math/exp/) | | [FLOOR](/influxdb/v1/query_language/functions#floor) | [math.floor()](/flux/v0/stdlib/math/floor/) | | [HISTOGRAM](/influxdb/v1/query_language/functions#histogram) | [histogram()](/flux/v0/stdlib/universe/histogram/) | | [LN](/influxdb/v1/query_language/functions#ln) | [math.log()](/flux/v0/stdlib/math/log/) | | [LOG](/influxdb/v1/query_language/functions#log) | [math.logb()](/flux/v0/stdlib/math/logb/) | | [LOG2](/influxdb/v1/query_language/functions#log2) | [math.log2()](/flux/v0/stdlib/math/log2/) | | [LOG10](/influxdb/v1/query_language/functions/#log10) | [math.log10()](/flux/v0/stdlib/math/log10/) | | [MOVING\_AVERAGE](/influxdb/v1/query_language/functions#moving_average) | [movingAverage()](/flux/v0/stdlib/universe/movingaverage/) | | [NON\_NEGATIVE\_DERIVATIVE](/influxdb/v1/query_language/functions#non_negative_derivative) | [derivative(nonNegative:true)](/flux/v0/stdlib/universe/derivative/) | | [NON\_NEGATIVE\_DIFFERENCE](/influxdb/v1/query_language/functions#non_negative_difference) | [difference(nonNegative:true)](/flux/v0/stdlib/universe/derivative/) | | [POW](/influxdb/v1/query_language/functions#pow) | [math.pow()](/flux/v0/stdlib/math/pow/) | | [ROUND](/influxdb/v1/query_language/functions#round) | [math.round()](/flux/v0/stdlib/math/round/) | | [SIN](/influxdb/v1/query_language/functions#sin) | [math.sin()](/flux/v0/stdlib/math/sin/) | | [SQRT](/influxdb/v1/query_language/functions#sqrt) | [math.sqrt()](/flux/v0/stdlib/math/sqrt/) | | [TAN](/influxdb/v1/query_language/functions#tan) | [math.tan()](/flux/v0/stdlib/math/tan/) | | [HOLT\_WINTERS](/influxdb/v1/query_language/functions#holt_winters) | [holtWinters()](/flux/v0/stdlib/universe/holtwinters/) | | [CHANDE\_MOMENTUM\_OSCILLATOR](/influxdb/v1/query_language/functions#chande_momentum_oscillator) | [chandeMomentumOscillator()](/flux/v0/stdlib/universe/chandemomentumoscillator/) | | [EXPONENTIAL\_MOVING\_AVERAGE](/influxdb/v1/query_language/functions#exponential_moving_average) | [exponentialMovingAverage()](/flux/v0/stdlib/universe/exponentialmovingaverage/) | | [DOUBLE\_EXPONENTIAL\_MOVING\_AVERAGE](/influxdb/v1/query_language/functions#double_exponential_moving_average) | [doubleEMA()](/flux/v0/stdlib/universe/doubleema/) | | [KAUFMANS\_EFFICIENCY\_RATIO](/influxdb/v1/query_language/functions#kaufmans_efficiency_ratio) | [kaufmansER()](/flux/v0/stdlib/universe/kaufmanser/) | | [KAUFMANS\_ADAPTIVE\_MOVING\_AVERAGE](/influxdb/v1/query_language/functions#kaufmans_adaptive_moving_average) | [kaufmansAMA()](/flux/v0/stdlib/universe/kaufmansama/) | | [TRIPLE\_EXPONENTIAL\_MOVING\_AVERAGE](/influxdb/v1/query_language/functions#triple_exponential_moving_average) | [tripleEMA()](/flux/v0/stdlib/universe/tripleema/) | | [TRIPLE\_EXPONENTIAL\_DERIVATIVE](/influxdb/v1/query_language/functions#triple_exponential_derivative) | [tripleExponentialDerivative()](/flux/v0/stdlib/universe/tripleexponentialderivative/) | | [RELATIVE\_STRENGTH\_INDEX](/influxdb/v1/query_language/functions#relative_strength_index) | [relativeStrengthIndex()](/flux/v0/stdlib/universe/relativestrengthindex/) | *\* The `to()` function only writes to InfluxDB 2.0.* | InfluxQL | Flux Functions | | --- | --- | | InfluxQL | Flux Functions | | SELECT | filter() | | WHERE | filter() , range() | | GROUP BY | group() | | INTO | to() * | | ORDER BY | sort() | | LIMIT | limit() | | SLIMIT | – | | OFFSET | – | | SOFFSET | – | | SHOW DATABASES | buckets() | | SHOW MEASUREMENTS | v1.measurements | | SHOW FIELD KEYS | keys() | | SHOW RETENTION POLICIES | buckets() | | SHOW TAG KEYS | v1.tagKeys() , v1.measurementTagKeys() | | SHOW TAG VALUES | v1.tagValues() , v1.measurementTagValues() | | SHOW SERIES | – | | CREATE DATABASE | – | | DROP DATABASE | – | | DROP SERIES | – | | DELETE | – | | DROP MEASUREMENT | – | | DROP SHARD | – | | CREATE RETENTION POLICY | – | | ALTER RETENTION POLICY | – | | DROP RETENTION POLICY | – | | COUNT | count() | | DISTINCT | distinct() | | INTEGRAL | integral() | | MEAN | mean() | | MEDIAN | median() | | MODE | mode() | | SPREAD | spread() | | STDDEV | stddev() | | SUM | sum() | | BOTTOM | bottom() | | FIRST | first() | | LAST | last() | | MAX | max() | | MIN | min() | | PERCENTILE | quantile() | | SAMPLE | sample() | | TOP | top() | | ABS | math.abs() | | ACOS | math.acos() | | ASIN | math.asin() | | ATAN | math.atan() | | ATAN2 | math.atan2() | | CEIL | math.ceil() | | COS | math.cos() | | CUMULATIVE_SUM | cumulativeSum() | | DERIVATIVE | derivative() | | DIFFERENCE | difference() | | ELAPSED | elapsed() | | EXP | math.exp() | | FLOOR | math.floor() | | HISTOGRAM | histogram() | | LN | math.log() | | LOG | math.logb() | | LOG2 | math.log2() | | LOG10 | math.log10() | | MOVING_AVERAGE | movingAverage() | | NON_NEGATIVE_DERIVATIVE | derivative(nonNegative:true) | | NON_NEGATIVE_DIFFERENCE | difference(nonNegative:true) | | POW | math.pow() | | ROUND | math.round() | | SIN | math.sin() | | SQRT | math.sqrt() | | TAN | math.tan() | | HOLT_WINTERS | holtWinters() | | CHANDE_MOMENTUM_OSCILLATOR | chandeMomentumOscillator() | | EXPONENTIAL_MOVING_AVERAGE | exponentialMovingAverage() | | DOUBLE_EXPONENTIAL_MOVING_AVERAGE | doubleEMA() | | KAUFMANS_EFFICIENCY_RATIO | kaufmansER() | | KAUFMANS_ADAPTIVE_MOVING_AVERAGE | kaufmansAMA() | | TRIPLE_EXPONENTIAL_MOVING_AVERAGE | tripleEMA() | | TRIPLE_EXPONENTIAL_DERIVATIVE | tripleExponentialDerivative() | | RELATIVE_STRENGTH_INDEX | relativeStrengthIndex() | --- # Get started with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/get-started/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Flux is InfluxData’s new functional data scripting language designed for querying, analyzing, and acting on data. This multi-part getting started guide walks through important concepts related to Flux. It covers querying time series data from InfluxDB using Flux, and introduces Flux syntax and functions. ## What you will need ##### InfluxDB v1.8+ Flux v0.65 is built into InfluxDB v1.8+ and can be used to query data stored in InfluxDB. The `influx` CLI also includes a `-type=flux` option that starts an interactive Flux Read-Eval-Print-Loop (REPL), which lets you write and run Flux queries from the command line. *For information about downloading and installing InfluxDB, see [InfluxDB installation](/influxdb/v1/introduction/installation).* ##### Chronograf v1.8+ **Not required but strongly recommended**. Chronograf v1.8’s Data Explorer provides a user interface (UI) for writing Flux scripts and visualizing results. Dashboards in Chronograf v1.8+ also support Flux queries. *For information about downloading and installing Chronograf, see [Chronograf installation](/chronograf/v1/introduction/installation).* ## Key concepts Flux introduces important new concepts you should understand as you get started. ### Buckets Flux introduces “buckets,” a new data storage concept for InfluxDB. A **bucket** is a named location where data is stored that has a retention policy. It’s similar to an InfluxDB v1.x “database,” but is a combination of both a database and a retention policy. When using multiple retention policies, each retention policy is treated as is its own bucket. Flux’s [`from()` function](/flux/v0/stdlib/influxdata/influxdb/from/), which defines an InfluxDB data source, requires a `bucket` parameter. When using Flux with InfluxDB v1.x, use the following bucket naming convention which combines the database name and retention policy into a single bucket name: ###### InfluxDB v1.x bucket naming convention ```js // Pattern from(bucket:"/") // Example from(bucket:"telegraf/autogen") ``` ### Pipe-forward operator Flux uses pipe-forward operators (`|>`) extensively to chain operations together. After each function or operation, Flux returns a table or collection of tables containing data. The pipe-forward operator pipes those tables into the next function or operation where they are further processed or manipulated. ### Tables Flux structures all data in tables. When data is streamed from data sources, Flux formats it as annotated comma-separated values (CSV), representing tables. Functions then manipulate or process them and output new tables. This makes it easy to chain together functions to build sophisticated queries. #### Group keys Every table has a **group key** which describes the contents of the table. It’s a list of columns for which every row in the table will have the same value. Columns with unique values in each row are **not** part of the group key. As functions process and transform data, each modifies the group keys of output tables. Understanding how tables and group keys are modified by functions is key to properly shaping your data for the desired output. ###### Example group key ```js [_start, _stop, _field, _measurement, host] ``` Note that `_time` and `_value` are excluded from the example group key because they are unique to each row. ## Tools for working with Flux You have multiple [options for writing and running Flux queries](/influxdb/v1/flux/guides/execute-queries/), but as you’re getting started, we recommend using the following: ### 1. Chronograf’s Data Explorer Chronograf’s Data Explorer makes it easy to write your first Flux script and visualize the results. To use Chronograf’s Flux UI, open the **Data Explorer** and to the right of the source dropdown above the graph placeholder, select **Flux** as the source type. This will provide **Schema**, **Script**, and **Functions** panes. The Schema pane allows you to explore your data. The Script pane is where you write your Flux script. The Functions pane provides a list of functions available in your Flux queries. ### 2. influx CLI The `influx` CLI is an interactive shell for querying InfluxDB. Use the `-type=flux` and `-path-prefix=/api/v2/query` options to open a Flux REPL where you write and run Flux queries. For the required InfluxDB version and the authentication options, see[Execute Flux queries](/influxdb/v1/flux/guides/execute-queries/). ```bash influx -type=flux -path-prefix=/api/v2/query ``` [Query InfluxDB with Flux](/influxdb/v1/flux/get-started/query-influxdb/) --- # Query InfluxDB with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/get-started/query-influxdb/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. This guide walks through the basics of using Flux to query data from InfluxDB.***If you haven’t already, make sure to install InfluxDB v1.8+, [enable Flux](/influxdb/v1/flux/installation), and choose a [tool for writing Flux queries](/influxdb/v1/flux/get-started#tools-for-working-with-flux).*** Every Flux query needs the following: 1. [A data source](#1-define-your-data-source) 2. [A time range](#2-specify-a-time-range) 3. [Data filters](#3-filter-your-data) ## 1. Define your data source Flux’s [`from()`](/flux/v0/stdlib/influxdata/influxdb/from/) function defines an InfluxDB data source. It requires a [`bucket`](/influxdb/v1/flux/get-started/#buckets) parameter. For this example, use `telegraf/autogen`, a combination of the default database and retention policy provided by the TICK stack. ```js from(bucket:"telegraf/autogen") ``` ## 2. Specify a time range Flux requires a time range when querying time series data. “Unbounded” queries are very resource-intensive and as a protective measure, Flux will not query the database without a specified range. Use the pipe-forward operator (`|>`) to pipe data from your data source into the [`range()`](/flux/v0/stdlib/universe/range)function, which specifies a time range for your query. It accepts two properties: `start` and `stop`. Ranges can be **relative** using negative [durations](/flux/v0/language/lexical-elements#duration-literals)or **absolute** using [timestamps](/flux/v0/language/lexical-elements#date-and-time-literals). ###### Example relative time ranges ```js // Relative time range with start only. Stop defaults to now. from(bucket:"telegraf/autogen") |> range(start: -1h) // Relative time range with start and stop from(bucket:"telegraf/autogen") |> range(start: -1h, stop: -10m) ``` > > > Relative ranges are relative to “now.” > > ###### Example absolute time range ```js from(bucket:"telegraf/autogen") |> range(start: 2018-11-05T23:30:00Z, stop: 2018-11-06T00:00:00Z) ``` #### Use the following: For this guide, use the relative time range, `-15m`, to limit query results to data from the last 15 minutes: ```js from(bucket:"telegraf/autogen") |> range(start: -15m) ``` ## 3. Filter your data Pass your ranged data into the `filter()` function to narrow results based on data attributes or columns. The `filter()` function has one parameter, `fn`, which expects an anonymous function with logic that filters data based on columns or attributes. Flux’s anonymous function syntax is very similar to JavaScript’s. Records or rows are passed into the `filter()` function as a record (`r`). The anonymous function takes the record and evaluates it to see if it matches the defined filters. Use the `AND` relational operator to chain multiple filters. ```js // Pattern (r) => (r.recordProperty comparisonOperator comparisonExpression) // Example with single filter (r) => (r._measurement == "cpu") // Example with multiple filters (r) => (r._measurement == "cpu") and (r._field != "usage_system" ) ``` #### Use the following: For this example, filter by the `cpu` measurement, the `usage_system` field, and the `cpu-total` tag value: ```js from(bucket:"telegraf/autogen") |> range(start: -15m) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total" ) ``` ## 4. Yield your queried data Use Flux’s `yield()` function to output the filtered tables as the result of the query. ```js from(bucket:"telegraf/autogen") |> range(start: -15m) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total" ) |> yield() ``` > > > Chronograf and the `influx` CLI automatically assume a `yield()` function at > the end of each script in order to output and visualize the data. > Best practice is to include a `yield()` function, but it is not always necessary. > > ## Congratulations! You have now queried data from InfluxDB using Flux. The query shown here is a barebones example. Flux queries can be extended in many ways to form powerful scripts. [Get started with Flux](/influxdb/v1/flux/get-started/)[Transform your data](/influxdb/v1/flux/get-started/transform-data/) --- # Flux syntax basics Source: https://docs.influxdata.com/influxdb/v1/flux/get-started/syntax-basics/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Flux is a scripting language designed specifically for working with data. This guide walks through a handful of simple expressions and how they are handled in Flux. ## Use the influx CLI Use the `influx` CLI in “Flux mode” as you follow this guide. When started with `-type=flux` and `-path-prefix=/api/v2/query`, the `influx`CLI is an interactive read-eval-print-loop (REPL) that supports Flux syntax. ##### Start in the influx CLI in Flux mode ```bash influx -type=flux -path-prefix=/api/v2/query ``` For the required InfluxDB version and the authentication options, see[Execute Flux queries](/influxdb/v1/flux/guides/execute-queries/). ## Basic Flux syntax The code blocks below provide commands that illustrate the basic syntax of Flux. Run these commands in the `influx` CLI’s Flux REPL. ### Simple expressions Flux is a scripting language that supports basic expressions. For example, simple addition: ```js > 1 + 1 2 ``` ### Variables Assign an expression to a variable using the assignment operator, `=`. ``` s = "this is a string" i = 1 // an integer f = 2.0 // a floating point number ``` Type the name of a variable to print its value: ```js > s this is a string > i 1 > f 2 ``` ### Records Flux also supports records. Each value in a record can be a different data type. ```js o = {name:"Jim", age: 42, "favorite color": "red"} ``` Use **dot notation** to access a properties of a record: ```js > o.name Jim > o.age 42 ``` Or **bracket notation**: ```js > o["name"] Jim > o["age"] 42 > o["favorite color"] red ``` > [!Note] > Use bracket notation to reference record properties with special or > white space characters in the property key. ### Lists Flux supports lists. List values must be the same type. ```js > n = 4 > l = [1,2,3,n] > l [1, 2, 3, 4] ``` ### Functions Flux uses functions for most of its heavy lifting. Below is a simple function that squares a number, `n`. ```js > square = (n) => n * n > square(n:3) 9 ``` > > > Flux does not support positional arguments or parameters. > Parameters must always be named when calling a function. > > ### Pipe-forward operator Flux uses the pipe-forward operator (`|>`) extensively to chain operations together. After each function or operation, Flux returns a table or collection of tables containing data. The pipe-forward operator pipes those tables into the next function where they are further processed or manipulated. ```js data |> someFunction() |> anotherFunction() ``` ## Real-world application of basic syntax This likely seems familiar if you’ve already been through the other [getting started guides](/influxdb/v1/flux/get-started). Flux’s syntax is inspired by JavaScript and other functional scripting languages. As you begin to apply these basic principles in real-world use cases such as creating data stream variables, custom functions, etc., the power of Flux and its ability to query and process data will become apparent. The examples below provide both multi-line and single-line versions of each input command. Carriage returns in Flux aren’t necessary, but do help with readability. Both single- and multi-line commands can be copied and pasted into the `influx` CLI running in Flux mode. [Multi-line inputs](#multi-line-inputs)[Single-line inputs](#single-line-inputs) ### Define data stream variables A common use case for variable assignments in Flux is creating variables for one or more input data streams. ```js timeRange = -1h cpuUsageUser = from(bucket: "telegraf/autogen") |> range(start: timeRange) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_user" and r.cpu == "cpu-total") memUsagePercent = from(bucket: "telegraf/autogen") |> range(start: timeRange) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") ``` These variables can be used in other functions, such as `join()`, while keeping the syntax minimal and flexible. ### Define custom functions Create a function that returns the `N` number rows in the input stream with the highest `_value`s. To do this, pass the input stream (`tables`) and the number of results to return (`n`) into a custom function. Then using Flux’s `sort()` and `limit()` functions to find the top `n` results in the data set. ```js topN = (tables=<-, n) => tables |> sort(desc: true) |> limit(n: n) ``` *More information about creating custom functions is available in the [Custom functions](/influxdb/v2/query-data/flux/custom-functions) documentation.* Using this new custom function `topN` and the `cpuUsageUser` data stream variable defined above, find the top five data points and yield the results. ```js cpuUsageUser |> topN(n: 5) |> yield() ``` ### Define data stream variables A common use case for variable assignments in Flux is creating variables for multiple filtered input data streams. ```js timeRange = -1h cpuUsageUser = from(bucket: "telegraf/autogen") |> range(start: timeRange) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_user" and r.cpu == "cpu-total") memUsagePercent = from(bucket: "telegraf/autogen") |> range(start: timeRange) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") ``` These variables can be used in other functions, such as `join()`, while keeping the syntax minimal and flexible. ### Define custom functions Let’s create a function that returns the `N` number rows in the input data stream with the highest `_value`s. To do this, pass the input stream (`tables`) and the number of results to return (`n`) into a custom function. Then using Flux’s `sort()` and `limit()` functions to find the top `n` results in the data set. ```js topN = (tables=<-, n) => tables |> sort(desc: true) |> limit(n: n) ``` *More information about creating custom functions is available in the [Custom functions](/influxdb/v2/query-data/flux/custom-functions) documentation.* Using the `cpuUsageUser` data stream variable defined [above](#define-data-stream-variables), find the top five data points with the custom `topN` function and yield the results. ```js cpuUsageUser |> topN(n:5) |> yield() ``` This query will return the five data points with the highest user CPU usage over the last hour. [Transform your data](/influxdb/v1/flux/get-started/transform-data/) --- # Transform data with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/get-started/transform-data/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. When [querying data from InfluxDB](/influxdb/v1/flux/get-started/query-influxdb), you often need to transform that data in some way. Common examples are aggregating data into averages, downsampling data, etc. This guide demonstrates using [Flux functions](/flux/v0/stdlib/) to transform your data. It walks through creating a Flux script that partitions data into windows of time, averages the `_value`s in each window, and outputs the averages as a new table. It’s important to understand how the “shape” of your data changes through each of these operations. ## Query data Use the query built in the previous [Query data from InfluxDB](/influxdb/v1/flux/get-started/query-influxdb)guide, but update the range to pull data from the last hour: ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total") ``` ## Flux functions Flux provides a number of functions that perform specific operations, transformations, and tasks. You can also [create custom functions](/influxdb/v2/query-data/flux/custom-functions) in your Flux queries.*Functions are covered in detail in the [Flux standard library](/flux/v0/stdlib/) documentation.* A common type of function used when transforming data queried from InfluxDB is an aggregate function. Aggregate functions take a set of `_value`s in a table, aggregate them, and transform them into a new value. This example uses the [`mean()` function](/flux/v0/stdlib/universe/mean)to average values within time windows. > > > The following example walks through the steps required to window and aggregate data, > but there is a [`aggregateWindow()` helper function](#helper-functions) that does it for you. > It’s just good to understand the steps in the process. > > ## Window your data Flux’s [`window()` function](/flux/v0/stdlib/universe/window) partitions records based on a time value. Use the `every` parameter to define a duration of time for each window. > [!Note] > #### Calendar months and years > > `every` supports all [valid duration units](/flux/v0/spec/types/#duration-types), > including **calendar months (`1mo`)** and **years (`1y`)**. For this example, window data in five minute intervals (`5m`). ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total") |> window(every: 5m) ``` As data is gathered into windows of time, each window is output as its own table. When visualized, each table is assigned a unique color. ![Windowed data tables](/img/flux/windowed-data.png) ## Aggregate windowed data Flux aggregate functions take the `_value`s in each table and aggregate them in some way. Use the [`mean()` function](/flux/v0/stdlib/universe/mean) to average the `_value`s of each table. ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total") |> window(every: 5m) |> mean() ``` As rows in each window are aggregated, their output table contains only a single row with the aggregate value. Windowed tables are all still separate and, when visualized, will appear as single, unconnected points. ![Windowed aggregate data](/img/flux/windowed-aggregates.png) ## Add times to your aggregates As values are aggregated, the resulting tables do not have a `_time` column because the records used for the aggregation all have different timestamps. Aggregate functions don’t infer what time should be used for the aggregate value. Therefore the `_time` column is dropped. A `_time` column is required in the [next operation](#unwindow-aggregate-tables). To add one, use the [`duplicate()` function](/flux/v0/stdlib/universe/duplicate)to duplicate the `_stop` column as the `_time` column for each windowed table. ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total") |> window(every: 5m) |> mean() |> duplicate(column: "_stop", as: "_time") ``` ## Unwindow aggregate tables Use the `window()` function with the `every: inf` parameter to gather all points into a single, infinite window. ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total") |> window(every: 5m) |> mean() |> duplicate(column: "_stop", as: "_time") |> window(every: inf) ``` Once ungrouped and combined into a single table, the aggregate data points will appear connected in your visualization. ![Unwindowed aggregate data](/img/flux/windowed-aggregates-ungrouped.png) ## Helper functions This may seem like a lot of coding just to build a query that aggregates data, however going through the process helps to understand how data changes “shape” as it is passed through each function. Flux provides (and allows you to create) “helper” functions that abstract many of these steps. The same operation performed in this guide can be accomplished using the[`aggregateWindow()` function](/flux/v0/stdlib/universe/aggregatewindow). ```js from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_system" and r.cpu == "cpu-total") |> aggregateWindow(every: 5m, fn: mean) ``` ## Congratulations! You have now constructed a Flux query that uses Flux functions to transform your data. There are many more ways to manipulate your data using both Flux’s primitive functions and your own custom functions, but this is a good introduction into the basic syntax and query structure. *For a deeper dive into windowing and aggregating data with example data output for each transformation, view the [Windowing and aggregating data](/influxdb/v1/flux/guides/window-aggregate) guide.* [Query InfluxDB](/influxdb/v1/flux/get-started/query-influxdb/)[Syntax basics](/influxdb/v1/flux/get-started/syntax-basics/) --- # Query data with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The following guides walk through both common and complex queries and use cases for Flux. > [!Note] > #### Example data variable > > Many of the examples provided in the following guides use a `data` variable, > which represents a basic query that filters data by measurement and field.`data` is defined as: > > ``` > data = from(bucket: "db/rp") > |> range(start: -1h) > |> filter(fn: (r) => r._measurement == "example-measurement" and r._field == "example-field") > ``` ## Flux query guides * [Execute Flux queries](#execute-flux-queries) * [Query fields and tags](#query-fields-and-tags) * [Group](#group) * [Sort and limit](#sort-and-limit) * [Window & aggregate](#window-amp-aggregate) * [Transform data with math](#transform-data-with-math) * [Calculate percentages](#calculate-percentages) * [Increase](#increase) * [Moving Average](#moving-average) * [Rate](#rate) * [Histograms](#histograms) * [Fill](#fill) * [Median](#median) * [Percentile & quantile](#percentile-amp-quantile) * [Join](#join) * [Cumulative sum](#cumulative-sum) * [First and last](#first-and-last) * [Exists](#exists) * [Extract scalar values](#extract-scalar-values) * [Manipulate timestamps](#manipulate-timestamps) * [Monitor states](#monitor-states) * [Query SQL data](#query-sql-data) * [Conditional logic](#conditional-logic) * [Regular expressions](#regular-expressions) * [Geo-temporal data](#geo-temporal-data) * [Optimize Flux queries](#optimize-flux-queries) * [Use Flux in Chronograf dashboards](#use-flux-in-chronograf-dashboards) ### [Execute Flux queries](/influxdb/v1/flux/guides/execute-queries/) Use the InfluxDB CLI, API, and the Chronograf Data Explorer to execute Flux queries. ### [Query fields and tags](/influxdb/v1/flux/guides/query-fields/) Use the `filter()` function to query data based on fields, tags, or any other column value. `filter()` performs operations similar to the `SELECT` statement and the `WHERE` clause in InfluxQL and other SQL-like query languages. ```js from(bucket: "db/rp") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "example-measurement" and r._field == "example-field" and r.tag == "example-tag" ) ``` ### [Group](/influxdb/v1/flux/guides/group-data/) Use the `group()` function to group data with common values in specific columns. ```js data |> group(columns: ["host"], mode: "by") ``` ###### Input: | \_time |host |\_value| |--------------------|-----|-------| |2020-01-01T00:01:00Z|host1| 1.0 | |2020-01-01T00:01:00Z|host2| 2.0 | |2020-01-01T00:02:00Z|host1| 1.0 | |2020-01-01T00:02:00Z|host2| 3.0 | ###### Output: | \_time |host |\_value| |--------------------|-----|-------| |2020-01-01T00:01:00Z|host1| 1.0 | |2020-01-01T00:02:00Z|host1| 1.0 | | \_time |host |\_value| |--------------------|-----|-------| |2020-01-01T00:01:00Z|host2| 2.0 | |2020-01-01T00:02:00Z|host2| 3.0 | ### [Sort and limit](/influxdb/v1/flux/guides/sort-limit/) Use the `sort()`function to order records within each table by specific columns and the `limit()` function to limit the number of records in output tables to a fixed number, `n`. ```js data |> sort(columns: ["host", "_value"]) |> limit(n: 4) ``` ###### Input: | \_time |host|\_value| |--------------------|----|-------| |2020-01-01T00:01:00Z| A | 1.0 | |2020-01-01T00:02:00Z| B | 1.2 | |2020-01-01T00:03:00Z| A | 1.8 | |2020-01-01T00:04:00Z| B | 0.9 | |2020-01-01T00:05:00Z| B | 1.4 | |2020-01-01T00:06:00Z| B | 2.0 | ###### Output: | \_time |host|\_value| |--------------------|----|-------| |2020-01-01T00:03:00Z| A | 1.8 | |2020-01-01T00:01:00Z| A | 1.0 | |2020-01-01T00:06:00Z| B | 2.0 | |2020-01-01T00:05:00Z| B | 1.4 | ### [Window & aggregate](/influxdb/v1/flux/guides/window-aggregate/) This guide walks through windowing and aggregating data with Flux and outlines how it shapes your data in the process. ```js data |> aggregateWindow(every: 20m, fn: mean) ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:00Z| 250 | |2020-01-01T00:04:00Z| 160 | |2020-01-01T00:12:00Z| 150 | |2020-01-01T00:19:00Z| 220 | |2020-01-01T00:32:00Z| 200 | |2020-01-01T00:51:00Z| 290 | |2020-01-01T01:00:00Z| 340 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:20:00Z| 195 | |2020-01-01T00:40:00Z| 200 | |2020-01-01T01:00:00Z| 290 | |2020-01-01T01:20:00Z| 340 | ### [Transform data with math](/influxdb/v1/flux/guides/mathematic-operations/) Use the `map()` function to remap column values and apply mathematic operations. ```js data |> map(fn: (r) => ({ r with _value: r._value * r._value })) ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 2 | |2020-01-01T00:02:00Z| 4 | |2020-01-01T00:03:00Z| 3 | |2020-01-01T00:04:00Z| 5 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 4 | |2020-01-01T00:02:00Z| 16 | |2020-01-01T00:03:00Z| 9 | |2020-01-01T00:04:00Z| 25 | ### [Calculate percentages](/influxdb/v1/flux/guides/calculate-percentages/) Use `pivot()` or `join()` and the `map()` function to align operand values into rows and calculate a percentage. ```js data |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value") |> map( fn: (r) => ({ _time: r._time, _field: "used_percent", _value: float(v: r.used) / float(v: r.total) * 100.0, }), ) ``` ###### Input: | \_time |\_field|\_value| |--------------------|-------|-------| |2020-01-01T00:00:00Z| used | 2.5 | |2020-01-01T00:00:10Z| used | 3.1 | |2020-01-01T00:00:20Z| used | 4.2 | | \_time |\_field|\_value| |--------------------|-------|-------| |2020-01-01T00:00:00Z| total | 8.0 | |2020-01-01T00:00:10Z| total | 8.0 | |2020-01-01T00:00:20Z| total | 8.0 | ###### Output: | \_time | \_field |\_value| |--------------------|-------------|-------| |2020-01-01T00:00:00Z|used\_percent| 31.25 | |2020-01-01T00:00:10Z|used\_percent| 38.75 | |2020-01-01T00:00:20Z|used\_percent| 52.50 | ### [Increase](/influxdb/v1/flux/guides/increase/) Use the `increase()` function to track increases across multiple columns in a table. This function is especially useful when tracking changes in counter values that wrap over time or periodically reset. ```js data |> increase() ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1 | |2020-01-01T00:02:00Z| 2 | |2020-01-01T00:03:00Z| 8 | |2020-01-01T00:04:00Z| 10 | |2020-01-01T00:05:00Z| 0 | |2020-01-01T00:06:00Z| 4 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:02:00Z| 1 | |2020-01-01T00:03:00Z| 7 | |2020-01-01T00:04:00Z| 9 | |2020-01-01T00:05:00Z| 9 | |2020-01-01T00:06:00Z| 13 | ### [Moving Average](/influxdb/v1/flux/guides/moving-average/) Use the `movingAverage()` or `timedMovingAverage()` functions to return the moving average of data. ```js data |> movingAverage(n: 3) ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.2 | |2020-01-01T00:03:00Z| 1.8 | |2020-01-01T00:04:00Z| 0.9 | |2020-01-01T00:05:00Z| 1.4 | |2020-01-01T00:06:00Z| 2.0 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:03:00Z| 1.33 | |2020-01-01T00:04:00Z| 1.30 | |2020-01-01T00:05:00Z| 1.36 | |2020-01-01T00:06:00Z| 1.43 | ```js data |> timedMovingAverage(every: 2m, period: 4m) ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.2 | |2020-01-01T00:03:00Z| 1.8 | |2020-01-01T00:04:00Z| 0.9 | |2020-01-01T00:05:00Z| 1.4 | |2020-01-01T00:06:00Z| 2.0 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:02:00Z| 1.000 | |2020-01-01T00:04:00Z| 1.333 | |2020-01-01T00:06:00Z| 1.325 | |2020-01-01T00:06:00Z| 1.150 | ### [Rate](/influxdb/v1/flux/guides/rate/) Use the `derivative()` function to calculate the rate of change between subsequent values or the `aggregate.rate()` function to calculate the average rate of change per window of time. If time between points varies, these functions normalize points to a common time interval making values easily comparable. ```js data |> derivative(unit: 1m, nonNegative: true) ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:00Z| 250 | |2020-01-01T00:04:00Z| 160 | |2020-01-01T00:12:00Z| 150 | |2020-01-01T00:19:00Z| 220 | |2020-01-01T00:32:00Z| 200 | |2020-01-01T00:51:00Z| 290 | |2020-01-01T01:00:00Z| 340 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:04:00Z| | |2020-01-01T00:12:00Z| | |2020-01-01T00:19:00Z| 10.0 | |2020-01-01T00:32:00Z| | |2020-01-01T00:51:00Z| 4.74 | |2020-01-01T01:00:00Z| 5.56 | ```js import "experimental/aggregate" data |> aggregate.rate(every: 20m, unit: 1m) ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:00Z| 250 | |2020-01-01T00:04:00Z| 160 | |2020-01-01T00:12:00Z| 150 | |2020-01-01T00:19:00Z| 220 | |2020-01-01T00:32:00Z| 200 | |2020-01-01T00:51:00Z| 290 | |2020-01-01T01:00:00Z| 340 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:20:00Z| | |2020-01-01T00:40:00Z| 10.0 | |2020-01-01T01:00:00Z| 4.74 | |2020-01-01T01:20:00Z| 5.56 | ### [Histograms](/influxdb/v1/flux/guides/histograms/) Use the `histogram()` function to create cumulative histograms with Flux. ```js data |> histogram( column: "_value", upperBoundColumn: "le", countColumn: "_value", bins: [100.0, 200.0, 300.0, 400.0], ) ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:00Z| 250.0 | |2020-01-01T00:01:00Z| 160.0 | |2020-01-01T00:02:00Z| 150.0 | |2020-01-01T00:03:00Z| 220.0 | |2020-01-01T00:04:00Z| 200.0 | |2020-01-01T00:05:00Z| 290.0 | |2020-01-01T01:00:00Z| 340.0 | ###### Output: | le |\_value| |-----|-------| |100.0| 0.0 | |200.0| 3.0 | |300.0| 6.0 | |400.0| 7.0 | ### [Fill](/influxdb/v1/flux/guides/fill/) Use the `fill()` function to replace *null* values. ```js data |> fill(usePrevious: true) ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| null | |2020-01-01T00:02:00Z| 0.8 | |2020-01-01T00:03:00Z| null | |2020-01-01T00:04:00Z| null | |2020-01-01T00:05:00Z| 1.4 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| null | |2020-01-01T00:02:00Z| 0.8 | |2020-01-01T00:03:00Z| 0.8 | |2020-01-01T00:04:00Z| 0.8 | |2020-01-01T00:05:00Z| 1.4 | ```js data |> fill(value: 0.0) ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| null | |2020-01-01T00:02:00Z| 0.8 | |2020-01-01T00:03:00Z| null | |2020-01-01T00:04:00Z| null | |2020-01-01T00:05:00Z| 1.4 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 0.0 | |2020-01-01T00:02:00Z| 0.8 | |2020-01-01T00:03:00Z| 0.0 | |2020-01-01T00:04:00Z| 0.0 | |2020-01-01T00:05:00Z| 1.4 | ### [Median](/influxdb/v1/flux/guides/median/) Use the `median()` function to return a value representing the `0.5` quantile or median of input data. ```js data |> median() ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | ###### Output: |\_value| |-------| | 1.5 | ### [Percentile & quantile](/influxdb/v1/flux/guides/percentile-quantile/) Use the `quantile()` function to return all values within the `q` quantile or percentile of input data. ```js data |> quantile(q: 0.99, method: "estimate_tdigest") ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | ###### Output: |\_value| |-------| | 3.0 | ### [Join](/influxdb/v1/flux/guides/join/) This guide walks through joining data with Flux and outlines how it shapes your data in the process. ``` t1 = from(bucket: "example-bucket") |> range(start: 2020-01-01T00:00:00Z) |> filter(fn: (r) => r.m == "foo") t2 = from(bucket: "example-bucket") |> range(start: 2020-01-01T00:00:00Z) |> filter(fn: (r) => r.m == "bar") join(tables: {t1: t1, t2: t2}, on: ["_time"]) ``` ###### Input: ###### t1 | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1 | |2020-01-01T00:02:00Z| 2 | |2020-01-01T00:03:00Z| 1 | |2020-01-01T00:04:00Z| 3 | ###### t2 | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 5 | |2020-01-01T00:02:00Z| 2 | |2020-01-01T00:03:00Z| 3 | |2020-01-01T00:04:00Z| 4 | ###### Output: | \_time |\_value\_t1|\_value\_t2| |--------------------|-----------|-----------| |2020-01-01T00:01:00Z| 1 | 5 | |2020-01-01T00:02:00Z| 2 | 2 | |2020-01-01T00:03:00Z| 1 | 3 | |2020-01-01T00:04:00Z| 3 | 4 | ### [Cumulative sum](/influxdb/v1/flux/guides/cumulativesum/) Use the `cumulativeSum()` function to calculate a running total of values. ```js data |> cumulativeSum() ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1 | |2020-01-01T00:02:00Z| 2 | |2020-01-01T00:03:00Z| 1 | |2020-01-01T00:04:00Z| 3 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1 | |2020-01-01T00:02:00Z| 3 | |2020-01-01T00:03:00Z| 4 | |2020-01-01T00:04:00Z| 7 | ### [First and last](/influxdb/v1/flux/guides/first-last/) Use the `first()` or `last()` functions to return the first or last point in an input table. ```js data |> first() ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | ```js data |> last() ``` ###### Input: | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | ###### Output: | \_time |\_value| |--------------------|-------| |2020-01-01T00:04:00Z| 3.0 | ### [Exists](/influxdb/v1/flux/guides/exists/) Use the `exists` operator to check if a row record contains a column or if a column’s value is *null*. ##### Filter null values ```js data |> filter(fn: (r) => exists r._value) ``` ### [Extract scalar values](/influxdb/v1/flux/guides/scalar-values/) Use Flux stream and table functions to extract scalar values from Flux query output. This lets you, for example, dynamically set variables using query results. ```js scalarValue = { _record = data |> tableFind(fn: key => true) |> getRecord(idx: 0) return _record._value } ``` ### [Manipulate timestamps](/influxdb/v1/flux/guides/manipulate-timestamps/) Use Flux to process and manipulate timestamps. ### [Monitor states](/influxdb/v1/flux/guides/monitor-states/) Flux provides several functions to help monitor states and state changes in your data. ### [Query SQL data](/influxdb/v1/flux/guides/sql/) The Flux `sql` package provides functions for working with SQL data sources. Use `sql.from()` to query SQL databases like PostgreSQL and MySQL ```js import "sql" sql.from( driverName: "postgres", dataSourceName: "postgresql://user:password@localhost", query: "SELECT * FROM example_table" ) ``` ### [Conditional logic](/influxdb/v1/flux/guides/conditional-logic/) This guide describes how to use Flux conditional expressions, such as `if`, `else`, and `then`, to query and transform data. **Flux evaluates statements from left to right and stops evaluating once a condition matches.** ```js if color == "green" then "008000" else "ffffff" ``` ### [Regular expressions](/influxdb/v1/flux/guides/regular-expressions/) This guide walks through using regular expressions in evaluation logic in Flux functions. ```js data |> filter(fn: (r) => r.tag =~ /^foo[1-3]/) ``` ###### Input: | \_time |tag |\_value| |--------------------|----|-------| |2020-01-01T00:01:00Z|foo1| 1.0 | |2020-01-01T00:02:00Z|foo5| 1.2 | |2020-01-01T00:03:00Z|bar3| 1.8 | |2020-01-01T00:04:00Z|foo3| 0.9 | |2020-01-01T00:05:00Z|foo2| 1.4 | |2020-01-01T00:06:00Z|bar1| 2.0 | ###### Output: | \_time |tag |\_value| |--------------------|----|-------| |2020-01-01T00:01:00Z|foo1| 1.0 | |2020-01-01T00:04:00Z|foo3| 0.9 | |2020-01-01T00:05:00Z|foo2| 1.4 | ### [Geo-temporal data](/influxdb/v1/flux/guides/geo/) Use the Flux Geo package to filter geo-temporal data and group by geographic location or track. ```js import "experimental/geo" sampleGeoData |> geo.filterRows(region: {lat: 30.04, lon: 31.23, radius: 200.0}) |> geo.groupByArea(newColumn: "geoArea", level: 5) ``` ### [Optimize Flux queries](/influxdb/v1/flux/guides/optimize-queries/) Optimize your Flux queries to reduce their memory and compute (CPU) requirements. ### [Use Flux in Chronograf dashboards](/influxdb/v1/flux/guides/flux-in-dashboards/) This guide walks through using Flux queries in Chronograf dashboard cells, what template variables are available, and how to use them. | _time | host | _value | | --- | --- | --- | | _time | host | _value | | 2020-01-01T00:01:00Z | host1 | 1.0 | | 2020-01-01T00:01:00Z | host2 | 2.0 | | 2020-01-01T00:02:00Z | host1 | 1.0 | | 2020-01-01T00:02:00Z | host2 | 3.0 | | _time | host | _value | | --- | --- | --- | | _time | host | _value | | 2020-01-01T00:01:00Z | host1 | 1.0 | | 2020-01-01T00:02:00Z | host1 | 1.0 | | _time | host | _value | | --- | --- | --- | | _time | host | _value | | 2020-01-01T00:01:00Z | host2 | 2.0 | | 2020-01-01T00:02:00Z | host2 | 3.0 | | _time | host | _value | | --- | --- | --- | | _time | host | _value | | 2020-01-01T00:01:00Z | A | 1.0 | | 2020-01-01T00:02:00Z | B | 1.2 | | 2020-01-01T00:03:00Z | A | 1.8 | | 2020-01-01T00:04:00Z | B | 0.9 | | 2020-01-01T00:05:00Z | B | 1.4 | | 2020-01-01T00:06:00Z | B | 2.0 | | _time | host | _value | | --- | --- | --- | | _time | host | _value | | 2020-01-01T00:03:00Z | A | 1.8 | | 2020-01-01T00:01:00Z | A | 1.0 | | 2020-01-01T00:06:00Z | B | 2.0 | | 2020-01-01T00:05:00Z | B | 1.4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:00Z | 250 | | 2020-01-01T00:04:00Z | 160 | | 2020-01-01T00:12:00Z | 150 | | 2020-01-01T00:19:00Z | 220 | | 2020-01-01T00:32:00Z | 200 | | 2020-01-01T00:51:00Z | 290 | | 2020-01-01T01:00:00Z | 340 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:20:00Z | 195 | | 2020-01-01T00:40:00Z | 200 | | 2020-01-01T01:00:00Z | 290 | | 2020-01-01T01:20:00Z | 340 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 2 | | 2020-01-01T00:02:00Z | 4 | | 2020-01-01T00:03:00Z | 3 | | 2020-01-01T00:04:00Z | 5 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 4 | | 2020-01-01T00:02:00Z | 16 | | 2020-01-01T00:03:00Z | 9 | | 2020-01-01T00:04:00Z | 25 | | _time | _field | _value | | --- | --- | --- | | _time | _field | _value | | 2020-01-01T00:00:00Z | used | 2.5 | | 2020-01-01T00:00:10Z | used | 3.1 | | 2020-01-01T00:00:20Z | used | 4.2 | | _time | _field | _value | | --- | --- | --- | | _time | _field | _value | | 2020-01-01T00:00:00Z | total | 8.0 | | 2020-01-01T00:00:10Z | total | 8.0 | | 2020-01-01T00:00:20Z | total | 8.0 | | _time | _field | _value | | --- | --- | --- | | _time | _field | _value | | 2020-01-01T00:00:00Z | used_percent | 31.25 | | 2020-01-01T00:00:10Z | used_percent | 38.75 | | 2020-01-01T00:00:20Z | used_percent | 52.50 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1 | | 2020-01-01T00:02:00Z | 2 | | 2020-01-01T00:03:00Z | 8 | | 2020-01-01T00:04:00Z | 10 | | 2020-01-01T00:05:00Z | 0 | | 2020-01-01T00:06:00Z | 4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:02:00Z | 1 | | 2020-01-01T00:03:00Z | 7 | | 2020-01-01T00:04:00Z | 9 | | 2020-01-01T00:05:00Z | 9 | | 2020-01-01T00:06:00Z | 13 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.2 | | 2020-01-01T00:03:00Z | 1.8 | | 2020-01-01T00:04:00Z | 0.9 | | 2020-01-01T00:05:00Z | 1.4 | | 2020-01-01T00:06:00Z | 2.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:03:00Z | 1.33 | | 2020-01-01T00:04:00Z | 1.30 | | 2020-01-01T00:05:00Z | 1.36 | | 2020-01-01T00:06:00Z | 1.43 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.2 | | 2020-01-01T00:03:00Z | 1.8 | | 2020-01-01T00:04:00Z | 0.9 | | 2020-01-01T00:05:00Z | 1.4 | | 2020-01-01T00:06:00Z | 2.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:02:00Z | 1.000 | | 2020-01-01T00:04:00Z | 1.333 | | 2020-01-01T00:06:00Z | 1.325 | | 2020-01-01T00:06:00Z | 1.150 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:00Z | 250 | | 2020-01-01T00:04:00Z | 160 | | 2020-01-01T00:12:00Z | 150 | | 2020-01-01T00:19:00Z | 220 | | 2020-01-01T00:32:00Z | 200 | | 2020-01-01T00:51:00Z | 290 | | 2020-01-01T01:00:00Z | 340 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:04:00Z | | | 2020-01-01T00:12:00Z | | | 2020-01-01T00:19:00Z | 10.0 | | 2020-01-01T00:32:00Z | | | 2020-01-01T00:51:00Z | 4.74 | | 2020-01-01T01:00:00Z | 5.56 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:00Z | 250 | | 2020-01-01T00:04:00Z | 160 | | 2020-01-01T00:12:00Z | 150 | | 2020-01-01T00:19:00Z | 220 | | 2020-01-01T00:32:00Z | 200 | | 2020-01-01T00:51:00Z | 290 | | 2020-01-01T01:00:00Z | 340 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:20:00Z | | | 2020-01-01T00:40:00Z | 10.0 | | 2020-01-01T01:00:00Z | 4.74 | | 2020-01-01T01:20:00Z | 5.56 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:00Z | 250.0 | | 2020-01-01T00:01:00Z | 160.0 | | 2020-01-01T00:02:00Z | 150.0 | | 2020-01-01T00:03:00Z | 220.0 | | 2020-01-01T00:04:00Z | 200.0 | | 2020-01-01T00:05:00Z | 290.0 | | 2020-01-01T01:00:00Z | 340.0 | | le | _value | | --- | --- | | le | _value | | 100.0 | 0.0 | | 200.0 | 3.0 | | 300.0 | 6.0 | | 400.0 | 7.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | null | | 2020-01-01T00:02:00Z | 0.8 | | 2020-01-01T00:03:00Z | null | | 2020-01-01T00:04:00Z | null | | 2020-01-01T00:05:00Z | 1.4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | null | | 2020-01-01T00:02:00Z | 0.8 | | 2020-01-01T00:03:00Z | 0.8 | | 2020-01-01T00:04:00Z | 0.8 | | 2020-01-01T00:05:00Z | 1.4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | null | | 2020-01-01T00:02:00Z | 0.8 | | 2020-01-01T00:03:00Z | null | | 2020-01-01T00:04:00Z | null | | 2020-01-01T00:05:00Z | 1.4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 0.0 | | 2020-01-01T00:02:00Z | 0.8 | | 2020-01-01T00:03:00Z | 0.0 | | 2020-01-01T00:04:00Z | 0.0 | | 2020-01-01T00:05:00Z | 1.4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _value | | --- | | _value | | 1.5 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _value | | --- | | _value | | 3.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1 | | 2020-01-01T00:02:00Z | 2 | | 2020-01-01T00:03:00Z | 1 | | 2020-01-01T00:04:00Z | 3 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 5 | | 2020-01-01T00:02:00Z | 2 | | 2020-01-01T00:03:00Z | 3 | | 2020-01-01T00:04:00Z | 4 | | _time | _value_t1 | _value_t2 | | --- | --- | --- | | _time | _value_t1 | _value_t2 | | 2020-01-01T00:01:00Z | 1 | 5 | | 2020-01-01T00:02:00Z | 2 | 2 | | 2020-01-01T00:03:00Z | 1 | 3 | | 2020-01-01T00:04:00Z | 3 | 4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1 | | 2020-01-01T00:02:00Z | 2 | | 2020-01-01T00:03:00Z | 1 | | 2020-01-01T00:04:00Z | 3 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1 | | 2020-01-01T00:02:00Z | 3 | | 2020-01-01T00:03:00Z | 4 | | 2020-01-01T00:04:00Z | 7 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:04:00Z | 3.0 | | _time | tag | _value | | --- | --- | --- | | _time | tag | _value | | 2020-01-01T00:01:00Z | foo1 | 1.0 | | 2020-01-01T00:02:00Z | foo5 | 1.2 | | 2020-01-01T00:03:00Z | bar3 | 1.8 | | 2020-01-01T00:04:00Z | foo3 | 0.9 | | 2020-01-01T00:05:00Z | foo2 | 1.4 | | 2020-01-01T00:06:00Z | bar1 | 2.0 | | _time | tag | _value | | --- | --- | --- | | _time | tag | _value | | 2020-01-01T00:01:00Z | foo1 | 1.0 | | 2020-01-01T00:04:00Z | foo3 | 0.9 | | 2020-01-01T00:05:00Z | foo2 | 1.4 | --- # Calculate percentages with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/calculate-percentages/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Calculating percentages from queried data is a common use case for time series data. To calculate a percentage in Flux, operands must be in each row. Use `map()` to re-map values in the row and calculate a percentage. **To calculate percentages** 1. Use [`from()`](/flux/v0/stdlib/influxdata/influxdb/from/),[`range()`](/flux/v0/stdlib/universe/range/) and[`filter()`](/flux/v0/stdlib/universe/filter/) to query operands. 2. Use [`pivot()` or `join()`](/influxdb/v1/flux/guides/mathematic-operations/#pivot-vs-join)to align operand values into rows. 3. Use [`map()`](/flux/v0/stdlib/universe/map/)to divide the numerator operand value by the denominator operand value and multiply by 100. > [!Note] > The following examples use `pivot()` to align operands into rows because`pivot()` works in most cases and is more performant than `join()`.*See [Pivot vs join](/influxdb/v1/flux/guides/mathematic-operations/#pivot-vs-join).* ```js from(bucket: "db/rp") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "m1" and r._field =~ /field[1-2]/) |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") |> map(fn: (r) => ({r with _value: r.field1 / r.field2 * 100.0})) ``` ## GPU monitoring example The following example queries data from the gpu-monitor bucket and calculates the percentage of GPU memory used over time. Data includes the following: * **`gpu` measurement** * **`mem_used` field**: used GPU memory in bytes * **`mem_total` field**: total GPU memory in bytes ### Query mem\_used and mem\_total fields ```js from(bucket: "gpu-monitor") |> range(start: 2020-01-01T00:00:00Z) |> filter(fn: (r) => r._measurement == "gpu" and r._field =~ /mem_/) ``` ###### Returns the following stream of tables: | \_time |\_measurement| \_field | \_value | |--------------------|-------------|---------|----------| |2020-01-01T00:00:00Z| gpu |mem\_used|2517924577| |2020-01-01T00:00:10Z| gpu |mem\_used|2695091978| |2020-01-01T00:00:20Z| gpu |mem\_used|2576980377| |2020-01-01T00:00:30Z| gpu |mem\_used|3006477107| |2020-01-01T00:00:40Z| gpu |mem\_used|3543348019| |2020-01-01T00:00:50Z| gpu |mem\_used|4402341478| | \_time |\_measurement| \_field | \_value | |--------------------|-------------|----------|----------| |2020-01-01T00:00:00Z| gpu |mem\_total|8589934592| |2020-01-01T00:00:10Z| gpu |mem\_total|8589934592| |2020-01-01T00:00:20Z| gpu |mem\_total|8589934592| |2020-01-01T00:00:30Z| gpu |mem\_total|8589934592| |2020-01-01T00:00:40Z| gpu |mem\_total|8589934592| |2020-01-01T00:00:50Z| gpu |mem\_total|8589934592| ### Pivot fields into columns Use `pivot()` to pivot the `mem_used` and `mem_total` fields into columns. Output includes `mem_used` and `mem_total` columns with values for each corresponding `_time`. ```js // ... |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value") ``` ###### Returns the following: | \_time |\_measurement|mem\_used |mem\_total| |--------------------|-------------|----------|----------| |2020-01-01T00:00:00Z| gpu |2517924577|8589934592| |2020-01-01T00:00:10Z| gpu |2695091978|8589934592| |2020-01-01T00:00:20Z| gpu |2576980377|8589934592| |2020-01-01T00:00:30Z| gpu |3006477107|8589934592| |2020-01-01T00:00:40Z| gpu |3543348019|8589934592| |2020-01-01T00:00:50Z| gpu |4402341478|8589934592| ### Map new values Each row now contains the values necessary to calculate a percentage. Use `map()` to re-map values in each row. Divide `mem_used` by `mem_total` and multiply by 100 to return the percentage. > [!Note] > To return a precise float percentage value that includes decimal points, the example > below casts integer field values to floats and multiplies by a float value (`100.0`). ```js // ... |> map( fn: (r) => ({ _time: r._time, _measurement: r._measurement, _field: "mem_used_percent", _value: float(v: r.mem_used) / float(v: r.mem_total) * 100.0 }) ) ``` ##### Query results: | \_time |\_measurement| \_field |\_value| |--------------------|-------------|------------------|-------| |2020-01-01T00:00:00Z| gpu |mem\_used\_percent| 29.31 | |2020-01-01T00:00:10Z| gpu |mem\_used\_percent| 31.37 | |2020-01-01T00:00:20Z| gpu |mem\_used\_percent| 30.00 | |2020-01-01T00:00:30Z| gpu |mem\_used\_percent| 35.00 | |2020-01-01T00:00:40Z| gpu |mem\_used\_percent| 41.25 | |2020-01-01T00:00:50Z| gpu |mem\_used\_percent| 51.25 | ### Full query ```js from(bucket: "gpu-monitor") |> range(start: 2020-01-01T00:00:00Z) |> filter(fn: (r) => r._measurement == "gpu" and r._field =~ /mem_/ ) |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value") |> map( fn: (r) => ({ _time: r._time, _measurement: r._measurement, _field: "mem_used_percent", _value: float(v: r.mem_used) / float(v: r.mem_total) * 100.0 }) ) ``` ## Examples #### Calculate percentages using multiple fields ```js from(bucket: "db/rp") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "example-measurement") |> filter(fn: (r) => r._field == "used_system" or r._field == "used_user" or r._field == "total") |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") |> map(fn: (r) => ({r with _value: float(v: r.used_system + r.used_user) / float(v: r.total) * 100.0})) ``` #### Calculate percentages using multiple measurements 1. Ensure measurements are in the same [bucket](/influxdb/v1/flux/get-started/#buckets). 2. Use `filter()` to include data from both measurements. 3. Use `group()` to ungroup data and return a single table. 4. Use `pivot()` to pivot fields into columns. 5. Use `map()` to re-map rows and perform the percentage calculation. ```js from(bucket: "db/rp") |> range(start: -1h) |> filter(fn: (r) => (r._measurement == "m1" or r._measurement == "m2") and (r._field == "field1" or r._field == "field2")) |> group() |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") |> map(fn: (r) => ({r with _value: r.field1 / r.field2 * 100.0})) ``` #### Calculate percentages using multiple data sources ```js import "sql" import "influxdata/influxdb/secrets" pgUser = secrets.get(key: "POSTGRES_USER") pgPass = secrets.get(key: "POSTGRES_PASSWORD") pgHost = secrets.get(key: "POSTGRES_HOST") t1 = sql.from( driverName: "postgres", dataSourceName: "postgresql://${pgUser}:${pgPass}@${pgHost}", query: "SELECT id, name, available FROM exampleTable", ) t2 = from(bucket: "db/rp") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "example-measurement" and r._field == "example-field") join(tables: {t1: t1, t2: t2}, on: ["id"]) |> map(fn: (r) => ({r with _value: r._value_t2 / r.available_t1 * 100.0})) ``` | _time | _measurement | _field | _value | | --- | --- | --- | --- | | _time | _measurement | _field | _value | | 2020-01-01T00:00:00Z | gpu | mem_used | 2517924577 | | 2020-01-01T00:00:10Z | gpu | mem_used | 2695091978 | | 2020-01-01T00:00:20Z | gpu | mem_used | 2576980377 | | 2020-01-01T00:00:30Z | gpu | mem_used | 3006477107 | | 2020-01-01T00:00:40Z | gpu | mem_used | 3543348019 | | 2020-01-01T00:00:50Z | gpu | mem_used | 4402341478 | | _time | _measurement | _field | _value | | --- | --- | --- | --- | | _time | _measurement | _field | _value | | 2020-01-01T00:00:00Z | gpu | mem_total | 8589934592 | | 2020-01-01T00:00:10Z | gpu | mem_total | 8589934592 | | 2020-01-01T00:00:20Z | gpu | mem_total | 8589934592 | | 2020-01-01T00:00:30Z | gpu | mem_total | 8589934592 | | 2020-01-01T00:00:40Z | gpu | mem_total | 8589934592 | | 2020-01-01T00:00:50Z | gpu | mem_total | 8589934592 | | _time | _measurement | mem_used | mem_total | | --- | --- | --- | --- | | _time | _measurement | mem_used | mem_total | | 2020-01-01T00:00:00Z | gpu | 2517924577 | 8589934592 | | 2020-01-01T00:00:10Z | gpu | 2695091978 | 8589934592 | | 2020-01-01T00:00:20Z | gpu | 2576980377 | 8589934592 | | 2020-01-01T00:00:30Z | gpu | 3006477107 | 8589934592 | | 2020-01-01T00:00:40Z | gpu | 3543348019 | 8589934592 | | 2020-01-01T00:00:50Z | gpu | 4402341478 | 8589934592 | | _time | _measurement | _field | _value | | --- | --- | --- | --- | | _time | _measurement | _field | _value | | 2020-01-01T00:00:00Z | gpu | mem_used_percent | 29.31 | | 2020-01-01T00:00:10Z | gpu | mem_used_percent | 31.37 | | 2020-01-01T00:00:20Z | gpu | mem_used_percent | 30.00 | | 2020-01-01T00:00:30Z | gpu | mem_used_percent | 35.00 | | 2020-01-01T00:00:40Z | gpu | mem_used_percent | 41.25 | | 2020-01-01T00:00:50Z | gpu | mem_used_percent | 51.25 | --- # Query using conditional logic Source: https://docs.influxdata.com/influxdb/v1/flux/guides/conditional-logic/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Flux provides `if`, `then`, and `else` conditional expressions that allow for powerful and flexible Flux queries. ##### Conditional expression syntax ```js // Pattern if then else // Example if color == "green" then "008000" else "ffffff" ``` Conditional expressions are most useful in the following contexts: * When defining variables. * When using functions that operate on a single row at a time ([`filter()`](/flux/v0/stdlib/universe/filter/),[`map()`](/flux/v0/stdlib/universe/map/),[`reduce()`](/flux/v0/stdlib/universe/reduce) ). ## Evaluating conditional expressions Flux evaluates statements in order and stops evaluating once a condition matches. For example, given the following statement: ```js if r._value > 95.0000001 and r._value <= 100.0 then "critical" else if r._value > 85.0000001 and r._value <= 95.0 then "warning" else if r._value > 70.0000001 and r._value <= 85.0 then "high" else "normal" ``` When `r._value` is 96, the output is “critical” and the remaining conditions are not evaluated. ## Examples * [Conditionally set the value of a variable](#conditionally-set-the-value-of-a-variable) * [Create conditional filters](#create-conditional-filters) * [Conditionally transform column values with map()](#conditionally-transform-column-values-with-map) * [Conditionally increment a count with reduce()](#conditionally-increment-a-count-with-reduce) ### Conditionally set the value of a variable The following example sets the `overdue` variable based on the`dueDate` variable’s relation to `now()`. ```js dueDate = 2019-05-01T00:00:00Z overdue = if dueDate < now() then true else false ``` ### Create conditional filters The following example uses an example `metric` variable to change how the query filters data.`metric` has three possible values: * Memory * CPU * Disk ```js metric = "Memory" from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter( fn: (r) => if v.metric == "Memory" then r._measurement == "mem" and r._field == "used_percent" else if v.metric == "CPU" then r._measurement == "cpu" and r._field == "usage_user" else if v.metric == "Disk" then r._measurement == "disk" and r._field == "used_percent" else r._measurement != "", ) ``` ### Conditionally transform column values with map() The following example uses the [`map()` function](/flux/v0/stdlib/universe/map/)to conditionally transform column values. It sets the `level` column to a specific string based on `_value` column. [No Comments](#no-comments)[Comments](#comments) ```js from(bucket: "telegraf/autogen") |> range(start: -5m) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") |> map( fn: (r) => ({r with level: if r._value >= 95.0000001 and r._value <= 100.0 then "critical" else if r._value >= 85.0000001 and r._value <= 95.0 then "warning" else if r._value >= 70.0000001 and r._value <= 85.0 then "high" else "normal", }), ) ``` ```js from(bucket: "telegraf/autogen") |> range(start: -5m) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") |> map( fn: (r) => ({ // Retain all existing columns in the mapped row r with // Set the level column value based on the _value column level: if r._value >= 95.0000001 and r._value <= 100.0 then "critical" else if r._value >= 85.0000001 and r._value <= 95.0 then "warning" else if r._value >= 70.0000001 and r._value <= 85.0 then "high" else "normal", }), ) ``` ### Conditionally increment a count with reduce() The following example uses the [`aggregateWindow()`](/flux/v0/stdlib/universe/aggregatewindow/)and [`reduce()`](/flux/v0/stdlib/universe/reduce/)functions to count the number of records in every five minute window that exceed a defined threshold. [No Comments](#no-comments-2)[Comments](#comments-2) ```js threshold = 65.0 from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") |> aggregateWindow( every: 5m, fn: (column, tables=<-) => tables |> reduce( identity: {above_threshold_count: 0.0}, fn: (r, accumulator) => ({ above_threshold_count: if r._value >= threshold then accumulator.above_threshold_count + 1.0 else accumulator.above_threshold_count + 0.0, }), ), ) ``` ```js threshold = 65.0 from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") // Aggregate data into 5 minute windows using a custom reduce() function |> aggregateWindow( every: 5m, // Use a custom function in the fn parameter. // The aggregateWindow fn parameter requires 'column' and 'tables' parameters. fn: (column, tables=<-) => tables |> reduce( identity: {above_threshold_count: 0.0}, fn: (r, accumulator) => ({ // Conditionally increment above_threshold_count if // r.value exceeds the threshold above_threshold_count: if r._value >= threshold then accumulator.above_threshold_count + 1.0 else accumulator.above_threshold_count + 0.0, }), ), ) ``` --- # Query cumulative sum Source: https://docs.influxdata.com/influxdb/v1/flux/guides/cumulativesum/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`cumulativeSum()` function](/flux/v0/stdlib/universe/cumulativesum/)to calculate a running total of values.`cumulativeSum` sums the values of subsequent records and returns each row updated with the summed total. **Given the following input table:** |\_time|\_value| |------|-------| | 0001 | 1 | | 0002 | 2 | | 0003 | 1 | | 0004 | 3 | **`cumulativeSum()` returns:** |\_time|\_value| |------|-------| | 0001 | 1 | | 0002 | 3 | | 0003 | 4 | | 0004 | 7 | > [!Note] > The examples below use the [example data variable](/influxdb/v1/flux/guides/#example-data-variable). ##### Calculate the running total of values ```js data |> cumulativeSum() ``` ## Use cumulativeSum() with aggregateWindow() [`aggregateWindow()`](/flux/v0/stdlib/universe/aggregatewindow/)segments data into windows of time, aggregates data in each window into a single point, then removes the time-based segmentation. It is primarily used to downsample data. `aggregateWindow()` expects an aggregate function that returns a single row for each time window. To use `cumulativeSum()` with `aggregateWindow`, use `sum` in `aggregateWindow()`, then calculate the running total of the aggregate values with `cumulativeSum()`. ```js data |> aggregateWindow(every: 5m, fn: sum) |> cumulativeSum() ``` | _time | _value | | --- | --- | | _time | _value | | 0001 | 1 | | 0002 | 2 | | 0003 | 1 | | 0004 | 3 | | _time | _value | | --- | --- | | _time | _value | | 0001 | 1 | | 0002 | 3 | | 0003 | 4 | | 0004 | 7 | --- # Execute Flux queries Source: https://docs.influxdata.com/influxdb/v1/flux/guides/execute-queries/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. There are multiple ways to execute Flux queries with InfluxDB and Chronograf v1.8+. This guide covers the different options: 1. [Chronograf’s Data Explorer](#chronograf-s-data-explorer) 2. [Influx CLI](#influx-cli) 3. [InfluxDB API](#influxdb-api) > > > Before attempting these methods, make sure Flux is enabled by setting`flux-enabled = true` in the `[http]` section of your InfluxDB configuration file. > > ## Chronograf’s Data Explorer Chronograf v1.8+ supports Flux in its Data Explorer. Flux queries can be built, executed, and visualized from within the Chronograf user interface. ## Influx CLI To start an interactive Flux read-eval-print-loop (REPL) with the InfluxDB 1.8+`influx` CLI, run the `influx` command with the following flags: * `-type=flux` * `-path-prefix=/api/v2/query` > [!Note] > If [authentication is enabled](/influxdb/v1/administration/authentication_and_authorization)on your InfluxDB instance, use the `-username` flag to provide your InfluxDB username and > the `-password` flag to provide your password. ##### Enter an interactive Flux REPL [No Auth](#no-auth)[Auth Enabled](#auth-enabled) ```bash influx -type=flux -path-prefix=/api/v2/query ``` ```bash influx -type=flux \ -path-prefix=/api/v2/query \ -username myuser \ -password PasSw0rd ``` Any Flux query can be executed within the REPL. ### Submit a Flux query via parameter Flux queries can also be passed to the Flux REPL as a parameter using the `influx` CLI’s `-type=flux` option and the `-execute` parameter. The accompanying string is executed as a Flux query and results are output in your terminal. [No Auth](#no-auth-2)[Auth Enabled](#auth-enabled-2) ```bash influx -type=flux \ -path-prefix=/api/v2/query \ -execute '' ``` ```bash influx -type=flux \ -path-prefix=/api/v2/query \ -username myuser \ -password PasSw0rd \ -execute '' ``` ### Submit a Flux query via via STDIN Flux queries an be piped into the `influx` CLI via STDIN. Query results are otuput in your terminal. [No Auth](#no-auth-3)[Auth Enabled](#auth-enabled-3) ```bash echo '' | influx -type=flux -path-prefix=/api/v2/query ``` ```bash echo '' | influx -type=flux \ -path-prefix=/api/v2/query \ -username myuser \ -password PasSw0rd ``` ## InfluxDB API Flux can be used to query InfluxDB through InfluxDB’s `/api/v2/query` endpoint. Queried data is returned in annotated CSV format. In your request, set the following: * `Accept` header to `application/csv` * `Content-type` header to `application/vnd.flux` * If [authentication is enabled](/influxdb/v1/administration/authentication_and_authorization)on your InfluxDB instance, `Authorization` header to `Token :` This allows you to POST the Flux query in plain text and receive the annotated CSV response. Below is an example `curl` command that queries InfluxDB using Flux: [No Auth](#no-auth-4)[Auth Enabled](#auth-enabled-4) ```bash curl -XPOST localhost:8086/api/v2/query -sS \ -H 'Accept:application/csv' \ -H 'Content-type:application/vnd.flux' \ -d 'from(bucket:"telegraf") |> range(start:-5m) |> filter(fn:(r) => r._measurement == "cpu")' ``` ```bash curl -XPOST localhost:8086/api/v2/query -sS \ -H 'Accept:application/csv' \ -H 'Content-type:application/vnd.flux' \ -H 'Authorization: Token :' \ -d 'from(bucket:"telegraf") |> range(start:-5m) |> filter(fn:(r) => r._measurement == "cpu")' ``` --- # Check if a value exists Source: https://docs.influxdata.com/influxdb/v1/flux/guides/exists/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the `exists` operator to check if a row record contains a column or if a column’s value is *null*. ```js (r) => exists r.column ``` If you’re just getting started with Flux queries, check out the following: * [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query. * [Execute queries](/influxdb/v1/flux/guides/execute-queries/) to discover a variety of ways to run your queries. Use `exists` with row functions ([`filter()`](/flux/v0/stdlib/universe/filter/),[`map()`](/flux/v0/stdlib/universe/map/),[`reduce()`](/flux/v0/stdlib/universe/reduce/)) to check if a row includes a column or if the value for that column is *null*. #### Filter null values ```js from(bucket: "db/rp") |> range(start: -5m) |> filter(fn: (r) => exists r._value) ``` #### Map values based on existence ```js from(bucket: "db/rp") |> range(start: -30s) |> map( fn: (r) => ({r with human_readable: if exists r._value then "${r._field} is ${string(v: r._value)}." else "${r._field} has no value.", }), ) ``` #### Ignore null values in a custom aggregate function ```js customSumProduct = (tables=<-) => tables |> reduce( identity: {sum: 0.0, product: 1.0}, fn: (r, accumulator) => ({r with sum: if exists r._value then r._value + accumulator.sum else accumulator.sum, product: if exists r._value then r.value * accumulator.product else accumulator.product, }), ) ``` #### Check if a statically defined record contains a key When you use the [record literal syntax](/flux/v0/data-types/composite/record/#record-syntax)to statically define a record, Flux knows the record type and what keys to expect. * If the key exists in the static record, `exists` returns `true`. * If the key does not exist in the static record, because the record type is statically known, `exists` returns an error. ``` p = { firstName: "John", lastName: "Doe", age: 42, } exists p.firstName // Returns true exists p.height // Returns "error: record is missing label height" ``` > [!Note] > Static record types are enforced by the Flux version bundled with InfluxDB. > InfluxDB v1.11+ returns the error above. > InfluxDB v1.8 bundles Flux v0.65, which accepts `exists` on a key that the > record literal does not define. --- # Fill null values in data Source: https://docs.influxdata.com/influxdb/v1/flux/guides/fill/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`fill()` function](/flux/v0/stdlib/universe/fill/)to replace *null* values with: * [the previous non-null value](#fill-with-the-previous-value) * [a specified value](#fill-with-a-specified-value) ```js data |> fill(usePrevious: true) // OR data |> fill(value: 0.0) ``` > [!Note] > #### Fill empty windows of time > > The `fill()` function **does not** fill empty windows of time. > It only replaces *null* values in existing data. > Filling empty windows of time requires time interpolation*(see [influxdata/flux#2428](https://github.com/influxdata/flux/issues/2428))*. ## Fill with the previous value To fill *null* values with the previous **non-null** value, set the `usePrevious` parameter to `true`. > [!Note] > Values remain *null* if there is no previous non-null value in the table. ```js data |> fill(usePrevious: true) ``` **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| null | |2020-01-01T00:02:00Z| 0.8 | |2020-01-01T00:03:00Z| null | |2020-01-01T00:04:00Z| null | |2020-01-01T00:05:00Z| 1.4 | **`fill(usePrevious: true)` returns:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| null | |2020-01-01T00:02:00Z| 0.8 | |2020-01-01T00:03:00Z| 0.8 | |2020-01-01T00:04:00Z| 0.8 | |2020-01-01T00:05:00Z| 1.4 | ## Fill with a specified value To fill *null* values with a specified value, use the `value` parameter to specify the fill value.*The fill value must match the [data type](/flux/v0/spec/types/#basic-types)of the [column](/flux/v0/stdlib/universe/fill/#column).* ```js data |> fill(value: 0.0) ``` **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| null | |2020-01-01T00:02:00Z| 0.8 | |2020-01-01T00:03:00Z| null | |2020-01-01T00:04:00Z| null | |2020-01-01T00:05:00Z| 1.4 | **`fill(value: 0.0)` returns:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 0.0 | |2020-01-01T00:02:00Z| 0.8 | |2020-01-01T00:03:00Z| 0.0 | |2020-01-01T00:04:00Z| 0.0 | |2020-01-01T00:05:00Z| 1.4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | null | | 2020-01-01T00:02:00Z | 0.8 | | 2020-01-01T00:03:00Z | null | | 2020-01-01T00:04:00Z | null | | 2020-01-01T00:05:00Z | 1.4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | null | | 2020-01-01T00:02:00Z | 0.8 | | 2020-01-01T00:03:00Z | 0.8 | | 2020-01-01T00:04:00Z | 0.8 | | 2020-01-01T00:05:00Z | 1.4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | null | | 2020-01-01T00:02:00Z | 0.8 | | 2020-01-01T00:03:00Z | null | | 2020-01-01T00:04:00Z | null | | 2020-01-01T00:05:00Z | 1.4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 0.0 | | 2020-01-01T00:02:00Z | 0.8 | | 2020-01-01T00:03:00Z | 0.0 | | 2020-01-01T00:04:00Z | 0.0 | | 2020-01-01T00:05:00Z | 1.4 | --- # Query first and last values Source: https://docs.influxdata.com/influxdb/v1/flux/guides/first-last/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`first()`](/flux/v0/stdlib/universe/first/) or[`last()`](/flux/v0/stdlib/universe/last/) functions to return the first or last record in an input table. ```js data |> first() // OR data |> last() ``` > [!Note] > By default, InfluxDB returns results sorted by time, however you can use the[`sort()` function](/flux/v0/stdlib/universe/sort/)to change how results are sorted.`first()` and `last()` respect the sort order of input data and return records > based on the order they are received in. ### first `first()` returns the first non-null record in an input table. **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | **The following function returns:** ```js |> first() ``` | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | ### last `last()` returns the last non-null record in an input table. **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | **The following function returns:** ```js |> last() ``` | \_time |\_value| |--------------------|-------| |2020-01-01T00:04:00Z| 3.0 | ## Use first() or last() with aggregateWindow() Use `first()` and `last()` with [`aggregateWindow()`](/flux/v0/stdlib/universe/aggregatewindow/)to select the first or last records in time-based groups.`aggregateWindow()` segments data into windows of time, aggregates data in each window into a single point using aggregate or selector functions, and then removes the time-based segmentation. **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:00Z| 10 | |2020-01-01T00:00:15Z| 12 | |2020-01-01T00:00:45Z| 9 | |2020-01-01T00:01:05Z| 9 | |2020-01-01T00:01:10Z| 15 | |2020-01-01T00:02:30Z| 11 | **The following function returns:** [first](#first-2)[last](#last-2) ```js |> aggregateWindow( every: 1h, fn: first, ) ``` | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:59Z| 10 | |2020-01-01T00:01:59Z| 9 | |2020-01-01T00:02:59Z| 11 | ```js |> aggregateWindow( every: 1h, fn: last, ) ``` | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:59Z| 9 | |2020-01-01T00:01:59Z| 15 | |2020-01-01T00:02:59Z| 11 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:04:00Z | 3.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:00Z | 10 | | 2020-01-01T00:00:15Z | 12 | | 2020-01-01T00:00:45Z | 9 | | 2020-01-01T00:01:05Z | 9 | | 2020-01-01T00:01:10Z | 15 | | 2020-01-01T00:02:30Z | 11 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:59Z | 10 | | 2020-01-01T00:01:59Z | 9 | | 2020-01-01T00:02:59Z | 11 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:59Z | 9 | | 2020-01-01T00:01:59Z | 15 | | 2020-01-01T00:02:59Z | 11 | --- # Use Flux in Chronograf dashboards Source: https://docs.influxdata.com/influxdb/v1/flux/guides/flux-in-dashboards/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. [Chronograf](/chronograf/v1/) is the web user interface for managing for the InfluxData platform that lest you create and customize dashboards that visualize your data. Visualized data is retrieved using either an InfluxQL or Flux query. This guide walks through using Flux queries in Chronograf dashboard cells. ## Using Flux in dashboard cells ***Chronograf v1.8+** and **InfluxDB v1.8+ with [Flux enabled](/influxdb/v1/flux/installation)**are required to use Flux in dashboards.* To use Flux in a dashboard cell, either create a new cell or edit an existing cell by clicking the **pencil** icon in the top right corner of the cell. To the right of the **Source dropdown** above the graph preview, select **Flux** as the source type. Flux in Chronograf dashboard cells > > > The Flux source type is only available if your data source has[Flux enabled](/influxdb/v1/flux/installation). > > This will provide **Schema**, **Script**, and **Functions** panes. ### Schema pane The Schema pane allows you to explore your data and add filters for specific measurements, fields, and tags to your Flux script. ### Script pane The Script pane is where you write your Flux script. In its default state, the **Script** pane includes an optional [Script Wizard](/chronograf/v1/guides/querying-data/#explore-data-with-flux)that uses selected options to build a Flux query for you. The generated query includes all the relevant functions and [template variables](#template-variables-in-flux)required to return your desired data. ### Functions pane The Functions pane provides a list of functions available in your Flux queries. Clicking on a function will add it to the end of the script in the Script pane. Hovering over a function provides documentation for the function as well as links to deep documentation. ### Dynamic sources Chronograf can be configured with multiple data sources. The **Sources dropdown** allows you to select a specific data source to connect to, but a **Dynamic Source** options is also available. With a dynamic source, the cell will query data from whatever data source to which Chronograf is currently connected. Connections are managed under Chronograf’s **Configuration** tab. ### View raw data As you’re building your Flux scripts, each function processes or transforms your data is ways specific to the function. It can be helpful to view the actual data in order to see how it is being shaped. The **View Raw Data** toggle above the data visualization switches between graphed data and raw data shown in table form. View raw data *The **View Raw Data** toggle is only available when using Flux.* ## Template variables in Flux Chronograf [template variables](/chronograf/v1/guides/dashboard-template-variables/)allow you to alter specific components of cells’ queries using elements provided in the Chronograf user interface. In your Flux query, reference template variables just as you would reference defined Flux variables. The following example uses Chronograf’s [predefined template variables](#predefined-template-variables),`dashboardTime`, `upperDashboardTime`, and `autoInterval`: ```js from(bucket: "telegraf/autogen") |> filter(fn: (r) => r._measurement == "cpu") |> range(start: dashboardTime, stop: upperDashboardTime) |> window(every: autoInterval) ``` ### Predefined template variables #### dashboardTime The `dashboardTime` template variable represents the lower time bound of ranged data. It’s value is controlled by the time dropdown in your dashboard. It should be used to define the `start` parameter of the `range()` function. ```js dataSet |> range(start: dashboardTime) ``` #### upperDashboardTime The `upperDashboardTime` template variable represents the upper time bound of ranged data. It’s value is modified by the time dropdown in your dashboard when using an absolute time range. It should be used to define the `stop` parameter of the `range()` function. ```js dataSet |> range(start: dashboardTime, stop: upperDashboardTime) ``` > > > As a best practice, always set the `stop` parameter of the `range()` function to `upperDashboardTime` in cell queries. > Without it, `stop` defaults to “now” and the absolute upper range bound selected in the time dropdown is not honored, > potentially causing unnecessary load on InfluxDB. > > #### autoInterval The `autoInterval` template variable represents the refresh interval of the dashboard and is controlled by the refresh interval dropdown. It’s typically used to align window intervals created in[windowing and aggregation](/influxdb/v1/flux/guides/window-aggregate) operations with dashboard refreshes. ```js dataSet |> range(start: dashboardTime, stop: upperDashboardTime) |> aggregateWindow(every: autoInterval, fn: mean) ``` ### Custom template variables \<% warn %\> Chronograf does not support the use of custom template variables in Flux queries. \<% /warn %\> ## Using Flux and InfluxQL Within individual dashboard cells, the use of Flux and InfluxQL is mutually exclusive. However, a dashboard may consist of different cells, each using Flux or InfluxQL. --- # Work with geo-temporal data Source: https://docs.influxdata.com/influxdb/v1/flux/guides/geo/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [Flux Geo package](/flux/v0/stdlib/experimental/geo) to filter geo-temporal data and group by geographic location or track. The Geo package is experimental and subject to change at any time. By using it, you agree to the [risks of experimental functions](/flux/v0/stdlib/experimental/#experimental-packages-are-subject-to-change). **To work with geo-temporal data:** 1. Import the `experimental/geo` package. ``` import "experimental/geo" ``` 2. Load geo-temporal data. *See below for [sample geo-temporal data](#sample-data).* 3. Do one or more of the following: * [Shape data to work with the Geo package](#shape-data-to-work-with-the-geo-package) * [Filter data by region](#filter-geo-temporal-data-by-region) (using strict or non-strict filters) * [Group data by area or by track](#group-geo-temporal-data) ### [Shape data to work with the Geo package](/influxdb/v1/flux/guides/geo/shape-geo-data/) Functions in the Flux Geo package require **lat** and **lon** fields and an **s2\_cell\_id** tag. Rename latitude and longitude fields and generate S2 cell ID tokens. ```js import "experimental/geo" sampleGeoData |> geo.shapeData(latField: "latitude", lonField: "longitude", level: 10) ``` ### [Filter geo-temporal data by region](/influxdb/v1/flux/guides/geo/filter-by-region/) Use the `geo.filterRows` function to filter geo-temporal data by box-shaped, circular, or polygonal geographic regions. ```js import "experimental/geo" sampleGeoData |> geo.filterRows( region: {lat: 30.04, lon: 31.23, radius: 200.0}, strict: true ) ``` ### [Group geo-temporal data](/influxdb/v1/flux/guides/geo/group-geo-data/) Use the `geo.groupByArea()` to group geo-temporal data by area and `geo.asTracks()` to group data into tracks or routes. ```js import "experimental/geo" sampleGeoData |> geo.groupByArea(newColumn: "geoArea", level: 5) |> geo.asTracks(groupBy: ["id"],orderBy: ["_time"]) ``` ## Sample data Many of the examples in this section use a `sampleGeoData` variable that represents a sample set of geo-temporal data. The [Bird Migration Sample Data](https://github.com/influxdata/influxdb2-sample-data/tree/master/bird-migration-data)available on GitHub provides sample geo-temporal data that meets the[requirements of the Flux Geo package](/flux/v0/stdlib/experimental/geo/#geo-schema-requirements). ### Load annotated CSV sample data Use the [experimental `csv.from()` function](/flux/v0/stdlib/experimental/csv/from/)to load the sample bird migration annotated CSV data from GitHub: ```js import `experimental/csv` sampleGeoData = csv.from( url: "https://github.com/influxdata/influxdb2-sample-data/blob/master/bird-migration-data/bird-migration.csv" ) ``` > [!Note] > `csv.from(url: ...)` downloads sample data each time you execute the query **(\~1.3 MB)**. > If bandwidth is a concern, use the [`to()` function](/flux/v0/stdlib/influxdata/influxdb/to/)to write the data to a bucket, and then query the bucket with [`from()`](/flux/v0/stdlib/influxdata/influxdb/from/). ### Write sample data to InfluxDB with line protocol Use `curl` and the `influx write` command to write bird migration line protocol to InfluxDB. Replace `db/rp` with your destination bucket: ```sh curl https://raw.githubusercontent.com/influxdata/influxdb2-sample-data/master/bird-migration-data/bird-migration.line --output ./tmp-data influx write -b db/rp @./tmp-data rm -f ./tmp-data ``` Use Flux to query the bird migration data and assign it to the `sampleGeoData` variable: ```js sampleGeoData = from(bucket: "db/rp") |> range(start: 2019-01-01T00:00:00Z, stop: 2019-12-31T23:59:59Z) |> filter(fn: (r) => r._measurement == "migration") ``` --- # Filter geo-temporal data by region Source: https://docs.influxdata.com/influxdb/v1/flux/guides/geo/filter-by-region/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`geo.filterRows` function](/flux/v0/stdlib/experimental/geo/filterrows/)to filter geo-temporal data by geographic region: 1. [Define a geographic region](#define-a-geographic-region) 2. [Use strict or non-strict filtering](#strict-and-non-strict-filtering) The following example uses the [sample bird migration data](/influxdb/v1/flux/guides/geo/#sample-data)and queries data points **within 200km of Cairo, Egypt**: ```js import "experimental/geo" sampleGeoData |> geo.filterRows(region: {lat: 30.04, lon: 31.23, radius: 200.0}, strict: true) ``` ## Define a geographic region Many functions in the Geo package filter data based on geographic region. Define a geographic region using one of the following shapes: * [box](#box) * [circle](#circle) * [polygon](#polygon) ### box Define a box-shaped region by specifying a record containing the following properties: * **minLat:** minimum latitude in decimal degrees (WGS 84) *(Float)* * **maxLat:** maximum latitude in decimal degrees (WGS 84) *(Float)* * **minLon:** minimum longitude in decimal degrees (WGS 84) *(Float)* * **maxLon:** maximum longitude in decimal degrees (WGS 84) *(Float)* ##### Example box-shaped region ```js { minLat: 40.51757813, maxLat: 40.86914063, minLon: -73.65234375, maxLon: -72.94921875, } ``` ### circle Define a circular region by specifying a record containing the following properties: * **lat**: latitude of the circle center in decimal degrees (WGS 84) *(Float)* * **lon**: longitude of the circle center in decimal degrees (WGS 84) *(Float)* * **radius**: radius of the circle in kilometers (km) *(Float)* ##### Example circular region ```js { lat: 40.69335938, lon: -73.30078125, radius: 20.0, } ``` ### polygon Define a polygonal region with a record containing the latitude and longitude for each point in the polygon: * **points**: points that define the custom polygon *(Array of records)* Define each point with a record containing the following properties: ``` - **lat**: latitude in decimal degrees (WGS 84) _(Float)_ - **lon**: longitude in decimal degrees (WGS 84) _(Float)_ ``` ##### Example polygonal region ```js { points: [ {lat: 40.671659, lon: -73.936631}, {lat: 40.706543, lon: -73.749177}, {lat: 40.791333, lon: -73.880327}, ] } ``` ## Strict and non-strict filtering In most cases, the specified geographic region does not perfectly align with S2 grid cells. * **Non-strict filtering** returns points that may be outside of the specified region but inside S2 grid cells partially covered by the region. * **Strict filtering** returns only points inside the specified region. *Strict filtering is less performant, but more accurate than non-strict filtering.* S2 grid cell Filter region Returned point **Strict filtering** **Non-strict filtering** #### Related * [geo package](/flux/v0/stdlib/experimental/geo/) * [geo.filterRows() function](/flux/v0/stdlib/experimental/geo/filterrows/) --- # Group geo-temporal data Source: https://docs.influxdata.com/influxdb/v1/flux/guides/geo/group-geo-data/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the `geo.groupByArea()` to group geo-temporal data by area and `geo.asTracks()`to group data into tracks or routes. * [Group data by area](#group-data-by-area) * [Group data into tracks or routes](#group-data-by-track-or-route) ### Group data by area Use the [`geo.groupByArea()` function](/flux/v0/stdlib/experimental/geo/groupbyarea/)to group geo-temporal data points by geographic area. Areas are determined by [S2 grid cells](https://s2geometry.io/devguide/s2cell_hierarchy.html#s2cellid-numbering) * Specify a new column to store the unique area identifier for each point with the `newColumn` parameter. * Specify the [S2 cell level](https://s2geometry.io/resources/s2cell_statistics)to use when calculating geographic areas with the `level` parameter. The following example uses the [sample bird migration data](/influxdb/v1/flux/guides/geo/#sample-data)to query data points within 200km of Cairo, Egypt and group them by geographic area: ```js import "experimental/geo" sampleGeoData |> geo.filterRows(region: {lat: 30.04, lon: 31.23, radius: 200.0}) |> geo.groupByArea(newColumn: "geoArea", level: 5) ``` ### Group data by track or route Use [`geo.asTracks()` function](/flux/v0/stdlib/experimental/geo/astracks/)to group data points into tracks or routes and order them by time or other columns. Data must contain a unique identifier for each track. For example: `id` or `tid`. * Specify columns that uniquely identify each track or route with the `groupBy` parameter. * Specify which columns to sort by with the `orderBy` parameter. Default is `["_time"]`. The following example uses the [sample bird migration data](/influxdb/v1/flux/guides/geo/#sample-data)to query data points within 200km of Cairo, Egypt and group them into routes unique to each bird: ```js import "experimental/geo" sampleGeoData |> geo.filterRows(region: {lat: 30.04, lon: 31.23, radius: 200.0}) |> geo.asTracks(groupBy: ["id"], orderBy: ["_time"]) ``` #### Related * [geo package](/flux/v0/stdlib/experimental/geo/) * [geo.groupByArea() function](/flux/v0/stdlib/experimental/geo/groupbyarea/) * [geo.asTracks() function](/flux/v0/stdlib/experimental/geo/astracks/) --- # Shape data to work with the Geo package Source: https://docs.influxdata.com/influxdb/v1/flux/guides/geo/shape-geo-data/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Functions in the Geo package require the following data schema: * an **s2\_cell\_id** tag containing the [S2 Cell ID](https://s2geometry.io/devguide/s2cell_hierarchy.html#s2cellid-numbering)**as a token** * a **`lat` field** containing the **latitude in decimal degrees** (WGS 84) * a **`lon` field** containing the **longitude in decimal degrees** (WGS 84) ## Shape geo-temporal data If your data already contains latitude and longitude fields, use the[`geo.shapeData()`function](/flux/v0/stdlib/experimental/geo/shapedata/)to rename the fields to match the requirements of the Geo package, pivot the data into row-wise sets, and generate S2 cell ID tokens for each point. ```js import "experimental/geo" from(bucket: "example-bucket") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "example-measurement") |> geo.shapeData(latField: "latitude", lonField: "longitude", level: 10) ``` ## Generate S2 cell ID tokens The Geo package uses the [S2 Geometry Library](https://s2geometry.io/) to represent geographic coordinates on a three-dimensional sphere. The sphere is divided into [cells](https://s2geometry.io/devguide/s2cell_hierarchy), each with a unique 64-bit identifier (S2 cell ID). Grid and S2 cell ID accuracy are defined by a [level](https://s2geometry.io/resources/s2cell_statistics). > [!Note] > To filter more quickly, use higher S2 Cell ID levels, > but know that higher levels increase [series cardinality](/influxdb/v1/concepts/glossary/#series-cardinality). The Geo package requires S2 cell IDs as tokens. To generate add S2 cell IDs tokens to your data, use one of the following options: * [Generate S2 cell ID tokens with Telegraf](#generate-s2-cell-id-tokens-with-telegraf) * [Generate S2 cell ID tokens language-specific libraries](#generate-s2-cell-id-tokens-language-specific-libraries) * [Generate S2 cell ID tokens with Flux](#generate-s2-cell-id-tokens-with-flux) ### Generate S2 cell ID tokens with Telegraf Enable the [Telegraf S2 Geo (`s2geo`) processor](https://github.com/influxdata/telegraf/tree/master/plugins/processors/s2geo)to generate S2 cell ID tokens at a specified `cell_level` using `lat` and `lon` field values. Add the `processors.s2geo` configuration to your Telegraf configuration file (`telegraf.conf`): ```toml [[processors.s2geo]] ## The name of the lat and lon fields containing WGS-84 latitude and ## longitude in decimal degrees. lat_field = "lat" lon_field = "lon" ## New tag to create tag_key = "s2_cell_id" ## Cell level (see https://s2geometry.io/resources/s2cell_statistics.html) cell_level = 9 ``` Telegraf stores the S2 cell ID token in the `s2_cell_id` tag. ### Generate S2 cell ID tokens language-specific libraries Many programming languages offer S2 Libraries with methods for generating S2 cell ID tokens. Use latitude and longitude with the `s2.CellID.ToToken` endpoint of the S2 Geometry Library to generate `s2_cell_id` tags. For example: * **Go:** [s2.CellID.ToToken()](https://godoc.org/github.com/golang/geo/s2#CellID.ToToken) * **Python:** [s2sphere.CellId.to\_token()](https://s2sphere.readthedocs.io/en/latest/api.html#s2sphere.CellId) * **JavaScript:** [s2.cellid.toToken()](https://github.com/mapbox/node-s2/blob/master/API.md#cellidtotoken---string) ### Generate S2 cell ID tokens with Flux Use the [`geo.s2CellIDToken()` function](/flux/v0/stdlib/experimental/geo/s2cellidtoken/)with existing longitude (`lon`) and latitude (`lat`) field values to generate and add the S2 cell ID token. First, use the [`geo.toRows()` function](/flux/v0/stdlib/experimental/geo/torows/)to pivot **lat** and **lon** fields into row-wise sets: ```js import "experimental/geo" from(bucket: "example-bucket") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "example-measurement") |> geo.toRows() |> map(fn: (r) => ({r with s2_cell_id: geo.s2CellIDToken(point: {lon: r.lon, lat: r.lat}, level: 10)})) ``` > [!Note] > The [`geo.shapeData()`function](/flux/v0/stdlib/experimental/geo/shapedata/)generates S2 cell ID tokens as well. #### Related * [geo package](/flux/v0/stdlib/experimental/geo/) * [geo.shapeData() function](/flux/v0/stdlib/experimental/geo/shapedata/) --- # Group data in InfluxDB with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/group-data/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. With Flux, you can group data by any column in your queried data set. “Grouping” partitions data into tables in which each row shares a common value for specified columns. This guide walks through grouping data in Flux and provides examples of how data is shaped in the process. If you’re just getting started with Flux queries, check out the following: * [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query. * [Execute queries](/influxdb/v1/flux/guides/execute-queries/) to discover a variety of ways to run your queries. ## Group keys Every table has a **group key**—a list of columns which for which every row in the table has the same value. ###### Example group key ```js [_start, _stop, _field, _measurement, host] ``` Grouping data in Flux is essentially defining the group key of output tables. Understanding how modifying group keys shapes output data is key to successfully grouping and transforming data into your desired output. ## group() Function Flux’s [`group()` function](/flux/v0/stdlib/universe/group) defines the group key for output tables, i.e. grouping records based on values for specific columns. ###### group() example ```js dataStream |> group(columns: ["cpu", "host"]) ``` ###### Resulting group key ```js [cpu, host] ``` The `group()` function has the following parameters: ### columns The list of columns to include or exclude (depending on the [mode](#mode)) in the grouping operation. ### mode The method used to define the group and resulting group key. Possible values include `by` and `except`. ## Example grouping operations To illustrate how grouping works, define a `dataSet` variable that queries System CPU usage from the `db/rp` bucket. Filter the `cpu` tag so it only returns results for each numbered CPU core. ### Data set CPU used by system operations for all numbered CPU cores. It uses a regular expression to filter only numbered cores. ```js dataSet = from(bucket: "db/rp") |> range(start: -2m) |> filter(fn: (r) => r._field == "usage_system" and r.cpu =~ /cpu[0-9*]/) |> drop(columns: ["host"]) ``` > [!Note] > This example drops the `host` column from the returned data since the CPU data > is only tracked for a single host and it simplifies the output tables. > Don’t drop the `host` column if monitoring multiple hosts. ``` Table: keys: [_start, _stop, _field, _measurement, cpu] _start:time _stop:time _field:string _measurement:string cpu:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:00.000000000Z 7.892107892107892 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:10.000000000Z 7.2 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:20.000000000Z 7.4 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:30.000000000Z 5.5 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:40.000000000Z 7.4 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:34:50.000000000Z 7.5 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:00.000000000Z 10.3 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:10.000000000Z 9.2 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:20.000000000Z 8.4 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:30.000000000Z 8.5 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:40.000000000Z 8.6 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:35:50.000000000Z 10.2 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu0 2018-11-05T21:36:00.000000000Z 10.6 Table: keys: [_start, _stop, _field, _measurement, cpu] _start:time _stop:time _field:string _measurement:string cpu:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:00.000000000Z 0.7992007992007992 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:10.000000000Z 0.7 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:20.000000000Z 0.7 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:30.000000000Z 0.4 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:40.000000000Z 0.7 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:34:50.000000000Z 0.7 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:00.000000000Z 1.4 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:10.000000000Z 1.2 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:20.000000000Z 0.8 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:30.000000000Z 0.8991008991008991 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:40.000000000Z 0.8008008008008008 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:35:50.000000000Z 0.999000999000999 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu1 2018-11-05T21:36:00.000000000Z 1.1022044088176353 Table: keys: [_start, _stop, _field, _measurement, cpu] _start:time _stop:time _field:string _measurement:string cpu:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:00.000000000Z 4.1 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:10.000000000Z 3.6 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:20.000000000Z 3.5 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:30.000000000Z 2.6 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:40.000000000Z 4.5 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:34:50.000000000Z 4.895104895104895 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:00.000000000Z 6.906906906906907 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:10.000000000Z 5.7 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:20.000000000Z 5.1 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:30.000000000Z 4.7 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:40.000000000Z 5.1 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:35:50.000000000Z 5.9 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu2 2018-11-05T21:36:00.000000000Z 6.4935064935064934 Table: keys: [_start, _stop, _field, _measurement, cpu] _start:time _stop:time _field:string _measurement:string cpu:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:00.000000000Z 0.5005005005005005 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:10.000000000Z 0.5 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:20.000000000Z 0.5 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:30.000000000Z 0.3 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:40.000000000Z 0.6 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:34:50.000000000Z 0.6 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:00.000000000Z 1.3986013986013985 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:10.000000000Z 0.9 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:20.000000000Z 0.5005005005005005 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:30.000000000Z 0.7 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:40.000000000Z 0.6 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:35:50.000000000Z 0.8 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z usage_system cpu cpu3 2018-11-05T21:36:00.000000000Z 0.9 ``` [](#) **Note that the group key is output with each table: `Table: keys: `.** ![Group example data set](/img/flux/grouping-data-set.png) ### Group by CPU Group the `dataSet` stream by the `cpu` column. ```js dataSet |> group(columns: ["cpu"]) ``` This won’t actually change the structure of the data since it already has `cpu`in the group key and is therefore grouped by `cpu`. However, notice that it does change the group key: ###### Group by CPU output tables ``` Table: keys: [cpu] cpu:string _stop:time _time:time _value:float _field:string _measurement:string _start:time ---------------------- ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 7.892107892107892 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:10.000000000Z 7.2 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:20.000000000Z 7.4 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:30.000000000Z 5.5 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:40.000000000Z 7.4 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:50.000000000Z 7.5 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:00.000000000Z 10.3 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:10.000000000Z 9.2 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:20.000000000Z 8.4 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:30.000000000Z 8.5 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:40.000000000Z 8.6 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:50.000000000Z 10.2 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:36:00.000000000Z 10.6 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [cpu] cpu:string _stop:time _time:time _value:float _field:string _measurement:string _start:time ---------------------- ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 0.7992007992007992 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:10.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:20.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:30.000000000Z 0.4 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:40.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:50.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:00.000000000Z 1.4 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:10.000000000Z 1.2 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:20.000000000Z 0.8 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:30.000000000Z 0.8991008991008991 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:40.000000000Z 0.8008008008008008 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:50.000000000Z 0.999000999000999 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.1022044088176353 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [cpu] cpu:string _stop:time _time:time _value:float _field:string _measurement:string _start:time ---------------------- ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 4.1 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:10.000000000Z 3.6 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:20.000000000Z 3.5 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:30.000000000Z 2.6 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:40.000000000Z 4.5 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:50.000000000Z 4.895104895104895 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:00.000000000Z 6.906906906906907 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:10.000000000Z 5.7 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:20.000000000Z 5.1 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:30.000000000Z 4.7 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:40.000000000Z 5.1 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:50.000000000Z 5.9 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:36:00.000000000Z 6.4935064935064934 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [cpu] cpu:string _stop:time _time:time _value:float _field:string _measurement:string _start:time ---------------------- ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 0.5005005005005005 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:10.000000000Z 0.5 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:20.000000000Z 0.5 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:30.000000000Z 0.3 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:40.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:50.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:00.000000000Z 1.3986013986013985 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:10.000000000Z 0.9 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:20.000000000Z 0.5005005005005005 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:30.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:40.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:35:50.000000000Z 0.8 usage_system cpu 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu 2018-11-05T21:34:00.000000000Z ``` [](#) The visualization remains the same. ![Group by CPU](/img/flux/grouping-data-set.png) ### Group by time Grouping data by the `_time` column is a good illustration of how grouping changes the structure of your data. ```js dataSet |> group(columns: ["_time"]) ``` When grouping by `_time`, all records that share a common `_time` value are grouped into individual tables. So each output table represents a single point in time. ###### Group by time output tables ``` Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:34:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.892107892107892 usage_system cpu cpu0 2018-11-05T21:34:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7992007992007992 usage_system cpu cpu1 2018-11-05T21:34:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 4.1 usage_system cpu cpu2 2018-11-05T21:34:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.5005005005005005 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:34:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.2 usage_system cpu cpu0 2018-11-05T21:34:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu1 2018-11-05T21:34:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 3.6 usage_system cpu cpu2 2018-11-05T21:34:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.5 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:34:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.4 usage_system cpu cpu0 2018-11-05T21:34:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu1 2018-11-05T21:34:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 3.5 usage_system cpu cpu2 2018-11-05T21:34:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.5 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:34:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.5 usage_system cpu cpu0 2018-11-05T21:34:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.4 usage_system cpu cpu1 2018-11-05T21:34:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 2.6 usage_system cpu cpu2 2018-11-05T21:34:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.3 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:34:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.4 usage_system cpu cpu0 2018-11-05T21:34:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu1 2018-11-05T21:34:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 4.5 usage_system cpu cpu2 2018-11-05T21:34:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:34:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 7.5 usage_system cpu cpu0 2018-11-05T21:34:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu1 2018-11-05T21:34:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 4.895104895104895 usage_system cpu cpu2 2018-11-05T21:34:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:35:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 10.3 usage_system cpu cpu0 2018-11-05T21:35:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.4 usage_system cpu cpu1 2018-11-05T21:35:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 6.906906906906907 usage_system cpu cpu2 2018-11-05T21:35:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.3986013986013985 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:35:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 9.2 usage_system cpu cpu0 2018-11-05T21:35:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.2 usage_system cpu cpu1 2018-11-05T21:35:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.7 usage_system cpu cpu2 2018-11-05T21:35:10.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:35:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 8.4 usage_system cpu cpu0 2018-11-05T21:35:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.8 usage_system cpu cpu1 2018-11-05T21:35:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.1 usage_system cpu cpu2 2018-11-05T21:35:20.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.5005005005005005 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:35:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 8.5 usage_system cpu cpu0 2018-11-05T21:35:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.8991008991008991 usage_system cpu cpu1 2018-11-05T21:35:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 4.7 usage_system cpu cpu2 2018-11-05T21:35:30.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:35:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 8.6 usage_system cpu cpu0 2018-11-05T21:35:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.8008008008008008 usage_system cpu cpu1 2018-11-05T21:35:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.1 usage_system cpu cpu2 2018-11-05T21:35:40.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:35:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 10.2 usage_system cpu cpu0 2018-11-05T21:35:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.999000999000999 usage_system cpu cpu1 2018-11-05T21:35:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 5.9 usage_system cpu cpu2 2018-11-05T21:35:50.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.8 usage_system cpu cpu3 Table: keys: [_time] _time:time _start:time _stop:time _value:float _field:string _measurement:string cpu:string ------------------------------ ------------------------------ ------------------------------ ---------------------------- ---------------------- ---------------------- ---------------------- 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 10.6 usage_system cpu cpu0 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 1.1022044088176353 usage_system cpu cpu1 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 6.4935064935064934 usage_system cpu cpu2 2018-11-05T21:36:00.000000000Z 2018-11-05T21:34:00.000000000Z 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu cpu3 ``` [](#) Because each timestamp is structured as a separate table, when visualized, all points that share the same timestamp appear connected. ![Group by time](/img/flux/grouping-by-time.png) > [!Note] > With some further processing, you could calculate the average CPU usage across all CPUs per point > of time and group them into a single table, but we won’t cover that in this example. > If you’re interested in running and visualizing this yourself, here’s what the query would look like: > > ``` > dataSet > |> group(columns: ["_time"]) > |> mean() > |> group(columns: ["_value", "_time"], mode: "except") > ``` ## Group by CPU and time Group by the `cpu` and `_time` columns. ```js dataSet |> group(columns: ["cpu", "_time"]) ``` This outputs a table for every unique `cpu` and `_time` combination: ###### Group by CPU and time output tables ``` Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.892107892107892 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7992007992007992 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 4.1 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.5005005005005005 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:10.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.2 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:10.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:10.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 3.6 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:10.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.5 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:20.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.4 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:20.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:20.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 3.5 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:20.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.5 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:30.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 5.5 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:30.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.4 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:30.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 2.6 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:30.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.3 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:40.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.4 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:40.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:40.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 4.5 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:40.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:50.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 7.5 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:50.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:50.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 4.895104895104895 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:34:50.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 10.3 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 1.4 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 6.906906906906907 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 1.3986013986013985 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:10.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 9.2 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:10.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 1.2 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:10.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 5.7 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:10.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:20.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 8.4 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:20.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.8 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:20.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 5.1 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:20.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.5005005005005005 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:30.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 8.5 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:30.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.8991008991008991 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:30.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 4.7 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:30.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.7 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:40.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 8.6 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:40.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.8008008008008008 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:40.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 5.1 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:40.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.6 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:50.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 10.2 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:50.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 0.999000999000999 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:50.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 5.9 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:35:50.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.8 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:36:00.000000000Z cpu0 2018-11-05T21:36:00.000000000Z 10.6 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:36:00.000000000Z cpu1 2018-11-05T21:36:00.000000000Z 1.1022044088176353 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:36:00.000000000Z cpu2 2018-11-05T21:36:00.000000000Z 6.4935064935064934 usage_system cpu 2018-11-05T21:34:00.000000000Z Table: keys: [_time, cpu] _time:time cpu:string _stop:time _value:float _field:string _measurement:string _start:time ------------------------------ ---------------------- ------------------------------ ---------------------------- ---------------------- ---------------------- ------------------------------ 2018-11-05T21:36:00.000000000Z cpu3 2018-11-05T21:36:00.000000000Z 0.9 usage_system cpu 2018-11-05T21:34:00.000000000Z ``` [](#) When visualized, tables appear as individual, unconnected points. ![Group by CPU and time](/img/flux/grouping-by-cpu-time.png) Grouping by `cpu` and `_time` is a good illustration of how grouping works. ## In conclusion Grouping is a powerful way to shape your data into your desired output format. It modifies the group keys of output tables, grouping records into tables that all share common values within specified columns. --- # Create histograms with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/histograms/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Histograms provide valuable insight into the distribution of your data. This guide walks through using Flux’s `histogram()` function to transform your data into a **cumulative histogram**. ## histogram() function The [`histogram()` function](/flux/v0/stdlib/universe/histogram) approximates the cumulative distribution of a dataset by counting data frequencies for a list of “bins.” A **bin** is simply a range in which a data point falls. All data points that are less than or equal to the bound are counted in the bin. In the histogram output, a column is added (`le`) that represents the upper bounds of each bin. Bin counts are cumulative. ```js from(bucket: "telegraf/autogen") |> range(start: -5m) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") |> histogram(bins: [0.0, 10.0, 20.0, 30.0]) ``` > > > Values output by the `histogram` function represent points of data aggregated over time. > Since values do not represent single points in time, there is no `_time` column in the output table. > > ## Bin helper functions Flux provides two helper functions for generating histogram bins. Each generates and outputs an array of floats designed to be used in the `histogram()` function’s `bins` parameter. ### linearBins() The [`linearBins()` function](/flux/v0/stdlib/universe/linearbins) generates a list of linearly separated floats. ``` linearBins(start: 0.0, width: 10.0, count: 10) // Generated list: [0, 10, 20, 30, 40, 50, 60, 70, 80, 90, +Inf] ``` ### logarithmicBins() The [`logarithmicBins()` function](/flux/v0/stdlib/universe/logarithmicbins) generates a list of exponentially separated floats. ``` logarithmicBins(start: 1.0, factor: 2.0, count: 10, infinity: true) // Generated list: [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, +Inf] ``` ## Examples ### Generating a histogram with linear bins ```js from(bucket: "telegraf/autogen") |> range(start: -5m) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") |> histogram(bins: linearBins(start: 65.5, width: 0.5, count: 20, infinity: false)) ``` ###### Output table ``` Table: keys: [_start, _stop, _field, _measurement, host] _start:time _stop:time _field:string _measurement:string host:string le:float _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------ ---------------------------- ---------------------------- 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 65.5 5 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 66 6 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 66.5 8 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 67 9 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 67.5 9 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 68 10 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 68.5 12 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 69 12 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 69.5 15 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 70 23 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 70.5 30 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 71 30 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 71.5 30 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 72 30 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 72.5 30 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 73 30 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 73.5 30 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 74 30 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 74.5 30 2018-11-07T22:19:58.423658000Z 2018-11-07T22:24:58.423658000Z used_percent mem Scotts-MacBook-Pro.local 75 30 ``` ### Generating a histogram with logarithmic bins ```js from(bucket: "telegraf/autogen") |> range(start: -5m) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") |> histogram(bins: logarithmicBins(start: 0.5, factor: 2.0, count: 10, infinity: false)) ``` ###### Output table ``` Table: keys: [_start, _stop, _field, _measurement, host] _start:time _stop:time _field:string _measurement:string host:string le:float _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------ ---------------------------- ---------------------------- 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 0.5 0 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 1 0 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 2 0 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 4 0 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 8 0 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 16 0 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 32 0 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 64 2 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 128 30 2018-11-07T22:23:36.860664000Z 2018-11-07T22:28:36.860664000Z used_percent mem Scotts-MacBook-Pro.local 256 30 ``` --- # Calculate the increase Source: https://docs.influxdata.com/influxdb/v1/flux/guides/increase/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`increase()` function](/flux/v0/stdlib/universe/increase/)to track increases across multiple columns in a table. This function is especially useful when tracking changes in counter values that wrap over time or periodically reset. ```js data |> increase() ``` `increase()` returns a cumulative sum of **non-negative** differences between rows in a table. For example: **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1 | |2020-01-01T00:02:00Z| 2 | |2020-01-01T00:03:00Z| 8 | |2020-01-01T00:04:00Z| 10 | |2020-01-01T00:05:00Z| 0 | |2020-01-01T00:06:00Z| 4 | **`increase()` returns:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:02:00Z| 1 | |2020-01-01T00:03:00Z| 7 | |2020-01-01T00:04:00Z| 9 | |2020-01-01T00:05:00Z| 9 | |2020-01-01T00:06:00Z| 13 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1 | | 2020-01-01T00:02:00Z | 2 | | 2020-01-01T00:03:00Z | 8 | | 2020-01-01T00:04:00Z | 10 | | 2020-01-01T00:05:00Z | 0 | | 2020-01-01T00:06:00Z | 4 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:02:00Z | 1 | | 2020-01-01T00:03:00Z | 7 | | 2020-01-01T00:04:00Z | 9 | | 2020-01-01T00:05:00Z | 9 | | 2020-01-01T00:06:00Z | 13 | --- # Join data with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/join/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The [`join()` function](/flux/v0/stdlib/universe/join) merges two or more input streams, whose values are equal on a set of common columns, into a single output stream. Flux allows you to join on any columns common between two data streams and opens the door for operations such as cross-measurement joins and math across measurements. To illustrate a join operation, use data captured by Telegraf and stored in InfluxDB: memory usage and processes. In this guide, we’ll join two data streams, one representing memory usage and the other representing the total number of running processes, then calculate the average memory usage per running process. If you’re just getting started with Flux queries, check out the following: * [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query. * [Execute queries](/influxdb/v1/flux/guides/execute-queries/) to discover a variety of ways to run your queries. ## Define stream variables In order to perform a join, you must have two streams of data. Assign a variable to each data stream. ### Memory used variable Define a `memUsed` variable that filters on the `mem` measurement and the `used` field. This returns the amount of memory (in bytes) used. ###### memUsed stream definition ```js memUsed = from(bucket: "db/rp") |> range(start: -5m) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used") ``` ###### memUsed data output ``` Table: keys: [_start, _stop, _field, _measurement, host] _start:time _stop:time _field:string _measurement:string host:string _time:time _value:int ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------ ------------------------------ -------------------------- 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:00.000000000Z 10956333056 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:10.000000000Z 11014008832 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:20.000000000Z 11373428736 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:30.000000000Z 11001421824 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:40.000000000Z 10985852928 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:50:50.000000000Z 10992279552 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:00.000000000Z 11053568000 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:10.000000000Z 11092242432 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:20.000000000Z 11612774400 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:30.000000000Z 11131961344 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:40.000000000Z 11124805632 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:51:50.000000000Z 11332464640 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:00.000000000Z 11176923136 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:10.000000000Z 11181068288 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:20.000000000Z 11182579712 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:30.000000000Z 11238862848 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:40.000000000Z 11275296768 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:52:50.000000000Z 11225411584 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:00.000000000Z 11252690944 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:10.000000000Z 11227029504 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:20.000000000Z 11201646592 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:30.000000000Z 11227897856 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:40.000000000Z 11330428928 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:53:50.000000000Z 11347976192 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:00.000000000Z 11368271872 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:10.000000000Z 11269623808 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:20.000000000Z 11295637504 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:30.000000000Z 11354423296 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:40.000000000Z 11379687424 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:54:50.000000000Z 11248926720 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z used mem host1.local 2018-11-06T05:55:00.000000000Z 11292524544 ``` [](#) ### Total processes variable Define a `procTotal` variable that filters on the `processes` measurement and the `total` field. This returns the number of running processes. ###### procTotal stream definition ```js procTotal = from(bucket: "db/rp") |> range(start: -5m) |> filter(fn: (r) => r._measurement == "processes" and r._field == "total") ``` ###### procTotal data output ``` Table: keys: [_start, _stop, _field, _measurement, host] _start:time _stop:time _field:string _measurement:string host:string _time:time _value:int ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------ ------------------------------ -------------------------- 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:00.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:10.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:20.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:30.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:40.000000000Z 469 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:50:50.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:00.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:10.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:20.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:30.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:40.000000000Z 469 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:51:50.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:00.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:10.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:20.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:30.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:40.000000000Z 472 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:52:50.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:00.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:10.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:20.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:30.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:40.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:53:50.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:00.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:10.000000000Z 470 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:20.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:30.000000000Z 473 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:40.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:54:50.000000000Z 471 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z total processes host1.local 2018-11-06T05:55:00.000000000Z 471 ``` [](#) ## Join the two data streams With the two data streams defined, use the `join()` function to join them together.`join()` requires two parameters: ##### `tables` A map of tables to join with keys by which they will be aliased. In the example below, `mem` is the alias for `memUsed` and `proc` is the alias for `procTotal`. ##### `on` An array of strings defining the columns on which the tables will be joined.***Both tables must have all columns specified in this list.*** ```js join(tables: {mem: memUsed, proc: procTotal}, on: ["_time", "_stop", "_start", "host"]) ``` ###### Joined output table ``` Table: keys: [_field_mem, _field_proc, _measurement_mem, _measurement_proc, _start, _stop, host] _field_mem:string _field_proc:string _measurement_mem:string _measurement_proc:string _start:time _stop:time host:string _time:time _value_mem:int _value_proc:int ---------------------- ---------------------- ----------------------- ------------------------ ------------------------------ ------------------------------ ------------------------ ------------------------------ -------------------------- -------------------------- used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:00.000000000Z 10956333056 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:10.000000000Z 11014008832 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:20.000000000Z 11373428736 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:30.000000000Z 11001421824 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:40.000000000Z 10985852928 469 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:50.000000000Z 10992279552 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:00.000000000Z 11053568000 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:10.000000000Z 11092242432 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:20.000000000Z 11612774400 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:30.000000000Z 11131961344 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:40.000000000Z 11124805632 469 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:50.000000000Z 11332464640 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:00.000000000Z 11176923136 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:10.000000000Z 11181068288 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:20.000000000Z 11182579712 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:30.000000000Z 11238862848 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:40.000000000Z 11275296768 472 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:50.000000000Z 11225411584 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:00.000000000Z 11252690944 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:10.000000000Z 11227029504 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:20.000000000Z 11201646592 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:30.000000000Z 11227897856 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:40.000000000Z 11330428928 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:50.000000000Z 11347976192 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:00.000000000Z 11368271872 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:10.000000000Z 11269623808 470 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:20.000000000Z 11295637504 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:30.000000000Z 11354423296 473 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:40.000000000Z 11379687424 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:50.000000000Z 11248926720 471 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:55:00.000000000Z 11292524544 471 ``` [](#) Notice the output table includes the following columns: * `_field_mem` * `_field_proc` * `_measurement_mem` * `_measurement_proc` * `_value_mem` * `_value_proc` These represent the columns with values unique to the two input tables. ## Calculate and create a new table With the two streams of data joined into a single table, use the[`map()` function](/flux/v0/stdlib/universe/map)to build a new table by mapping the existing `_time` column to a new `_time`column and dividing `_value_mem` by `_value_proc` and mapping it to a new `_value` column. ```js join(tables: {mem: memUsed, proc: procTotal}, on: ["_time", "_stop", "_start", "host"]) |> map(fn: (r) => ({_time: r._time, _value: r._value_mem / r._value_proc})) ``` ###### Mapped table ``` Table: keys: [_field_mem, _field_proc, _measurement_mem, _measurement_proc, _start, _stop, host] _field_mem:string _field_proc:string _measurement_mem:string _measurement_proc:string _start:time _stop:time host:string _time:time _value:int ---------------------- ---------------------- ----------------------- ------------------------ ------------------------------ ------------------------------ ------------------------ ------------------------------ -------------------------- used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:00.000000000Z 23311346 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:10.000000000Z 23434061 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:20.000000000Z 24147407 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:30.000000000Z 23407280 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:40.000000000Z 23423993 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:50:50.000000000Z 23338173 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:00.000000000Z 23518229 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:10.000000000Z 23600515 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:20.000000000Z 24708030 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:30.000000000Z 23685024 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:40.000000000Z 23720267 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:51:50.000000000Z 24060434 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:00.000000000Z 23730197 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:10.000000000Z 23789506 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:20.000000000Z 23792722 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:30.000000000Z 23861704 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:40.000000000Z 23888340 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:52:50.000000000Z 23833145 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:00.000000000Z 23941895 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:10.000000000Z 23887296 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:20.000000000Z 23833290 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:30.000000000Z 23838424 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:40.000000000Z 24056112 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:53:50.000000000Z 24093367 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:00.000000000Z 24136458 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:10.000000000Z 23977922 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:20.000000000Z 23982245 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:30.000000000Z 24005123 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:40.000000000Z 24160695 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:54:50.000000000Z 23883071 used total mem processes 2018-11-06T05:50:00.000000000Z 2018-11-06T05:55:00.000000000Z Scotts-MacBook-Pro.local 2018-11-06T05:55:00.000000000Z 23975635 ``` [](#) This table represents the average amount of memory in bytes per running process. ## Real world example The following function calculates the batch sizes written to an InfluxDB cluster by joining fields from `httpd` and `write` measurements in order to compare `pointReq` and `writeReq`. The results are grouped by cluster ID so you can make comparisons across clusters. ```js batchSize = (cluster_id, start=-1m, interval=10s) => { httpd = from(bucket: "telegraf") |> range(start: start) |> filter(fn: (r) => r._measurement == "influxdb_httpd" and r._field == "writeReq" and r.cluster_id == cluster_id) |> aggregateWindow(every: interval, fn: mean) |> derivative(nonNegative: true, unit: 60s) write = from(bucket: "telegraf") |> range(start: start) |> filter(fn: (r) => r._measurement == "influxdb_write" and r._field == "pointReq" and r.cluster_id == cluster_id) |> aggregateWindow(every: interval, fn: max) |> derivative(nonNegative: true, unit: 60s) return join(tables: {httpd: httpd, write: write}, on: ["_time", "_stop", "_start", "host"]) |> map(fn: (r) => ({_time: r._time, _value: r._value_httpd / r._value_write})) |> group(columns: cluster_id) } batchSize(cluster_id: "enter cluster id here") ``` --- # Manipulate timestamps with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/manipulate-timestamps/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Every point stored in InfluxDB has an associated timestamp. Use Flux to process and manipulate timestamps to suit your needs. * [Convert timestamp format](#convert-timestamp-format) * [Calculate the duration between two timestamps](#calculate-the-duration-between-two-timestamps) * [Retrieve the current time](#retrieve-the-current-time) * [Normalize irregular timestamps](#normalize-irregular-timestamps) * [Use timestamps and durations together](#use-timestamps-and-durations-together) > [!Note] > If you’re just getting started with Flux queries, check out the following: > > * [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query. > * [Execute queries](/influxdb/v1/flux/guides/execute-queries/) to discover a variety of ways to run your queries. ## Convert timestamp format * [Unix nanosecond to RFC3339](#unix-nanosecond-to-rfc3339) * [RFC3339 to Unix nanosecond](#rfc3339-to-unix-nanosecond) ### Unix nanosecond to RFC3339 Use the [`time()` function](/flux/v0/stdlib/universe/time/)to convert a [Unix **nanosecond** timestamp](/influxdb/v2/reference/glossary/#unix-timestamp)to an [RFC3339 timestamp](/influxdb/v2/reference/glossary/#rfc3339-timestamp). ``` time(v: 1568808000000000000) // Returns 2019-09-18T12:00:00.000000000Z ``` ### RFC3339 to Unix nanosecond Use the [`uint()` function](/flux/v0/stdlib/universe/uint/)to convert an RFC3339 timestamp to a Unix nanosecond timestamp. ``` uint(v: 2019-09-18T12:00:00.000000000Z) // Returns 1568808000000000000 ``` ## Calculate the duration between two timestamps Flux doesn’t support mathematical operations using [time type](/flux/v0/spec/types/#time-types) values. To calculate the duration between two timestamps: 1. Use the `uint()` function to convert each timestamp to a Unix nanosecond timestamp. 2. Subtract one Unix nanosecond timestamp from the other. 3. Use the `duration()` function to convert the result into a duration. ``` time1 = uint(v: 2019-09-17T21:12:05Z) time2 = uint(v: 2019-09-18T22:16:35Z) duration(v: time2 - time1) // Returns 25h4m30s ``` > [!Note] > Flux doesn’t support duration column types. > To store a duration in a column, use the [`string()` function](/flux/v0/stdlib/universe/string/)to convert the duration to a string. ## Retrieve the current time * [Current UTC time](#current-utc-time) * [Current system time](#current-system-time) ### Current UTC time Use the [`now()` function](/flux/v0/stdlib/universe/now/) to return the current UTC time in RFC3339 format. ```js now() ``` > [!Note] > `now()` is cached at runtime, so all instances of `now()` in a Flux script > return the same value. ### Current system time Import the `system` package and use the [`system.time()` function](/flux/v0/stdlib/system/time/)to return the current system time of the host machine in RFC3339 format. ```js import "system" system.time() ``` > [!Note] > `system.time()` returns the time it is executed, so each instance of `system.time()`in a Flux script returns a unique value. ## Normalize irregular timestamps To normalize irregular timestamps, truncate all `_time` values to a specified unit with the [`truncateTimeColumn()` function](/flux/v0/stdlib/universe/truncatetimecolumn/). This is useful in [`join()`](/flux/v0/stdlib/universe/join/)and [`pivot()`](/flux/v0/stdlib/universe/pivot/)operations where points should align by time, but timestamps vary slightly. ```js data |> truncateTimeColumn(unit: 1m) ``` **Input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:49Z| 2.0 | |2020-01-01T00:01:01Z| 1.9 | |2020-01-01T00:03:22Z| 1.8 | |2020-01-01T00:04:04Z| 1.9 | |2020-01-01T00:05:38Z| 2.1 | **Output:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:00Z| 2.0 | |2020-01-01T00:01:00Z| 1.9 | |2020-01-01T00:03:00Z| 1.8 | |2020-01-01T00:04:00Z| 1.9 | |2020-01-01T00:05:00Z| 2.1 | ## Use timestamps and durations together * [Add a duration to a timestamp](#add-a-duration-to-a-timestamp) * [Subtract a duration from a timestamp](#subtract-a-duration-from-a-timestamp) ### Add a duration to a timestamp The [`experimental.addDuration()` function](/flux/v0/stdlib/experimental/addduration/)adds a duration to a specified time and returns the resulting time. By using `experimental.addDuration()`, you accept the[risks of experimental functions](/flux/v0/stdlib/experimental/#experimental-packages-are-subject-to-change). ``` import "experimental" experimental.addDuration(d: 6h, to: 2019-09-16T12:00:00Z) // Returns 2019-09-16T18:00:00.000000000Z ``` ### Subtract a duration from a timestamp The [`experimental.subDuration()` function](/flux/v0/stdlib/experimental/subduration/)subtracts a duration from a specified time and returns the resulting time. By using `experimental.subDuration()`, you accept the[risks of experimental functions](/flux/v0/stdlib/experimental/#experimental-packages-are-subject-to-change). ``` import "experimental" experimental.subDuration(d: 6h, from: 2019-09-16T12:00:00Z) // Returns 2019-09-16T06:00:00.000000000Z ``` | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:49Z | 2.0 | | 2020-01-01T00:01:01Z | 1.9 | | 2020-01-01T00:03:22Z | 1.8 | | 2020-01-01T00:04:04Z | 1.9 | | 2020-01-01T00:05:38Z | 2.1 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:00Z | 2.0 | | 2020-01-01T00:01:00Z | 1.9 | | 2020-01-01T00:03:00Z | 1.8 | | 2020-01-01T00:04:00Z | 1.9 | | 2020-01-01T00:05:00Z | 2.1 | --- # Transform data with mathematic operations Source: https://docs.influxdata.com/influxdb/v1/flux/guides/mathematic-operations/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Flux supports mathematic expressions in data transformations. This article describes how to use [Flux arithmetic operators](/flux/v0/spec/operators/#arithmetic-operators)to “map” over data and transform values using mathematic operations. If you’re just getting started with Flux queries, check out the following: * [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query. * [Execute queries](/influxdb/v1/flux/guides/execute-queries/) to discover a variety of ways to run your queries. ##### Basic mathematic operations ```js // Examples executed using the Flux REPL > 9 + 9 18 > 22 - 14 8 > 6 * 5 30 > 21 / 7 3 ``` See [Flux read-eval-print-loop (REPL)](/influxdb/v1/flux/guides/execute-queries/#influx-cli). > [!Note] > #### Operands must be the same type > > Operands in Flux mathematic operations must be the same data type. > For example, integers cannot be used in operations with floats. > Otherwise, you will get an error similar to: > > ``` > Error: type error: float != int > ``` > > To convert operands to the same type, use [type-conversion functions](/flux/v0/function-types#type-conversions)or manually format operands. > The operand data type determines the output data type. > For example: > > ``` > 100 // Parsed as an integer > 100.0 // Parsed as a float > > // Example evaluations > > 20 / 8 > 2 > > > 20.0 / 8.0 > 2.5 > ``` ## Custom mathematic functions Flux lets you [create custom functions](/influxdb/v2/query-data/flux/custom-functions) that use mathematic operations. View the examples below. ###### Custom multiplication function ``` multiply = (x, y) => x * y multiply(x: 10, y: 12) // Returns 120 ``` ###### Custom percentage function ``` percent = (sample, total) => (sample / total) * 100.0 percent(sample: 20.0, total: 80.0) // Returns 25.0 ``` ### Transform values in a data stream To transform multiple values in an input stream, your function needs to: * [Handle piped-forward data](/influxdb/v2/query-data/flux/custom-functions/#use-piped-forward-data-in-a-custom-function). * Each operand necessary for the calculation exists in each row *(see [Pivot vs join](#pivot-vs-join) below)*. * Use the [`map()` function](/flux/v0/stdlib/universe/map) to iterate over each row. The example `multiplyByX()` function below includes: * A `tables` parameter that represents the input data stream (`<-`). * An `x` parameter which is the number by which values in the `_value` column are multiplied. * A `map()` function that iterates over each row in the input stream. It uses the `with` operator to preserve existing columns in each row. It also multiples the `_value` column by `x`. ```js multiplyByX = (x, tables=<-) => tables |> map(fn: (r) => ({r with _value: r._value * x})) data |> multiplyByX(x: 10) ``` ## Examples ### Convert bytes to gigabytes To convert active memory from bytes to gigabytes (GB), divide the `active` field in the `mem` measurement by 1,073,741,824. The `map()` function iterates over each row in the piped-forward data and defines a new `_value` by dividing the original `_value` by 1073741824. ```js from(bucket: "db/rp") |> range(start: -10m) |> filter(fn: (r) => r._measurement == "mem" and r._field == "active") |> map(fn: (r) => ({r with _value: r._value / 1073741824})) ``` You could turn that same calculation into a function: ```js bytesToGB = (tables=<-) => tables |> map(fn: (r) => ({r with _value: r._value / 1073741824})) data |> bytesToGB() ``` #### Include partial gigabytes Because the original metric (bytes) is an integer, the output of the operation is an integer and does not include partial GBs. To calculate partial GBs, convert the `_value` column and its values to floats using the[`float()` function](/flux/v0/stdlib/universe/float)and format the denominator in the division operation as a float. ```js bytesToGB = (tables=<-) => tables |> map(fn: (r) => ({r with _value: float(v: r._value) / 1073741824.0})) ``` ### Calculate a percentage To calculate a percentage, use simple division, then multiply the result by 100. ```js > 1.0 / 4.0 * 100.0 25.0 ``` *For an in-depth look at calculating percentages, see [Calculate percentages](/influxdb/v1/flux/guides/calculate-percentages).* ## Pivot vs join To query and use values in mathematical operations in Flux, operand values must exists in a single row. Both `pivot()` and `join()` will do this, but there are important differences between the two: #### Pivot is more performant `pivot()` reads and operates on a single stream of data.`join()` requires two streams of data and the overhead of reading and combining both streams can be significant, especially for larger data sets. #### Use join for multiple data sources Use `join()` when querying data from different buckets or data sources. ##### Pivot fields into columns for mathematic calculations ```js data |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") |> map(fn: (r) => ({r with _value: (r.field1 + r.field2) / r.field3 * 100.0})) ``` ##### Join multiple data sources for mathematic calculations ```js import "sql" import "influxdata/influxdb/secrets" pgUser = secrets.get(key: "POSTGRES_USER") pgPass = secrets.get(key: "POSTGRES_PASSWORD") pgHost = secrets.get(key: "POSTGRES_HOST") t1 = sql.from( driverName: "postgres", dataSourceName: "postgresql://${pgUser}:${pgPass}@${pgHost}", query: "SELECT id, name, available FROM exampleTable", ) t2 = from(bucket: "db/rp") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "example-measurement" and r._field == "example-field") join(tables: {t1: t1, t2: t2}, on: ["id"]) |> map(fn: (r) => ({r with _value: r._value_t2 / r.available_t1 * 100.0})) ``` --- # Find median values Source: https://docs.influxdata.com/influxdb/v1/flux/guides/median/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`median()` function](/flux/v0/stdlib/universe/median/) to return a value representing the `0.5` quantile (50th percentile) or median of input data. ## Select a method for calculating the median Select one of the following methods to calculate the median: * [estimate\_tdigest](#estimate_tdigest) * [exact\_mean](#exact_mean) * [exact\_selector](#exact_selector) ### estimate\_tdigest **(Default)** An aggregate method that uses a [t-digest data structure](https://github.com/tdunning/t-digest)to compute an accurate `0.5` quantile estimate on large data sources. Output tables consist of a single row containing the calculated median. **Given the following input table:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | **`estimate_tdigest` returns:** |\_value| |-------| | 1.5 | ### exact\_mean An aggregate method that takes the average of the two points closest to the `0.5` quantile value. Output tables consist of a single row containing the calculated median. **Given the following input table:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | **`exact_mean` returns:** |\_value| |-------| | 1.5 | ### exact\_selector A selector method that returns the data point for which at least 50% of points are less than. Output tables consist of a single row containing the calculated median. **Given the following input table:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | **`exact_selector` returns:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:02:00Z| 1.0 | > [!Note] > The examples below use the [example data variable](/influxdb/v1/flux/guides/#example-data-variable). ## Find the value that represents the median Use the default method, `"estimate_tdigest"`, to return all rows in a table that contain values in the 50th percentile of data in the table. ```js data |> median() ``` ## Find the average of values closest to the median Use the `exact_mean` method to return a single row per input table containing the average of the two values closest to the mathematical median of data in the table. ```js data |> median(method: "exact_mean") ``` ## Find the point with the median value Use the `exact_selector` method to return a single row per input table containing the value that 50% of values in the table are less than. ```js data |> median(method: "exact_selector") ``` ## Use median() with aggregateWindow() [`aggregateWindow()`](/flux/v0/stdlib/universe/aggregatewindow/)segments data into windows of time, aggregates data in each window into a single point, and then removes the time-based segmentation. It is primarily used to downsample data. To specify the [median calculation method](#select-a-method-for-calculating-the-median) in `aggregateWindow()`, use the[full function syntax](/flux/v0/stdlib/universe/aggregatewindow/#specify-parameters-of-the-aggregate-function): ```js data |> aggregateWindow(every: 5m, fn: (tables=<-, column) => tables |> median(method: "exact_selector")) ``` | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _value | | --- | | _value | | 1.5 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _value | | --- | | _value | | 1.5 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:02:00Z | 1.0 | --- # Monitor states Source: https://docs.influxdata.com/influxdb/v1/flux/guides/monitor-states/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Flux helps you monitor states in your metrics and events: * [Find how long a state persists](#find-how-long-a-state-persists) * [Count the number of consecutive states](#count-the-number-of-consecutive-states) * [Detect state changes](#example-query-to-count-machine-state) If you’re just getting started with Flux queries, check out the following: * [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux. * [Execute queries](/influxdb/v1/flux/guides/executing-queries/) to discover a variety of ways to run your queries. ## Find how long a state persists 1. Use the [`stateDuration()`](/flux/v0/stdlib/universe/stateduration/) function to calculate how long a column value has remained the same value (or state). Include the following information: * **Column to search:** any tag key, tag value, field key, field value, or measurement. * **Value:** the value (or state) to search for in the specified column. * **State duration column:** a new column to store the state duration─the length of time that the specified value persists. * **Unit:** the unit of time (`1s` (by default), `1m`, `1h`) used to increment the state duration. ``` data |> stateDuration( fn: (r) => r._column_to_search == "value_to_search_for", column: "state_duration", unit: 1s, ) ``` 2. Use `stateDuration()` to search each point for the specified value: * For the first point that evaluates `true`, the state duration is set to `0`. For each consecutive point that evaluates `true`, the state duration increases by the time interval between each consecutive point (in specified units). * If the state is `false`, the state duration is reset to `-1`. ### Example query with stateDuration() The following query searches the `doors` bucket over the past 5 minutes to find how many seconds a door has been `closed`. ```js from(bucket: "doors") |> range(start: -5m) |> stateDuration( fn: (r) => r._value == "closed", column: "door_closed", unit: 1s, ) ``` In this example, `door_closed` is the **State duration** column. If you write data to the `doors` bucket every minute, the state duration increases by `60s` for each consecutive point where `_value` is `closed`. If `_value` is not `closed`, the state duration is reset to `0`. #### Query results Results for the example query above may look like this (for simplicity, we’ve omitted the measurement, tag, and field columns): ```bash _time _value door_closed 2019-10-26T17:39:16Z closed 0 2019-10-26T17:40:16Z closed 60 2019-10-26T17:41:16Z closed 120 2019-10-26T17:42:16Z open -1 2019-10-26T17:43:16Z closed 0 2019-10-26T17:44:27Z closed 60 ``` ## Count the number of consecutive states 1. Use the `stateCount()` function and include the following information: * **Column to search:** any tag key, tag value, field key, field value, or measurement. * **Value:** to search for in the specified column. * **State count column:** a new column to store the state count─the number of consecutive records in which the specified value exists. ``` data |> stateCount( fn: (r) => r._column_to_search == "value_to_search_for", column: "state_count" ) ``` 2. Use `stateCount()` to search each point for the specified value: * For the first point that evaluates `true`, the state count is set to `1`. For each consecutive point that evaluates `true`, the state count increases by 1. * If the state is `false`, the state count is reset to `-1`. ### Example query with stateCount() The following query searches the `doors` bucket over the past 5 minutes and calculates how many points have `closed` as their `_value`. ```js from(bucket: "doors") |> range(start: -5m) |> stateDuration(fn: (r) => r._value == "closed", column: "door_closed") ``` This example stores the **state count** in the `door_closed` column. If you write data to the `doors` bucket every minute, the state count increases by `1` for each consecutive point where `_value` is `closed`. If `_value` is not `closed`, the state count is reset to `-1`. #### Query results Results for the example query above may look like this (for simplicity, we’ve omitted the measurement, tag, and field columns): ```bash _time _value door_closed 2019-10-26T17:39:16Z closed 1 2019-10-26T17:40:16Z closed 2 2019-10-26T17:41:16Z closed 3 2019-10-26T17:42:16Z open -1 2019-10-26T17:43:16Z closed 1 2019-10-26T17:44:27Z closed 2 ``` #### Example query to count machine state The following query checks the machine state every minute (idle, assigned, or busy). InfluxDB searches the `servers` bucket over the past hour and counts records with a machine state of `idle`, `assigned` or `busy`. ```js from(bucket: "servers") |> range(start: -1h) |> filter(fn: (r) => r.machine_state == "idle" or r.machine_state == "assigned" or r.machine_state == "busy") |> stateCount(fn: (r) => r.machine_state == "busy", column: "_count") |> stateCount(fn: (r) => r.machine_state == "assigned", column: "_count") |> stateCount(fn: (r) => r.machine_state == "idle", column: "_count") ``` --- # Calculate the moving average Source: https://docs.influxdata.com/influxdb/v1/flux/guides/moving-average/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`movingAverage()`](/flux/v0/stdlib/universe/movingaverage/)or [`timedMovingAverage()`](/flux/v0/stdlib/universe/timedmovingaverage/)functions to return the moving average of data. ```js data |> movingAverage(n: 5) // OR data |> timedMovingAverage(every: 5m, period: 10m) ``` ### movingAverage() For each row in a table, `movingAverage()` returns the average of the current value and**previous** values where `n` is the total number of values used to calculate the average. If `n = 3`: |Row #| Calculation | |-----|-----------------------------| | 1 |*Insufficient number of rows*| | 2 |*Insufficient number of rows*| | 3 | (Row1 + Row2 + Row3) / 3 | | 4 | (Row2 + Row3 + Row4) / 3 | | 5 | (Row3 + Row4 + Row5) / 3 | **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.2 | |2020-01-01T00:03:00Z| 1.8 | |2020-01-01T00:04:00Z| 0.9 | |2020-01-01T00:05:00Z| 1.4 | |2020-01-01T00:06:00Z| 2.0 | **The following would return:** ```js |> movingAverage(n: 3) ``` | \_time |\_value| |--------------------|-------| |2020-01-01T00:03:00Z| 1.33 | |2020-01-01T00:04:00Z| 1.30 | |2020-01-01T00:05:00Z| 1.36 | |2020-01-01T00:06:00Z| 1.43 | ### timedMovingAverage() For each row in a table, `timedMovingAverage()` returns the average of the current value and all row values in the **previous** `period` (duration). It returns moving averages at a frequency defined by the `every` parameter. Each color in the diagram below represents a period of time used to calculate an average and the time a point representing the average is returned. If `every = 30m` and `period = 1h`: 0:000:301:001:302:002:303:003:002:302:001:301:000:30 **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.2 | |2020-01-01T00:03:00Z| 1.8 | |2020-01-01T00:04:00Z| 0.9 | |2020-01-01T00:05:00Z| 1.4 | |2020-01-01T00:06:00Z| 2.0 | **The following would return:** ```js |> timedMovingAverage(every: 2m, period: 4m) ``` | \_time |\_value| |--------------------|-------| |2020-01-01T00:02:00Z| 1.000 | |2020-01-01T00:04:00Z| 1.333 | |2020-01-01T00:06:00Z| 1.325 | |2020-01-01T00:06:00Z| 1.150 | | Row # | Calculation | | --- | --- | | Row # | Calculation | | 1 | Insufficient number of rows | | 2 | Insufficient number of rows | | 3 | (Row1 + Row2 + Row3) / 3 | | 4 | (Row2 + Row3 + Row4) / 3 | | 5 | (Row3 + Row4 + Row5) / 3 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.2 | | 2020-01-01T00:03:00Z | 1.8 | | 2020-01-01T00:04:00Z | 0.9 | | 2020-01-01T00:05:00Z | 1.4 | | 2020-01-01T00:06:00Z | 2.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:03:00Z | 1.33 | | 2020-01-01T00:04:00Z | 1.30 | | 2020-01-01T00:05:00Z | 1.36 | | 2020-01-01T00:06:00Z | 1.43 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.2 | | 2020-01-01T00:03:00Z | 1.8 | | 2020-01-01T00:04:00Z | 0.9 | | 2020-01-01T00:05:00Z | 1.4 | | 2020-01-01T00:06:00Z | 2.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:02:00Z | 1.000 | | 2020-01-01T00:04:00Z | 1.333 | | 2020-01-01T00:06:00Z | 1.325 | | 2020-01-01T00:06:00Z | 1.150 | --- # Optimize Flux queries Source: https://docs.influxdata.com/influxdb/v1/flux/guides/optimize-queries/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Optimize your Flux queries to reduce their memory and compute (CPU) requirements. * [Start queries with pushdowns](#start-queries-with-pushdowns) * [Avoid processing filters inline](#avoid-processing-filters-inline) * [Avoid short window durations](#avoid-short-window-durations) * [Use “heavy” functions sparingly](#use-heavy-functions-sparingly) * [Use set() instead of map() when possible](#use-set-instead-of-map-when-possible) * [Balance time range and data precision](#balance-time-range-and-data-precision) * [Measure query performance with Flux profilers](#measure-query-performance-with-flux-profilers) ## Start queries with pushdowns **Pushdowns** are functions or function combinations that push data operations to the underlying data source rather than operating on data in memory. Start queries with pushdowns to improve query performance. Once a non-pushdown function runs, Flux pulls data into memory and runs all subsequent operations there. #### Pushdown functions and function combinations The following pushdowns are supported in InfluxDB OSS 1.11+. | Functions |Supported| |----------------------------|---------| | **count()** | | | **drop()** | | | **duplicate()** | | | **filter()** \* | | | **fill()** | | | **first()** | | | **group()** | | | **keep()** | | | **last()** | | | **max()** | | | **mean()** | | | **min()** | | | **range()** | | | **rename()** | | | **sum()** | | | **window()** | | | *Function combinations* | | | **group()** |\> **max()** | | | **group()** |\> **min()** | | |**window()** |\> **count()**| | |**window()** |\> **first()**| | |**window()** |\> **last()** | | | **window()** |\> **max()** | | | **window()** |\> **min()** | | | **window()** |\> **sum()** | | \* **filter()** only pushes down when all parameter values are static. See [Avoid processing filters inline](#avoid-processing-filters-inline). Use pushdown functions and function combinations at the beginning of your query. Once a non-pushdown function runs, Flux pulls data into memory and runs all subsequent operations there. ##### Pushdown functions in use ``` from(bucket: "db/rp") |> range(start: -1h) // |> filter(fn: (r) => r.sensor == "abc123") // |> group(columns: ["_field", "host"]) // Pushed to the data source |> aggregateWindow(every: 5m, fn: max) // |> filter(fn: (r) => r._value >= 90.0) // |> top(n: 10) // Run in memory ``` ### Avoid processing filters inline Avoid using mathematic operations or string manipulation inline to define data filters. Processing filter values inline prevents `filter()` from pushing its operation down to the underlying data source, so data returned by the previous function loads into memory. This often results in a significant performance hit. For example, the following query uses [Chronograf dashboard template variables](/chronograf/v1/guides/dashboard-template-variables/)and string concatenation to define a region to filter by. Because `filter()` uses string concatenation inline, it can’t push its operation to the underlying data source and loads all data returned from `range()` into memory. ```js from(bucket: "db/rp") |> range(start: -1h) |> filter(fn: (r) => r.region == v.provider + v.region) ``` To dynamically set filters and maintain the pushdown ability of the `filter()` function, use variables to define filter values outside of `filter()`: ```js region = v.provider + v.region from(bucket: "db/rp") |> range(start: -1h) |> filter(fn: (r) => r.region == region) ``` ## Avoid short window durations Windowing (grouping data based on time intervals) is commonly used to aggregate and downsample data. Increase performance by avoiding short window durations. More windows require more compute power to evaluate which window each row should be assigned to. Reasonable window durations depend on the total time range queried. ## Use “heavy” functions sparingly The following functions use more memory or CPU than others. Consider their necessity in your data processing before using them: * [map()](/flux/v0/stdlib/universe/map/) * [reduce()](/flux/v0/stdlib/universe/reduce/) * [join()](/flux/v0/stdlib/universe/join/) * [union()](/flux/v0/stdlib/universe/union/) * [pivot()](/flux/v0/stdlib/universe/pivot/) ## Use set() instead of map() when possible [`set()`](/flux/v0/stdlib/universe/set/),[`experimental.set()`](/flux/v0/stdlib/experimental/set/), and [`map`](/flux/v0/stdlib/universe/map/)can each set column values in data, however **set** functions have performance advantages over `map()`. Use the following guidelines to determine which to use: * If setting a column value to a predefined, static value, use `set()` or `experimental.set()`. * If dynamically setting a column value using **existing row data**, use `map()`. #### Set a column value to a static value The following queries are functionally the same, but using `set()` is more performant than using `map()`. ```js data |> map(fn: (r) => ({ r with foo: "bar" })) // Recommended data |> set(key: "foo", value: "bar") ``` #### Dynamically set a column value using existing row data ```js data |> map(fn: (r) => ({ r with foo: r.bar })) ``` ## Balance time range and data precision To ensure queries are performant, balance the time range and the precision of your data. For example, if you query data stored every second and request six months worth of data, results would include ≈15.5 million points per series. Depending on the number of series returned after `filter()`([cardinality](/influxdb/v1/concepts/glossary/#series-cardinality)), this can quickly become many billions of points. Flux must store these points in memory to generate a response. Use [pushdowns](#pushdown-functions-and-function-combinations) to optimize how many points are stored in memory. ## Measure query performance with Flux profilers Use the [Flux Profiler package](/flux/v0/stdlib/profiler/)to measure query performance and append performance metrics to your query output. The following Flux profilers are available: * **query**: provides statistics about the execution of an entire Flux script. * **operator**: provides statistics about each operation in a query. Import the `profiler` package and enable profilers with the `profile.enabledProfilers` option. ``` import "profiler" option profiler.enabledProfilers = ["query", "operator"] // Query to profile ``` For more information about Flux profilers, see the [Flux Profiler package](/flux/v0/stdlib/profiler/). | Functions | Supported | | --- | --- | | Functions | Supported | | count() | | | drop() | | | duplicate() | | | filter() * | | | fill() | | | first() | | | group() | | | keep() | | | last() | | | max() | | | mean() | | | min() | | | range() | | | rename() | | | sum() | | | window() | | | Function combinations | | | group() |> max() | | | group() |> min() | | | window() |> count() | | | window() |> first() | | | window() |> last() | | | window() |> max() | | | window() |> min() | | | window() |> sum() | | --- # Find percentile and quantile values Source: https://docs.influxdata.com/influxdb/v1/flux/guides/percentile-quantile/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`quantile()` function](/flux/v0/stdlib/universe/quantile/)to return a value representing the `q` quantile or percentile of input data. ## Percentile versus quantile Percentiles and quantiles are very similar, differing only in the number used to calculate return values. A percentile is calculated using numbers between `0` and `100`. A quantile is calculated using numbers between `0.0` and `1.0`. For example, the **`0.5` quantile** is the same as the **50th percentile**. ## Select a method for calculating the quantile Select one of the following methods to calculate the quantile: * [estimate\_tdigest](#estimate_tdigest) * [exact\_mean](#exact_mean) * [exact\_selector](#exact_selector) ### estimate\_tdigest **(Default)** An aggregate method that uses a [t-digest data structure](https://github.com/tdunning/t-digest)to compute a quantile estimate on large data sources. Output tables consist of a single row containing the calculated quantile. If calculating the `0.5` quantile or 50th percentile: **Given the following input table:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | **`estimate_tdigest` returns:** |\_value| |-------| | 1.5 | ### exact\_mean An aggregate method that takes the average of the two points closest to the quantile value. Output tables consist of a single row containing the calculated quantile. If calculating the `0.5` quantile or 50th percentile: **Given the following input table:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | **`exact_mean` returns:** |\_value| |-------| | 1.5 | ### exact\_selector A selector method that returns the data point for which at least `q` points are less than. Output tables consist of a single row containing the calculated quantile. If calculating the `0.5` quantile or 50th percentile: **Given the following input table:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:01:00Z| 1.0 | |2020-01-01T00:02:00Z| 1.0 | |2020-01-01T00:03:00Z| 2.0 | |2020-01-01T00:04:00Z| 3.0 | **`exact_selector` returns:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:02:00Z| 1.0 | > [!Note] > The examples below use the [example data variable](/influxdb/v1/flux/guides/#example-data-variable). ## Find the value representing the 99th percentile Use the default method, `"estimate_tdigest"`, to return all rows in a table that contain values in the 99th percentile of data in the table. ```js data |> quantile(q: 0.99) ``` ## Find the average of values closest to the quantile Use the `exact_mean` method to return a single row per input table containing the average of the two values closest to the mathematical quantile of data in the table. For example, to calculate the `0.99` quantile: ```js data |> quantile(q: 0.99, method: "exact_mean") ``` ## Find the point with the quantile value Use the `exact_selector` method to return a single row per input table containing the value that `q * 100`% of values in the table are less than. For example, to calculate the `0.99` quantile: ```js data |> quantile(q: 0.99, method: "exact_selector") ``` ## Use quantile() with aggregateWindow() [`aggregateWindow()`](/flux/v0/stdlib/universe/aggregatewindow/)segments data into windows of time, aggregates data in each window into a single point, and then removes the time-based segmentation. It is primarily used to downsample data. To specify the [quantile calculation method](#select-a-method-for-calculating-the-quantile) in`aggregateWindow()`, use the [full function syntax](/flux/v0/stdlib/universe/aggregatewindow/#specify-parameters-of-the-aggregate-function): ```js data |> aggregateWindow( every: 5m, fn: (tables=<-, column) => tables |> quantile(q: 0.99, method: "exact_selector"), ) ``` | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _value | | --- | | _value | | 1.5 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _value | | --- | | _value | | 1.5 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:01:00Z | 1.0 | | 2020-01-01T00:02:00Z | 1.0 | | 2020-01-01T00:03:00Z | 2.0 | | 2020-01-01T00:04:00Z | 3.0 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:02:00Z | 1.0 | --- # Query fields and tags Source: https://docs.influxdata.com/influxdb/v1/flux/guides/query-fields/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`filter()` function](/flux/v0/stdlib/universe/filter/)to query data based on fields, tags, or any other column value.`filter()` performs operations similar to the `SELECT` statement and the `WHERE`clause in InfluxQL and other SQL-like query languages. ## The filter() function `filter()` has an `fn` parameter that expects a **predicate function**, an anonymous function comprised of one or more **predicate expressions**. The predicate function evaluates each input row. Rows that evaluate to `true` are **included** in the output data. Rows that evaluate to `false` are **excluded** from the output data. ```js // ... |> filter(fn: (r) => r._measurement == "example-measurement" ) ``` The `fn` predicate function requires an `r` argument, which represents each row as `filter()` iterates over input data. Key-value pairs in the row record represent columns and their values. Use **dot notation** or **bracket notation** to reference specific column values in the predicate function. Use [logical operators](/flux/v0/spec/operators/#logical-operators)to chain multiple predicate expressions together. ```js // Row record r = {foo: "bar", baz: "quz"} // Example predicate function (r) => r.foo == "bar" and r["baz"] == "quz" // Evaluation results (r) => true and true ``` ## Filter by fields and tags The combination of [`from()`](/flux/v0/stdlib/influxdata/influxdb/from),[`range()`](/flux/v0/stdlib/universe/range), and `filter()` represent the most basic Flux query: 1. Use `from()` to define your [bucket](/influxdb/v1/flux/get-started/#buckets). 2. Use `range()` to limit query results by time. 3. Use `filter()` to identify what rows of data to output. ```js from(bucket: "db/rp") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "example-measurement" and r.tag == "example-tag") |> filter(fn: (r) => r._field == "example-field") ``` --- # Calculate the rate of change Source: https://docs.influxdata.com/influxdb/v1/flux/guides/rate/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`derivative()` function](/flux/v0/stdlib/universe/derivative/)to calculate the rate of change between subsequent values or the[`aggregate.rate()` function](/flux/v0/stdlib/experimental/aggregate/rate/)to calculate the average rate of change per window of time. If time between points varies, these functions normalize points to a common time interval making values easily comparable. * [Rate of change between subsequent values](#rate-of-change-between-subsequent-values) * [Average rate of change per window of time](#average-rate-of-change-per-window-of-time) ## Rate of change between subsequent values Use the [`derivative()` function](/flux/v0/stdlib/universe/derivative/)to calculate the rate of change per unit of time between subsequent *non-null* values. ```js data |> derivative(unit: 1s) ``` By default, `derivative()` returns only positive derivative values and replaces negative values with *null*. Calculated values are returned as [floats](/flux/v0/spec/types/#numeric-types). **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:00Z| 250 | |2020-01-01T00:04:00Z| 160 | |2020-01-01T00:12:00Z| 150 | |2020-01-01T00:19:00Z| 220 | |2020-01-01T00:32:00Z| 200 | |2020-01-01T00:51:00Z| 290 | |2020-01-01T01:00:00Z| 340 | **`derivative(unit: 1m)` returns:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:04:00Z| | |2020-01-01T00:12:00Z| | |2020-01-01T00:19:00Z| 10.0 | |2020-01-01T00:32:00Z| | |2020-01-01T00:51:00Z| 4.74 | |2020-01-01T01:00:00Z| 5.56 | Results represent the rate of change **per minute** between subsequent values with negative values set to *null*. ### Return negative derivative values To return negative derivative values, set the `nonNegative` parameter to `false`, **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:00Z| 250 | |2020-01-01T00:04:00Z| 160 | |2020-01-01T00:12:00Z| 150 | |2020-01-01T00:19:00Z| 220 | |2020-01-01T00:32:00Z| 200 | |2020-01-01T00:51:00Z| 290 | |2020-01-01T01:00:00Z| 340 | **The following returns:** ```js |> derivative(unit: 1m, nonNegative: false) ``` | \_time |\_value| |--------------------|-------| |2020-01-01T00:04:00Z|\-22.5 | |2020-01-01T00:12:00Z|\-1.25 | |2020-01-01T00:19:00Z| 10.0 | |2020-01-01T00:32:00Z|\-1.54 | |2020-01-01T00:51:00Z| 4.74 | |2020-01-01T01:00:00Z| 5.56 | Results represent the rate of change **per minute** between subsequent values and include negative values. ## Average rate of change per window of time Use the [`aggregate.rate()` function](/flux/v0/stdlib/experimental/aggregate/rate/)to calculate the average rate of change per window of time. ```js import "experimental/aggregate" data |> aggregate.rate(every: 1m, unit: 1s, groupColumns: ["tag1", "tag2"]) ``` `aggregate.rate()` returns the average rate of change (as a [float](/flux/v0/spec/types/#numeric-types)) per `unit` for time intervals defined by `every`. Negative values are replaced with *null*. > [!Note] > `aggregate.rate()` does not support `nonNegative: false`. **Given the following input:** | \_time |\_value| |--------------------|-------| |2020-01-01T00:00:00Z| 250 | |2020-01-01T00:04:00Z| 160 | |2020-01-01T00:12:00Z| 150 | |2020-01-01T00:19:00Z| 220 | |2020-01-01T00:32:00Z| 200 | |2020-01-01T00:51:00Z| 290 | |2020-01-01T01:00:00Z| 340 | **The following returns:** ```js |> aggregate.rate(every: 20m, unit: 1m) ``` | \_time |\_value| |--------------------|-------| |2020-01-01T00:20:00Z| | |2020-01-01T00:40:00Z| 10.0 | |2020-01-01T01:00:00Z| 4.74 | |2020-01-01T01:20:00Z| 5.56 | Results represent the **average change rate per minute** of every **20 minute interval**with negative values set to *null*. Timestamps represent the right bound of the time window used to average values. | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:00Z | 250 | | 2020-01-01T00:04:00Z | 160 | | 2020-01-01T00:12:00Z | 150 | | 2020-01-01T00:19:00Z | 220 | | 2020-01-01T00:32:00Z | 200 | | 2020-01-01T00:51:00Z | 290 | | 2020-01-01T01:00:00Z | 340 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:04:00Z | | | 2020-01-01T00:12:00Z | | | 2020-01-01T00:19:00Z | 10.0 | | 2020-01-01T00:32:00Z | | | 2020-01-01T00:51:00Z | 4.74 | | 2020-01-01T01:00:00Z | 5.56 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:00Z | 250 | | 2020-01-01T00:04:00Z | 160 | | 2020-01-01T00:12:00Z | 150 | | 2020-01-01T00:19:00Z | 220 | | 2020-01-01T00:32:00Z | 200 | | 2020-01-01T00:51:00Z | 290 | | 2020-01-01T01:00:00Z | 340 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:04:00Z | -22.5 | | 2020-01-01T00:12:00Z | -1.25 | | 2020-01-01T00:19:00Z | 10.0 | | 2020-01-01T00:32:00Z | -1.54 | | 2020-01-01T00:51:00Z | 4.74 | | 2020-01-01T01:00:00Z | 5.56 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:00:00Z | 250 | | 2020-01-01T00:04:00Z | 160 | | 2020-01-01T00:12:00Z | 150 | | 2020-01-01T00:19:00Z | 220 | | 2020-01-01T00:32:00Z | 200 | | 2020-01-01T00:51:00Z | 290 | | 2020-01-01T01:00:00Z | 340 | | _time | _value | | --- | --- | | _time | _value | | 2020-01-01T00:20:00Z | | | 2020-01-01T00:40:00Z | 10.0 | | 2020-01-01T01:00:00Z | 4.74 | | 2020-01-01T01:20:00Z | 5.56 | --- # Use regular expressions in Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/regular-expressions/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Regular expressions (regexes) are incredibly powerful when matching patterns in large collections of data. With Flux, regular expressions are primarily used for evaluation logic in predicate functions for things such as filtering rows, dropping and keeping columns, state detection, etc. This guide shows how to use regular expressions in your Flux scripts. If you’re just getting started with Flux queries, check out the following: * [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query. * [Execute queries](/influxdb/v1/flux/guides/execute-queries/) to discover a variety of ways to run your queries. ## Go regular expression syntax Flux uses Go’s [regexp package](https://golang.org/pkg/regexp/) for regular expression search. The links [below](#helpful-links) provide information about Go’s regular expression syntax. ## Regular expression operators Flux provides two comparison operators for use with regular expressions. #### `=~` When the expression on the left **MATCHES** the regular expression on the right, this evaluates to `true`. #### `!~` When the expression on the left **DOES NOT MATCH** the regular expression on the right, this evaluates to `true`. ## Regular expressions in Flux When using regex matching in your Flux scripts, enclose your regular expressions with `/`. The following is the basic regex comparison syntax: ###### Basic regex comparison syntax ```js expression =~ /regex/ expression !~ /regex/ ``` ## Examples ### Use a regex to filter by tag value The following example filters records by the `cpu` tag. It only keeps records for which the `cpu` is either `cpu0`, `cpu1`, or `cpu2`. ```js from(bucket: "db/rp") |> range(start: -15m) |> filter(fn: (r) => r._measurement == "cpu" and r._field == "usage_user" and r.cpu =~ /cpu[0-2]/) ``` ### Use a regex to filter by field key The following example excludes records that do not have `_percent` in a field key. ```js from(bucket: "db/rp") |> range(start: -15m) |> filter(fn: (r) => r._measurement == "mem" and r._field =~ /_percent/) ``` ### Drop columns matching a regex The following example drops columns whose names do not begin with `_`. ```js from(bucket: "db/rp") |> range(start: -15m) |> filter(fn: (r) => r._measurement == "mem") |> drop(fn: (column) => column !~ /_.*/) ``` ## Helpful links ##### Syntax documentation [regexp Syntax GoDoc](https://godoc.org/regexp/syntax) [RE2 Syntax Overview](https://github.com/google/re2/wiki/Syntax) ##### Go regex testers [Regex101](https://regex101.com/) --- # Extract scalar values in Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/scalar-values/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use Flux [dynamic query functions](/flux/v0/function-types/#dynamic-queries)to extract scalar values from Flux query output. This lets you, for example, dynamically set variables using query results. **To extract scalar values from output:** 1. [Extract a table](#extract-a-table). 2. [Extract a column from the table](#extract-a-column-from-the-table)***or*** [extract a row from the table](#extract-a-row-from-the-table). *The samples on this page use the [sample data provided below](#sample-data).* #### Current limitations * The InfluxDB user interface (UI) does not currently support raw scalar output. Use [`map()`](/flux/v0/stdlib/universe/map/) to add scalar values to output data. * The [Flux REPL](/influxdb/v1/flux/guides/execute-queries/#influx-cli) does not currently support Flux stream and table functions (also known as “dynamic queries”). See [#15321](https://github.com/influxdata/influxdb/issues/15231). ## Extract a table Flux formats query results as a stream of tables. To extract a scalar value from a stream of tables, you must first extract a single table. to extract a single table from the stream of tables. > [!Note] > If query results include only one table, it is still formatted as a stream of tables. > You still must extract that table from the stream. Use [`tableFind()`](/flux/v0/stdlib/universe/tablefind/)to extract the **first** table whose [group key](/influxdb/v1/flux/get-started/#group-keys)values match the `fn` **predicate function**. The predicate function requires a `key` record, which represents the group key of each table. ```js sampleData |> tableFind(fn: (key) => key._field == "temp" and key.location == "sfo") ``` The example above returns a single table: | \_time |location|\_field|\_value| |--------------------|--------|-------|-------| |2019-11-01T12:00:00Z| sfo | temp | 65.1 | |2019-11-01T13:00:00Z| sfo | temp | 66.2 | |2019-11-01T14:00:00Z| sfo | temp | 66.3 | |2019-11-01T15:00:00Z| sfo | temp | 66.8 | > [!Note] > #### Extract the correct table > > Flux functions do not guarantee table order and `tableFind()` returns only the**first** table that matches the `fn` predicate. > To extract the table that includes the data you actually want, be very specific in > your predicate function or filter and transform your data to minimize the number > of tables piped-forward into `tableFind()`. ## Extract a column from the table Use the [`getColumn()` function](/flux/v0/stdlib/universe/getcolumn/)to output an array of values from a specific column in the extracted table. ``` sampleData |> tableFind(fn: (key) => key._field == "temp" and key.location == "sfo") |> getColumn(column: "_value") // Returns [65.1, 66.2, 66.3, 66.8] ``` ### Use extracted column values Use a variable to store the array of values. In the example below, `SFOTemps` represents the array of values. Reference a specific index (integer starting from `0`) in the array to return the value at that index. ``` SFOTemps = sampleData |> tableFind(fn: (key) => key._field == "temp" and key.location == "sfo") |> getColumn(column: "_value") SFOTemps // Returns [65.1, 66.2, 66.3, 66.8] SFOTemps[0] // Returns 65.1 SFOTemps[2] // Returns 66.3 ``` ## Extract a row from the table Use the [`getRecord()` function](/flux/v0/stdlib/universe/getrecord/)to output data from a single row in the extracted table. Specify the index of the row to output using the `idx` parameter. The function outputs a record with key-value pairs for each column. ``` sampleData |> tableFind(fn: (key) => key._field == "temp" and key.location == "sfo") |> getRecord(idx: 0) // Returns { // _time:2019-11-11T12:00:00Z, // _field:"temp", // location:"sfo", // _value: 65.1 // } ``` ### Use an extracted row record Use a variable to store the extracted row record. In the example below, `tempInfo` represents the extracted row. Use [dot notation](/influxdb/v1/flux/get-started/syntax-basics/#records) to reference keys in the record. ``` tempInfo = sampleData |> tableFind(fn: (key) => key._field == "temp" and key.location == "sfo") |> getRecord(idx: 0) tempInfo // Returns { // _time:2019-11-11T12:00:00Z, // _field:"temp", // location:"sfo", // _value: 65.1 // } tempInfo._time // Returns 2019-11-11T12:00:00Z tempInfo.location // Returns sfo ``` ## Example helper functions Create custom helper functions to extract scalar values from query output. ##### Extract a scalar field value ``` // Define a helper function to extract field values getFieldValue = (tables=<-, field) => { extract = tables |> tableFind(fn: (key) => key._field == field) |> getColumn(column: "_value") return extract[0] } // Use the helper function to define a variable lastJFKTemp = sampleData |> filter(fn: (r) => r.location == "kjfk") |> last() |> getFieldValue(field: "temp") lastJFKTemp // Returns 71.2 ``` ##### Extract scalar row data ``` // Define a helper function to extract a row as a record getRow = (tables=<-, field, idx=0) => { extract = tables |> tableFind(fn: (key) => true) |> getRecord(idx: idx) return extract } // Use the helper function to define a variable lastReported = sampleData |> last() |> getRow(field: "temp") "The last location to report was ${lastReported.location}. The temperature was ${string(v: lastReported._value)}°F." // Returns: // The last location to report was kord. // The temperature was 38.9°F. ``` ## Sample data The following sample data set represents fictional temperature metrics collected from three locations. It’s formatted in [annotated CSV](/influxdb/v2/reference/syntax/annotated-csv/) and imported into the Flux query using the [`csv.from()` function](/flux/v0/stdlib/csv/from/). Place the following at the beginning of your query to use the sample data: ```js import "csv" sampleData = csv.from(csv: " #datatype,string,long,dateTime:RFC3339,string,string,double #group,false,true,false,true,true,false #default,,,,,, ,result,table,_time,location,_field,_value ,,0,2019-11-01T12:00:00Z,sfo,temp,65.1 ,,0,2019-11-01T13:00:00Z,sfo,temp,66.2 ,,0,2019-11-01T14:00:00Z,sfo,temp,66.3 ,,0,2019-11-01T15:00:00Z,sfo,temp,66.8 ,,1,2019-11-01T12:00:00Z,kjfk,temp,69.4 ,,1,2019-11-01T13:00:00Z,kjfk,temp,69.9 ,,1,2019-11-01T14:00:00Z,kjfk,temp,71.0 ,,1,2019-11-01T15:00:00Z,kjfk,temp,71.2 ,,2,2019-11-01T12:00:00Z,kord,temp,46.4 ,,2,2019-11-01T13:00:00Z,kord,temp,46.3 ,,2,2019-11-01T14:00:00Z,kord,temp,42.7 ,,2,2019-11-01T15:00:00Z,kord,temp,38.9 ") ``` [](#) | _time | location | _field | _value | | --- | --- | --- | --- | | _time | location | _field | _value | | 2019-11-01T12:00:00Z | sfo | temp | 65.1 | | 2019-11-01T13:00:00Z | sfo | temp | 66.2 | | 2019-11-01T14:00:00Z | sfo | temp | 66.3 | | 2019-11-01T15:00:00Z | sfo | temp | 66.8 | --- # Sort and limit data with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/sort-limit/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use the [`sort()`function](/flux/v0/stdlib/universe/sort)to order records within each table by specific columns and the[`limit()` function](/flux/v0/stdlib/universe/limit)to limit the number of records in output tables to a fixed number, `n`. If you’re just getting started with Flux queries, check out the following: * [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query. * [Execute queries](/influxdb/v1/flux/guides/execute-queries/) to discover a variety of ways to run your queries. ##### Example sorting system uptime The following example orders system uptime first by region, then host, then value. ```js from(bucket: "db/rp") |> range(start: -12h) |> filter(fn: (r) => r._measurement == "system" and r._field == "uptime") |> sort(columns: ["region", "host", "_value"]) ``` The [`limit()` function](/flux/v0/stdlib/universe/limit)limits the number of records in output tables to a fixed number, `n`. The following example shows up to 10 records from the past hour. ```js from(bucket:"db/rp") |> range(start:-1h) |> limit(n:10) ``` You can use `sort()` and `limit()` together to show the top N records. The example below returns the 10 top system uptime values sorted first by region, then host, then value. ```js from(bucket: "db/rp") |> range(start: -12h) |> filter(fn: (r) => r._measurement == "system" and r._field == "uptime") |> sort(columns: ["region", "host", "_value"]) |> limit(n: 10) ``` You now have created a Flux query that sorts and limits data. Flux also provides the [`top()`](/flux/v0/stdlib/universe/top)and [`bottom()`](/flux/v0/stdlib/universe/bottom)functions to perform both of these functions at the same time. --- # Query SQL data sources Source: https://docs.influxdata.com/influxdb/v1/flux/guides/sql/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The Flux `sql` package provides functions for working with SQL data sources.[`sql.from()`](/flux/v0/stdlib/sql/from/) lets you query SQL data sources like [PostgreSQL](https://www.postgresql.org/), [MySQL](https://www.mysql.com/), and [SQLite](https://www.sqlite.org/index.html), and use the results with InfluxDB dashboards, tasks, and other operations. * [Query a SQL data source](#query-a-sql-data-source) * [Join SQL data with data in InfluxDB](#join-sql-data-with-data-in-influxdb) * [Sample sensor data](#sample-sensor-data) ## Query a SQL data source To query a SQL data source: 1. Import the `sql` package in your Flux query 2. Use the `sql.from()` function to specify the driver, data source name (DSN), and query used to query data from your SQL data source: [PostgreSQL](#postgresql)[MySQL](#mysql)[SQLite](#sqlite) ```js import "sql" sql.from( driverName: "postgres", dataSourceName: "postgresql://user:password@localhost", query: "SELECT * FROM example_table", ) ``` ```js import "sql" sql.from( driverName: "mysql", dataSourceName: "user:password@tcp(localhost:3306)/db", query: "SELECT * FROM example_table", ) ``` ```js // NOTE: InfluxDB OSS and InfluxDB Cloud do not have access to // the local filesystem and cannot query SQLite data sources. // Use the Flux REPL to query an SQLite data source. import "sql" sql.from( driverName: "sqlite3", dataSourceName: "file:/path/to/test.db?cache=shared&mode=ro", query: "SELECT * FROM example_table", ) ``` *See the [`sql.from()` documentation](/flux/v0/stdlib/sql/from/) for information about required function parameters.* ## Join SQL data with data in InfluxDB One of the primary benefits of querying SQL data sources from InfluxDB is the ability to enrich query results with data stored outside of InfluxDB. Using the [air sensor sample data](#sample-sensor-data) below, the following query joins air sensor metrics stored in InfluxDB with sensor information stored in PostgreSQL. The joined data lets you query and filter results based on sensor information that isn’t stored in InfluxDB. ```js // Import the "sql" package import "sql" // Query data from PostgreSQL sensorInfo = sql.from( driverName: "postgres", dataSourceName: "postgresql://localhost?sslmode=disable", query: "SELECT * FROM sensors", ) // Query data from InfluxDB sensorMetrics = from(bucket: "telegraf/autogen") |> range(start: -1h) |> filter(fn: (r) => r._measurement == "airSensors") // Join InfluxDB query results with PostgreSQL query results join(tables: {metric: sensorMetrics, info: sensorInfo}, on: ["sensor_id"]) ``` ## Sample sensor data The [sample data generator](#download-and-run-the-sample-data-generator) and[sample sensor information](#import-the-sample-sensor-information) simulate a group of sensors that measure temperature, humidity, and carbon monoxide in rooms throughout a building. Each collected data point is stored in InfluxDB with a `sensor_id` tag that identifies the specific sensor it came from. Sample sensor information is stored in PostgreSQL. **Sample data includes:** * Simulated data collected from each sensor and stored in the `airSensors` measurement in **InfluxDB**: * temperature * humidity * co * Information about each sensor stored in the `sensors` table in **PostgreSQL**: * sensor\_id * location * model\_number * last\_inspected ### Import and generate sample sensor data #### Download and run the sample data generator `air-sensor-data.rb` is a script that generates air sensor data and stores the data in InfluxDB. To use `air-sensor-data.rb`: 1. [Create a database](/influxdb/v1/query_language/manage-database/#create-database) to store the data. 2. Download the sample data generator. *This tool requires [Ruby](https://www.ruby-lang.org/en/).* [Download Air Sensor Generator](/downloads/air-sensor-data.rb) 3. Give `air-sensor-data.rb` executable permissions: ``` chmod +x air-sensor-data.rb ``` 4. Start the generator. Specify your database. ``` ./air-sensor-data.rb -d database-name ``` The generator begins to write data to InfluxDB and will continue until stopped. Use `ctrl-c` to stop the generator. ***Note:** Use the `--help` flag to view other configuration options.* 5. Query your target database to ensure the generated data is writing successfully. The generator doesn’t catch errors from write requests, so it will continue running even if data is not writing to InfluxDB successfully. ``` from(bucket: "database-name/autogen") |> range(start: -1m) |> filter(fn: (r) => r._measurement == "airSensors") ``` #### Import the sample sensor information 1. [Download and install PostgreSQL](https://www.postgresql.org/download/). 2. Download the sample sensor information CSV. [Download Sample Data](/downloads/sample-sensor-info.csv) 3. Use a PostgreSQL client (`psql` or a GUI) to create the `sensors` table: ``` CREATE TABLE sensors ( sensor_id character varying(50), location character varying(50), model_number character varying(50), last_inspected date ); ``` 4. Import the downloaded CSV sample data.*Update the `FROM` file path to the path of the downloaded CSV sample data.* ``` COPY sensors(sensor_id,location,model_number,last_inspected) FROM '/path/to/sample-sensor-info.csv' DELIMITER ',' CSV HEADER; ``` 5. Query the table to ensure the data was imported correctly: ``` SELECT * FROM sensors; ``` --- # Window and aggregate data with Flux Source: https://docs.influxdata.com/influxdb/v1/flux/guides/window-aggregate/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. A common operation performed with time series data is grouping data into windows of time, or “windowing” data, then aggregating windowed values into a new value. This guide walks through windowing and aggregating data with Flux and demonstrates how data is shaped in the process. If you’re just getting started with Flux queries, check out the following: * [Get started with Flux](/influxdb/v1/flux/get-started/) for a conceptual overview of Flux and parts of a Flux query. * [Execute queries](/influxdb/v1/flux/guides/execute-queries/) to discover a variety of ways to run your queries. > [!Note] > The following example is an in-depth walk-through of the steps required to window and aggregate data. > The [`aggregateWindow()` function](#summing-up) performs these operations for you, but understanding > how data is shaped in the process helps to successfully create your desired output. ## Data set For the purposes of this guide, define a variable that represents your base data set. The following example queries the memory usage of the host machine. ```js dataSet = from(bucket: "db/rp") |> range(start: -5m) |> filter(fn: (r) => r._measurement == "mem" and r._field == "used_percent") |> drop(columns: ["host"]) ``` > [!Note] > This example drops the `host` column from the returned data since the memory data > is only tracked for a single host and it simplifies the output tables. > Dropping the `host` column is optional and not recommended if monitoring memory > on multiple hosts. `dataSet` can now be used to represent your base data, which will look similar to the following: ``` Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:00.000000000Z 71.11611366271973 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:10.000000000Z 67.39630699157715 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:20.000000000Z 64.16666507720947 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:30.000000000Z 64.19951915740967 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:40.000000000Z 64.2122745513916 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:50:50.000000000Z 64.22209739685059 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:00.000000000Z 64.6336555480957 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:10.000000000Z 64.16516304016113 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:20.000000000Z 64.18349742889404 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:30.000000000Z 64.20474052429199 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:40.000000000Z 68.65062713623047 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:50.000000000Z 67.20139980316162 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:00.000000000Z 70.9143877029419 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:10.000000000Z 64.14549350738525 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:20.000000000Z 64.15379047393799 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:30.000000000Z 64.1592264175415 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:40.000000000Z 64.18190002441406 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:50.000000000Z 64.28837776184082 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:00.000000000Z 64.29731845855713 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:10.000000000Z 64.36963081359863 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:20.000000000Z 64.37397003173828 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:30.000000000Z 64.44413661956787 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:40.000000000Z 64.42906856536865 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:50.000000000Z 64.44573402404785 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:00.000000000Z 64.48912620544434 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:10.000000000Z 64.49522972106934 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:20.000000000Z 64.48652744293213 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:30.000000000Z 64.49949741363525 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:40.000000000Z 64.4949197769165 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:50.000000000Z 64.49787616729736 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49816226959229 ``` [](#) ## Windowing data Use the [`window()` function](/flux/v0/stdlib/universe/window)to group your data based on time bounds. The most common parameter passed with the `window()` is `every` which defines the duration of time between windows. Other parameters are available, but for this example, window the base data set into one minute windows. ```js dataSet |> window(every: 1m) ``` > [!Note] > The `every` parameter supports all [valid duration units](/flux/v0/spec/types/#duration-types), > including **calendar months (`1mo`)** and **years (`1y`)**. Each window of time is output in its own table containing all records that fall within the window. ###### window() output tables ``` Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:00.000000000Z 71.11611366271973 2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:10.000000000Z 67.39630699157715 2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:20.000000000Z 64.16666507720947 2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:30.000000000Z 64.19951915740967 2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:40.000000000Z 64.2122745513916 2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:50:50.000000000Z 64.22209739685059 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:00.000000000Z 64.6336555480957 2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:10.000000000Z 64.16516304016113 2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:20.000000000Z 64.18349742889404 2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:30.000000000Z 64.20474052429199 2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:40.000000000Z 68.65062713623047 2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:51:50.000000000Z 67.20139980316162 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:00.000000000Z 70.9143877029419 2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:10.000000000Z 64.14549350738525 2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:20.000000000Z 64.15379047393799 2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:30.000000000Z 64.1592264175415 2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:40.000000000Z 64.18190002441406 2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:52:50.000000000Z 64.28837776184082 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:00.000000000Z 64.29731845855713 2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:10.000000000Z 64.36963081359863 2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:20.000000000Z 64.37397003173828 2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:30.000000000Z 64.44413661956787 2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:40.000000000Z 64.42906856536865 2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:53:50.000000000Z 64.44573402404785 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:00.000000000Z 64.48912620544434 2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:10.000000000Z 64.49522972106934 2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:20.000000000Z 64.48652744293213 2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:30.000000000Z 64.49949741363525 2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:40.000000000Z 64.4949197769165 2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:50.000000000Z 64.49787616729736 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:55:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49816226959229 ``` [](#) When visualized in the InfluxDB UI, each window table is displayed in a different color. ![Windowed data](/img/flux/simple-windowed-data.png) ## Aggregate data [Aggregate functions](/flux/v0/function-types#aggregates) take the values of all rows in a table and use them to perform an aggregate operation. The result is output as a new value in a single-row table. Since windowed data is split into separate tables, aggregate operations run against each table separately and output new tables containing only the aggregated value. For this example, use the [`mean()` function](/flux/v0/stdlib/universe/mean)to output the average of each window: ```js dataSet |> window(every: 1m) |> mean() ``` ###### mean() output tables ``` Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------------- 2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 65.88549613952637 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------------- 2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 65.50651391347249 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------------- 2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 65.30719598134358 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------------- 2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 64.39330975214641 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------------- 2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 64.49386278788249 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ---------------------------- 2018-11-03T17:55:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 64.49816226959229 ``` [](#) Because each data point is contained in its own table, when visualized, they appear as single, unconnected points. ![Aggregated windowed data](/img/flux/simple-windowed-aggregate-data.png) ### Recreate the time column **Notice the `_time` column is not in the [aggregated output tables](#mean-output-tables).**Because records in each table are aggregated together, their timestamps no longer apply and the column is removed from the group key and table. Also notice the `_start` and `_stop` columns still exist. These represent the lower and upper bounds of the time window. Many Flux functions rely on the `_time` column. To further process your data after an aggregate function, you need to re-add `_time`. Use the [`duplicate()` function](/flux/v0/stdlib/universe/duplicate) to duplicate either the `_start` or `_stop` column as a new `_time` column. ```js dataSet |> window(every: 1m) |> mean() |> duplicate(column: "_stop", as: "_time") ``` ###### duplicate() output tables ``` Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:50:00.000000000Z 2018-11-03T17:51:00.000000000Z used_percent mem 2018-11-03T17:51:00.000000000Z 65.88549613952637 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:51:00.000000000Z 2018-11-03T17:52:00.000000000Z used_percent mem 2018-11-03T17:52:00.000000000Z 65.50651391347249 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:52:00.000000000Z 2018-11-03T17:53:00.000000000Z used_percent mem 2018-11-03T17:53:00.000000000Z 65.30719598134358 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:53:00.000000000Z 2018-11-03T17:54:00.000000000Z used_percent mem 2018-11-03T17:54:00.000000000Z 64.39330975214641 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:54:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49386278788249 Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:55:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49816226959229 ``` [](#) ## “Unwindow” aggregate tables Keeping aggregate values in separate tables generally isn’t the format in which you want your data. Use the `window()` function to “unwindow” your data into a single infinite (`inf`) window. ```js dataSet |> window(every: 1m) |> mean() |> duplicate(column: "_stop", as: "_time") |> window(every: inf) ``` > [!Note] > Windowing requires a `_time` column which is why it’s necessary to[recreate the `_time` column](#recreate-the-time-column) after an aggregation. ###### Unwindowed output table ``` Table: keys: [_start, _stop, _field, _measurement] _start:time _stop:time _field:string _measurement:string _time:time _value:float ------------------------------ ------------------------------ ---------------------- ---------------------- ------------------------------ ---------------------------- 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:51:00.000000000Z 65.88549613952637 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:52:00.000000000Z 65.50651391347249 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:53:00.000000000Z 65.30719598134358 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:54:00.000000000Z 64.39330975214641 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49386278788249 2018-11-03T17:50:00.000000000Z 2018-11-03T17:55:00.000000000Z used_percent mem 2018-11-03T17:55:00.000000000Z 64.49816226959229 ``` With the aggregate values in a single table, data points in the visualization are connected. ![Unwindowed aggregate data](/img/flux/simple-unwindowed-data.png) ## Summing up You have now created a Flux query that windows and aggregates data. The data transformation process outlined in this guide should be used for all aggregation operations. Flux also provides the [`aggregateWindow()` function](/flux/v0/stdlib/universe/aggregatewindow)which performs all these separate functions for you. The following Flux query will return the same results: ###### aggregateWindow function ```js dataSet |> aggregateWindow(every: 1m, fn: mean) ``` --- # Enable Flux Source: https://docs.influxdata.com/influxdb/v1/flux/installation/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Flux is packaged with **InfluxDB v1.8+** and does not require any additional installation, however it is **disabled by default and needs to be enabled**. ## Enable Flux Enable Flux by setting the `flux-enabled` option to `true` under the `[http]` section of your `influxdb.conf`: ###### influxdb.conf ```toml # ... [http] # ... flux-enabled = true # ... ``` > > > The default location of your `influxdb.conf` depends on your operating system. > More information is available in the[Configuring InfluxDB](/influxdb/v1/administration/config/#using-the-configuration-file) guide. > > When InfluxDB starts, the Flux daemon starts as well and data can be queried using Flux. --- # InfluxDB guides Source: https://docs.influxdata.com/influxdb/v1/guides/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. * [Calculate percentages in a query](/influxdb/v1/guides/calculate_percentages/) * [Downsample and retain data](/influxdb/v1/guides/downsample_and_retain/) * [Hardware sizing guidelines](/influxdb/v1/guides/hardware_sizing/) * [Migrate from InfluxDB OSS to InfluxDB Enterprise](/influxdb/v1/guides/migrate-to-enterprise/) * [Query data with the InfluxDB API](/influxdb/v1/guides/query_data/) * [Write data with the InfluxDB API](/influxdb/v1/guides/write_data/) --- # Calculate percentages in a query Source: https://docs.influxdata.com/influxdb/v1/guides/calculate_percentages/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Use Flux or InfluxQL to calculate percentages in a query. [Flux](#flux)[InfluxQL](#influxql) [Flux](/flux/latest/) lets you perform simple math equations, for example, calculating a percentage. ## Calculate a percentage Learn how to calculate a percentage using the following examples: * [Basic calculations within a query](#basic-calculations-within-a-query) * [Calculate a percentage from two fields](#calculate-a-percentage-from-two-fields) * [Calculate a percentage using aggregate functions](#calculate-a-percentage-using-aggregate-functions) * [Calculate the percentage of total weight per apple variety](#calculate-the-percentage-of-total-weight-per-apple-variety) * [Calculate the aggregate percentage per variety](#calculate-the-percentage-of-total-weight-per-apple-variety) ## Basic calculations within a query When performing any math operation in a Flux query, you must complete the following steps: 1. Specify the [bucket](/influxdb/v2/query-data/get-started/#buckets) to query from and the time range to query. 2. Filter your data by measurements, fields, and other applicable criteria. 3. Align values in one row (required to perform math in Flux) by using one of the following functions: * To query **from multiple** data sources, use the [`join()` function](/flux/v0/stdlib/universe/join/). * To query **from the same** data source, use the [`pivot()` function](/flux/v0/stdlib/universe/pivot/). For examples using the `join()` function to calculate percentages and more examples of calculating percentages, see [Calculate percentages with Flux](/influxdb/v2/query-data/flux/calculate-percentages/). #### Data variable To shorten examples, we’ll store a basic Flux query in a `data` variable for reuse. Here’s how that looks in Flux: ```js // Query data from the past 15 minutes pivot fields into columns so each row // contains values for each field data = from(bucket:"your_db/your_retention_policy") |> range(start: -15m) |> filter(fn: (r) => r._measurement == "measurement_name" and r._field =~ /field[1-2]/) |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value") ``` Each row now contains the values necessary to perform a math operation. For example, to add two field keys, start with the `data` variable created above, and then use `map()` to re-map values in each row. ```js data |> map(fn: (r) => ({ r with _value: r.field1 + r.field2})) ``` > > > **Note:** Flux supports basic math operators such as `+`,`-`,`/`, `*`, and `()`. For example, to subtract `field2` from `field1`, change `+` to `-`. > > ## Calculate a percentage from two fields Use the `data` variable created above, and then use the [`map()` function](/flux/v0/stdlib/universe/map/) to divide one field by another, multiply by 100, and add a new `percent` field to store the percentage values in. ```js data |> map(fn: (r) => ({ _time: r._time, _measurement: r._measurement, _field: "percent", _value: field1 / field2 * 100.0 })) ``` > > > **Note:** In this example, `field1` and `field2` are float values, hence multiplied by 100.0. For integer values, multiply by 100 or use the `float()` function to cast integers to floats. > > ## Calculate a percentage using aggregate functions Use [`aggregateWindow()`](/flux/v0/stdlib/universe/aggregatewindow) to window data by time and perform an aggregate function on each window. ```js from(bucket:"/") |> range(start: -15m) |> filter(fn: (r) => r._measurement == "measurement_name" and r._field =~ /fieldkey[1-2]/) |> aggregateWindow(every: 1m, fn:sum) |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value") |> map(fn: (r) => ({ r with _value: r.field1 / r.field2 * 100.0 })) ``` ## Calculate the percentage of total weight per apple variety Use simulated apple stand data to track the weight of apples (by type) throughout a day. 1. [Download the sample data](https://gist.githubusercontent.com/sanderson/8f8aec94a60b2c31a61f44a37737bfea/raw/c29b239547fa2b8ee1690f7d456d31f5bd461386/apple_stand.txt) 2. Import the sample data: ```bash influx -import -path=path/to/apple_stand.txt -precision=ns -database=apple_stand ``` Use the following query to calculate the percentage of the total weight each variety accounts for at each given point in time. ```js from(bucket:"apple_stand/autogen") |> range(start: 2018-06-18T12:00:00Z, stop: 2018-06-19T04:35:00Z) |> filter(fn: (r) => r._measurement == "variety") |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value") |> map(fn: (r) => ({ r with granny_smith: r.granny_smith / r.total_weight * 100.0 , golden_delicious: r.golden_delicious / r.total_weight * 100.0 , fuji: r.fuji / r.total_weight * 100.0 , gala: r.gala / r.total_weight * 100.0 , braeburn: r.braeburn / r.total_weight * 100.0 ,})) ``` ## Calculate the average percentage of total weight per variety each hour With the apple stand data from the prior example, use the following query to calculate the average percentage of the total weight each variety accounts for per hour. ```js from(bucket:"apple_stand/autogen") |> range(start: 2018-06-18T00:00:00.00Z, stop: 2018-06-19T16:35:00.00Z) |> filter(fn: (r) => r._measurement == "variety") |> aggregateWindow(every:1h, fn: mean) |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value") |> map(fn: (r) => ({ r with granny_smith: r.granny_smith / r.total_weight * 100.0, golden_delicious: r.golden_delicious / r.total_weight * 100.0, fuji: r.fuji / r.total_weight * 100.0, gala: r.gala / r.total_weight * 100.0, braeburn: r.braeburn / r.total_weight * 100.0 })) ``` [InfluxQL](/influxdb/v1/query_language/) lets you perform simple math equations which makes calculating percentages using two fields in a measurement pretty simple. However there are some caveats of which you need to be aware. ## Basic calculations within a query `SELECT` statements support the use of basic math operators such as `+`,`-`,`/`, `*`, `()`, etc. ```sql -- Add two field keys SELECT field_key1 + field_key2 AS "field_key_sum" FROM "measurement_name" WHERE time < now() - 15m -- Subtract one field from another SELECT field_key1 - field_key2 AS "field_key_difference" FROM "measurement_name" WHERE time < now() - 15m -- Grouping and chaining mathematical calculations SELECT (field_key1 + field_key2) - (field_key3 + field_key4) AS "some_calculation" FROM "measurement_name" WHERE time < now() - 15m ``` ## Calculating a percentage in a query Using basic math functions, you can calculate a percentage by dividing one field value by another and multiplying the result by 100: ```sql SELECT (field_key1 / field_key2) * 100 AS "calculated_percentage" FROM "measurement_name" WHERE time < now() - 15m ``` ## Calculating a percentage using aggregate functions If using aggregate functions in your percentage calculation, all data must be referenced using aggregate functions.***You can’t mix aggregate and non-aggregate data.*** All Aggregate functions need a `GROUP BY time()` clause defining the time intervals in which data points are grouped and aggregated. ```sql SELECT (sum(field_key1) / sum(field_key2)) * 100 AS "calculated_percentage" FROM "measurement_name" WHERE time < now() - 15m GROUP BY time(1m) ``` ## Examples #### Sample data The following example uses simulated Apple Stand data that tracks the weight of baskets containing different varieties of apples throughout a day of business. 1. [Download the sample data](https://gist.githubusercontent.com/sanderson/8f8aec94a60b2c31a61f44a37737bfea/raw/c29b239547fa2b8ee1690f7d456d31f5bd461386/apple_stand.txt) 2. Import the sample data: ```bash influx -import -path=path/to/apple_stand.txt -precision=ns -database=apple_stand ``` ### Calculating percentage of total weight per apple variety The following query calculates the percentage of the total weight each variety accounts for at each given point in time. ```sql SELECT ("braeburn"/total_weight)*100, ("granny_smith"/total_weight)*100, ("golden_delicious"/total_weight)*100, ("fuji"/total_weight)*100, ("gala"/total_weight)*100 FROM "apple_stand"."autogen"."variety" ``` \\\* If visualized as a [stacked graph](/chronograf/v1/guides/visualization-types/#stacked-graph)in Chronograf, it would look like: ![Percentage of total per apple variety](/img/influxdb/1-5-calc-percentage-apple-variety.png) ### Calculating aggregate percentage per variety The following query calculates the average percentage of the total weight each variety accounts for per hour. ```sql SELECT (mean("braeburn")/mean(total_weight))*100, (mean("granny_smith")/mean(total_weight))*100, (mean("golden_delicious")/mean(total_weight))*100, (mean("fuji")/mean(total_weight))*100, (mean("gala")/mean(total_weight))*100 FROM "apple_stand"."autogen"."variety" WHERE time >= '2018-06-18T12:00:00Z' AND time <= '2018-06-19T04:35:00Z' GROUP BY time(1h) ``` ***Note the following about this query:*** * It uses aggregate functions (`mean()`) for pulling all data. * It includes a `GROUP BY time()` clause which aggregates data into 1 hour blocks. * It includes an explicitly limited time window. Without it, aggregate functions are very resource-intensive. If visualized as a [stacked graph](/chronograf/v1/guides/visualization-types/#stacked-graph)in Chronograf, it would look like: ![Hourly average percentage of total per apple variety](/img/influxdb/1-5-calc-percentage-hourly-apple-variety.png) --- # Downsample and retain data Source: https://docs.influxdata.com/influxdb/v1/guides/downsample_and_retain/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxDB can handle hundreds of thousands of data points per second. Working with that much data over a long period of time can create storage concerns. A natural solution is to downsample the data; keep the high precision raw data for only a limited time, and store the lower precision, summarized data longer. This guide describes how to automate the process of downsampling data and expiring old data using InfluxQL. To downsample and retain data using Flux and InfluxDB 2.0, see [Process data with InfluxDB tasks](/influxdb/v2/process-data/). ### Definitions * **Continuous query** (CQ) is an InfluxQL query that runs automatically and periodically within a database. CQs require a function in the `SELECT` clause and must include a `GROUP BY time()` clause. * **Retention policy** (RP) is the part of InfluxDB data structure that describes for how long InfluxDB keeps data. InfluxDB compares your local server’s timestamp to the timestamps on your data and deletes data older than the RP’s `DURATION`. A single database can have several RPs and RPs are unique per database. This guide doesn’t go into detail about the syntax for creating and managing CQs and RPs or tasks. If you’re new to these concepts, we recommend reviewing the following: * [CQ documentation](/influxdb/v1/query_language/continuous_queries/) and * [RP documentation](/influxdb/v1/query_language/manage-database/#retention-policy-management). ### Sample data This section uses fictional real-time data to track the number of food orders to a restaurant via phone and via website at ten second intervals. We store this data in a [database](/influxdb/v1/concepts/glossary/#database) or [bucket]() called `food_data`, in the [measurement](/influxdb/v1/concepts/glossary/#measurement) `orders`, and in the [fields](/influxdb/v1/concepts/glossary/#field) `phone` and `website`. Sample: ``` ## name: orders time phone website 2016-05-10T23:18:00Z 10 30 2016-05-10T23:18:10Z 12 39 2016-05-10T23:18:20Z 11 56 ``` ### Goal Assume that, in the long run, we’re only interested in the average number of orders by phone and by website at 30 minute intervals. In the next steps, we use RPs and CQs to: * Automatically aggregate the ten-second resolution data to 30-minute resolution data * Automatically delete the raw, ten-second resolution data that are older than two hours * Automatically delete the 30-minute resolution data that are older than 52 weeks ### Database preparation We perform the following steps before writing the data to the database`food_data`. We do this **before** inserting any data because CQs only run against recent data; that is, data with timestamps that are no older than `now()` minus the `FOR` clause of the CQ, or `now()` minus the `GROUP BY time()` interval if the CQ has no `FOR` clause. #### 1. Create the database ```sql CREATE DATABASE "food_data" ``` #### 2. Create a two-hour `DEFAULT` retention policy InfluxDB writes to the `DEFAULT` retention policy if we do not supply an explicit RP when writing a point to the database. We make the `DEFAULT` RP keep data for two hours, because we want InfluxDB to automatically write the incoming ten-second resolution data to that RP. Use the[`CREATE RETENTION POLICY`](/influxdb/v1/query_language/manage-database/#create-retention-policies-with-create-retention-policy)statement to create a `DEFAULT` RP: ```sql CREATE RETENTION POLICY "two_hours" ON "food_data" DURATION 2h REPLICATION 1 DEFAULT ``` That query creates an RP called `two_hours` that exists in the database`food_data`.`two_hours` keeps data for a `DURATION` of two hours (`2h`) and it’s the `DEFAULT`RP for the database `food_data`. The replication factor (`REPLICATION 1`) is a required parameter but must always be set to 1 for single node instances. > > > **Note:** When we created the `food_data` database in step 1, InfluxDB > automatically generated an RP named `autogen` and set it as the `DEFAULT`RP for the database. > The `autogen` RP has an infinite retention period. > With the query above, the RP `two_hours` replaces `autogen` as the `DEFAULT` RP > for the `food_data` database. > > #### 3. Create a 52-week retention policy Next we want to create another retention policy that keeps data for 52 weeks and is not the`DEFAULT` retention policy (RP) for the database. Ultimately, the 30-minute rollup data will be stored in this RP. Use the[`CREATE RETENTION POLICY`](/influxdb/v1/query_language/manage-database/#create-retention-policies-with-create-retention-policy)statement to create a non-`DEFAULT` retention policy: ```sql CREATE RETENTION POLICY "a_year" ON "food_data" DURATION 52w REPLICATION 1 ``` That query creates a retention policy (RP) called `a_year` that exists in the database`food_data`. The `a_year` setting keeps data for a `DURATION` of 52 weeks (`52w`). Leaving out the `DEFAULT` argument ensures that `a_year` is not the `DEFAULT`RP for the database `food_data`. That is, write and read operations against `food_data` that do not specify an RP will still go to the `two_hours` RP (the `DEFAULT` RP). #### 4. Create the continuous query Now that we’ve set up our RPs, we want to create a continuous query (CQ) that will automatically and periodically downsample the ten-second resolution data to the 30-minute resolution, and then store those results in a different measurement with a different retention policy. Use the[`CREATE CONTINUOUS QUERY`](/influxdb/v1/query_language/continuous_queries/)statement to generate a CQ: ```sql > CREATE CONTINUOUS QUERY "cq_30m" ON "food_data" BEGIN SELECT mean("website") AS "mean_website",mean("phone") AS "mean_phone" INTO "a_year"."downsampled_orders" FROM "orders" GROUP BY time(30m) END ``` That query creates a CQ called `cq_30m` in the database `food_data`.`cq_30m` tells InfluxDB to calculate the 30-minute average of the two fields`website` and `phone` in the measurement `orders` and in the `DEFAULT` RP`two_hours`. It also tells InfluxDB to write those results to the measurement`downsampled_orders` in the retention policy `a_year` with the field keys`mean_website` and `mean_phone`. InfluxDB will run this query every 30 minutes for the previous 30 minutes. > > > **Note:** Notice that we fully qualify (that is, we use the syntax`"".""`) the measurement in the `INTO`clause. > InfluxDB requires that syntax to write data to an RP other than the `DEFAULT`RP. > > ### Results With the new CQ and two new RPs, `food_data` is ready to start receiving data. After writing data to our database and letting things run for a bit, we see two measurements: `orders` and `downsampled_orders`. ``` > SELECT * FROM "orders" LIMIT 5 ## name: orders time phone website 2016-05-13T23:00:00Z 10 30 2016-05-13T23:00:10Z 12 39 2016-05-13T23:00:20Z 11 56 2016-05-13T23:00:30Z 8 34 2016-05-13T23:00:40Z 17 32 > SELECT * FROM "a_year"."downsampled_orders" LIMIT 5 ## name: downsampled_orders time mean_phone mean_website 2016-05-13T15:00:00Z 12 23 2016-05-13T15:30:00Z 13 32 2016-05-13T16:00:00Z 19 21 2016-05-13T16:30:00Z 3 26 2016-05-13T17:00:00Z 4 23 ``` The data in `orders` are the raw, ten-second resolution data that reside in the two-hour RP. The data in `downsampled_orders` are the aggregated, 30-minute resolution data that are subject to the 52-week RP. Notice that the first timestamps in `downsampled_orders` are older than the first timestamps in `orders`. This is because InfluxDB has already deleted data from `orders` with timestamps that are older than our local server’s timestamp minus two hours (assume we executed the `SELECT` queries at `2016-05-14T00:59:59Z`). InfluxDB will only start dropping data from `downsampled_orders` after 52 weeks. > > > **Notes:** > > * Notice that we fully qualify (that is, we use the syntax`"".""`) `downsampled_orders` in the second `SELECT` statement. We must specify the RP in that query to `SELECT`data that reside in an RP other than the `DEFAULT` RP. > * By default, InfluxDB checks to enforce an RP every 30 minutes. Between checks, `orders` may have data that are older than two hours. The rate at which InfluxDB checks to enforce an RP is a configurable setting, see[Database Configuration](/influxdb/v1/administration/config#check-interval). Using a combination of RPs and CQs, we’ve successfully set up our database to automatically keep the high precision raw data for a limited time, create lower precision data, and store that lower precision data for a longer period of time. Now that you have a general understanding of how these features can work together, check out the detailed documentation on [CQs](/influxdb/v1/query_language/continuous_queries/) and [RPs](/influxdb/v1/query_language/manage-database/#retention-policy-management)to see all that they can do for you. --- # Hardware sizing guidelines Source: https://docs.influxdata.com/influxdb/v1/guides/hardware_sizing/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Review configuration and hardware guidelines for InfluxDB OSS (open source) and InfluxDB Enterprise: * [Single node or cluster?](#single-node-or-cluster) * [Query guidelines](#query-guidelines) * [InfluxDB OSS guidelines](#influxdb-oss-guidelines) * [When do I need more RAM?](#when-do-i-need-more-ram) * [Recommended cluster configurations](#recommended-cluster-configurations) * [Storage: type, amount, and configuration](#storage-type-amount-and-configuration) > > > **Disclaimer:** Your numbers may vary from recommended guidelines. Guidelines provide estimated benchmarks for implementing the most performant system for your business. > > ## Single node or cluster? If you want a single node instance of InfluxDB that’s fully open source, requires fewer writes, queries, and unique series than listed above, and do **not require** redundancy, we recommend InfluxDB OSS. > > > **Note:** Without the redundancy of a cluster, writes and queries fail immediately when a server is unavailable. > > If your InfluxDB performance requires any of the following, a single node (InfluxDB OSS) may not support your needs: * more than 750,000 field writes per second * more than 100 moderate queries per second ([see Query guides](#query-guidelines)) * more than 10,000,000 [series cardinality](/influxdb/v1/concepts/glossary/#series-cardinality) We recommend InfluxDB Enterprise, which supports multiple data nodes (a cluster) across multiple server cores. InfluxDB Enterprise distributes multiple copies of your data across a cluster, providing high-availability and redundancy, so an unavailable node doesn’t significantly impact the cluster. For more information about sizing requirements for InfluxDB Enterprise, see [InfluxDB Enterprise hardware sizing guidelines](/enterprise_influxdb/v1/guides/hardware_sizing/). ## Query guidelines > > > Query complexity varies widely on system impact. Recommendations are based on **moderate** query loads. > > For **simple** or **complex** queries, we recommend testing and adjusting the suggested requirements as needed. Query complexity is defined by the following criteria: |Query complexity| Criteria | |----------------|-------------------------------------------------------------------------------------| | Simple | Have few or no functions and no regular expressions | | | Are bounded in time to a few minutes, hours, or 24 hours at most | | | Typically execute in a few milliseconds to a few dozen milliseconds | | Moderate | Have multiple functions and one or two regular expressions | | | May also have `GROUP BY` clauses or sample a time range of multiple weeks | | | Typically execute in a few hundred or a few thousand milliseconds | | Complex |Have multiple aggregation or transformation functions or multiple regular expressions| | | May sample a very large time range of months or years | | | Typically take multiple seconds to execute | ## InfluxDB OSS guidelines Run InfluxDB on locally attached solid state drives (SSDs). Other storage configurations have lower performance and may not be able to recover from small interruptions in normal processing. Estimated guidelines include writes per second, queries per second, and number of unique [series](/influxdb/v1/concepts/glossary/#series), CPU, RAM, and IOPS (input/output operations per second). |vCPU or CPU| RAM | IOPS |Writes per second|Queries\* per second|Unique series| |-----------|-------|--------|-----------------|--------------------|-------------| | 2-4 cores |2-4 GB | 500 | \< 5,000 | \< 5 | \< 100,000 | | 4-6 cores |8-32 GB|500-1000| \< 250,000 | \< 25 |\< 1,000,000 | | 8+ cores |32+ GB | 1000+ | \> 250,000 | \> 25 |\> 1,000,000 | * **Queries per second for moderate queries.** Queries vary widely in their impact on the system. For simple or complex queries, we recommend testing and adjusting the suggested requirements as needed. See [query guidelines](#query-guidelines) for details. ## Storage: type, amount, and configuration ### Storage volume and IOPS Consider the type of storage you need and the amount. InfluxDB is designed to run on solid state drives (SSDs) and memory-optimized cloud instances, for example, AWS EC2 R5 or R4 instances. InfluxDB isn’t tested on hard disk drives (HDDs). For best results, InfluxDB servers must have a minimum of 1000 IOPS on storage to ensure recovery and availability. We recommend at least 2000 IOPS for rapid recovery of cluster data nodes after downtime. See your cloud provider documentation for IOPS detail on your storage volumes. ### Bytes and compression Database names, [measurements](/influxdb/v1/concepts/glossary/#measurement), [tag keys](/influxdb/v1/concepts/glossary/#tag-key), [field keys](/influxdb/v1/concepts/glossary/#field-key), and [tag values](/influxdb/v1/concepts/glossary/#tag-value) are stored only once and always as strings. [Field values](/influxdb/v1/concepts/glossary/#field-value) and [timestamps](/influxdb/v1/concepts/glossary/#timestamp) are stored for every point. Non-string values require approximately three bytes. String values require variable space, determined by string compression. ### Separate `wal` and `data` directories When running InfluxDB in a production environment, store the `wal` directory and the `data` directory on separate storage devices. This optimization significantly reduces disk contention under heavy write load──an important consideration if the write load is highly variable. If the write load does not vary by more than 15%, the optimization is probably not necessary. | Query complexity | Criteria | | --- | --- | | Query complexity | Criteria | | Simple | Have few or no functions and no regular expressions | | | Are bounded in time to a few minutes, hours, or 24 hours at most | | | Typically execute in a few milliseconds to a few dozen milliseconds | | Moderate | Have multiple functions and one or two regular expressions | | | May also have GROUP BY clauses or sample a time range of multiple weeks | | | Typically execute in a few hundred or a few thousand milliseconds | | Complex | Have multiple aggregation or transformation functions or multiple regular expressions | | | May sample a very large time range of months or years | | | Typically take multiple seconds to execute | | vCPU or CPU | RAM | IOPS | Writes per second | Queries* per second | Unique series | | --- | --- | --- | --- | --- | --- | | vCPU or CPU | RAM | IOPS | Writes per second | Queries* per second | Unique series | | 2-4 cores | 2-4 GB | 500 | < 5,000 | < 5 | < 100,000 | | 4-6 cores | 8-32 GB | 500-1000 | < 250,000 | < 25 | < 1,000,000 | | 8+ cores | 32+ GB | 1000+ | > 250,000 | > 25 | > 1,000,000 | --- # Migrate from InfluxDB OSS to InfluxDB Enterprise Source: https://docs.influxdata.com/influxdb/v1/guides/migrate-to-enterprise/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. To migrate InfluxDB 1.13 to InfluxDB Enterprise, follow the instructions provided in[Migrate InfluxDB OSS instances to InfluxDB Enterprise clusters](/enterprise_influxdb/v1/guides/migration/)in the InfluxDB Enterprise documentation. --- # Query data with the InfluxDB API Source: https://docs.influxdata.com/influxdb/v1/guides/query_data/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. The InfluxDB API is the primary means for querying data in InfluxDB (see the [command line interface](/influxdb/v1/tools/shell/) and [client libraries](/influxdb/v1/tools/api_client_libraries/) for alternative ways to query the database). Query data with the InfluxDB API using [Flux](#query-data-with-flux) or [InfluxQL](#query-data-with-influxql). > > > **Note**: The following examples use `curl`, a command line tool that transfers data using URLs. Learn the basics of `curl` with the [HTTP Scripting Guide](https://curl.haxx.se/docs/httpscripting.html). > > ## Query data with Flux For Flux queries, the `/api/v2/query` endpoint accepts `POST` HTTP requests. Use the following HTTP headers: * `Accept: application/csv` * `Content-type: application/vnd.flux` If you have authentication enabled, provide your InfluxDB username and password with the `Authorization` header and `Token` schema. For example: `Authorization: Token username:password`. The following example queries Telegraf data using Flux: : ``` $ curl -XPOST localhost:8086/api/v2/query -sS \ -H 'Accept:application/csv' \ -H 'Content-type:application/vnd.flux' \ -d 'from(bucket:"telegraf") |> range(start:-5m) |> filter(fn:(r) => r._measurement == "cpu")' ``` Flux returns [annotated CSV](/influxdb/v2/reference/syntax/annotated-csv/): ``` {,result,table,_start,_stop,_time,_value,_field,_measurement,cpu,host ,_result,0,2020-04-07T18:02:54.924273Z,2020-04-07T19:02:54.924273Z,2020-04-07T18:08:19Z,4.152553004641827,usage_user,cpu,cpu-total,host1 ,_result,0,2020-04-07T18:02:54.924273Z,2020-04-07T19:02:54.924273Z,2020-04-07T18:08:29Z,7.608695652173913,usage_user,cpu,cpu-total,host1 ,_result,0,2020-04-07T18:02:54.924273Z,2020-04-07T19:02:54.924273Z,2020-04-07T18:08:39Z,2.9363988504310883,usage_user,cpu,cpu-total,host1 ,_result,0,2020-04-07T18:02:54.924273Z,2020-04-07T19:02:54.924273Z,2020-04-07T18:08:49Z,6.915093159934975,usage_user,cpu,cpu-total,host1} ``` The header row defines column labels for the table. The `cpu` [measurement](/influxdb/v1/concepts/glossary/#measurement) has four points, each represented by one of the record rows. For example the first point has a [timestamp](/influxdb/v1/concepts/glossary/#timestamp) of `2020-04-07T18:08:19`. ### Flux Check out the [Get started with Flux](/influxdb/v2/query-data/get-started/) to learn more about building queries with Flux. For more information about querying data with the InfluxDB API using Flux, see the [API reference documentation](/influxdb/v1/api/buckets-v2-compatible/). ## Query data with InfluxQL To perform an InfluxQL query, send a `GET` request to the `/query` endpoint, set the URL parameter `db` as the target database, and set the URL parameter `q` as your query. You can also use a `POST` request by sending the same parameters either as URL parameters or as part of the body with `application/x-www-form-urlencoded`. The example below uses the InfluxDB API to query the same database that you encountered in [Writing Data](/influxdb/v1/guides/writing_data/). ```bash curl -G 'http://localhost:8086/query?pretty=true' --data-urlencode "db=mydb" --data-urlencode "q=SELECT \"value\" FROM \"cpu_load_short\" WHERE \"region\"='us-west'" ``` InfluxDB returns JSON: ```json { "results": [ { "statement_id": 0, "series": [ { "name": "cpu_load_short", "columns": [ "time", "value" ], "values": [ [ "2015-01-29T21:55:43.702900257Z", 2 ], [ "2015-01-29T21:55:43.702900257Z", 0.55 ], [ "2015-06-11T20:46:02Z", 0.64 ] ] } ] } ] } ``` > > > **Note:** Appending `pretty=true` to the URL enables pretty-printed JSON output. > While this is useful for debugging or when querying directly with tools like `curl`, it is not recommended for production use as it consumes unnecessary network bandwidth. > > ### InfluxQL Check out the [Data Exploration page](/influxdb/v1/query_language/explore-data/) to get acquainted with InfluxQL. For more information about querying data with the InfluxDB API using InfluxQL, see the [API reference documentation](/influxdb/v1/api/). --- # Write data with the InfluxDB API Source: https://docs.influxdata.com/influxdb/v1/guides/write_data/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Write data into InfluxDB using the [command line interface](/influxdb/v1/tools/shell/), [client libraries](/influxdb/v1/clients/api/), and plugins for common data formats such as [Graphite](/influxdb/v1/write_protocols/graphite/). > > > **Note**: The following examples use `curl`, a command line tool that transfers data using URLs. Learn the basics of `curl` with the [HTTP Scripting Guide](https://curl.haxx.se/docs/httpscripting.html). > > ### Create a database using the InfluxDB API To create a database send a `POST` request to the `/query` endpoint and set the URL parameter `q` to `CREATE DATABASE `. The example below sends a request to InfluxDB running on `localhost` and creates the `mydb` database: ```bash curl -i -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE mydb" ``` ### Write data using the InfluxDB API The InfluxDB API is the primary means of writing data into InfluxDB. * To **write to a database using the InfluxDB v1 API**, send `POST` requests to the `/write` endpoint. The following example shows how to write a point to the `mydb` database. The point data consists of the [measurement](/influxdb/v1/concepts/glossary/#measurement) `cpu_load_short`, the [tag keys](/influxdb/v1/concepts/glossary/#tag-key) `host` and `region` with the [tag values](/influxdb/v1/concepts/glossary/#tag-value) `server01` and `us-west`, the [field key](/influxdb/v1/concepts/glossary/#field-key) `value` with a [field value](/influxdb/v1/concepts/glossary/#field-value) of `0.64`, and the [timestamp](/influxdb/v1/concepts/glossary/#timestamp) `1434055562000000000`. ```bash curl -i -XPOST 'http://localhost:8086/write?db=mydb' --data-binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000' ``` * To **write to a database using the InfluxDB 2.0 API (compatible with InfluxDB 1.8+)**, send `POST` requests to the [`/api/v2/write` endpoint](/influxdb/v1/api/write/): ```bash curl -i -XPOST 'http://localhost:8086/api/v2/write?bucket=db/rp&precision=ns' \ --header 'Authorization: Token username:password' \ --data-raw 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000' ``` When writing points, you must specify an existing database in the `db` query parameter. Points will be written to `db`’s default retention policy if you do not supply a retention policy via the `rp` query parameter. See the [InfluxDB API Reference](/influxdb/v1/api/write/) documentation for a complete list of the available query parameters. The body of the POST or [InfluxDB line protocol](/influxdb/v1/concepts/glossary/#influxdb-line-protocol) contains the time series data that you want to store. Data includes: * **Measurement (required)** * **Tags**: Strictly speaking, tags are optional but most series include tags to differentiate data sources and to make querying both easy and efficient. Both tag keys and tag values are strings. * **Fields (required)**: Field keys are required and are always strings, and, [by default](/influxdb/v1/write_protocols/line_protocol_reference/#data-types), field values are floats. * **Timestamp**: Supplied at the end of the line in Unix time in nanoseconds since January 1, 1970 UTC - is optional. If you do not specify a timestamp, InfluxDB uses the server’s local nanosecond timestamp in Unix epoch. Time in InfluxDB is in UTC format by default. > > > **Note:** Avoid using the following reserved keys: `_field`, `_measurement`, and `time`. If reserved keys are included as a tag or field key, the associated point is discarded. > > ### Configure gzip compression InfluxDB supports gzip compression. To reduce network traffic, consider the following options: * To accept compressed data from InfluxDB, add the `Accept-Encoding: gzip` header to InfluxDB API requests. * To compress data before sending it to InfluxDB, add the `Content-Encoding: gzip` header to InfluxDB API requests. For details about enabling gzip for client libraries, see your client library documentation. #### Enable gzip compression in the Telegraf InfluxDB output plugin * In the Telegraf configuration file (telegraf.conf), under [[outputs.influxdb]], change`content_encoding = "identity"` (default) to `content_encoding = "gzip"` > > > **Note**Writes to InfluxDB 2.x [[outputs.influxdb\_v2]] are configured to compress content in gzip format by default. > > ### Writing multiple points Post multiple points to multiple series at the same time by separating each point with a new line. Batching points in this manner results in much higher performance. The following example writes three points to the database `mydb`. The first point belongs to the series with the measurement `cpu_load_short` and tag set `host=server02` and has the server’s local timestamp. The second point belongs to the series with the measurement `cpu_load_short` and tag set `host=server02,region=us-west` and has the specified timestamp `1422568543702900257`. The third point has the same specified timestamp as the second point, but it is written to the series with the measurement `cpu_load_short` and tag set `direction=in,host=server01,region=us-west`. ```bash curl -i -XPOST 'http://localhost:8086/write?db=mydb' --data-binary 'cpu_load_short,host=server02 value=0.67 cpu_load_short,host=server02,region=us-west value=0.55 1422568543702900257 cpu_load_short,direction=in,host=server01,region=us-west value=2.0 1422568543702900257' ``` ### Writing points from a file Write points from a file by passing `@filename` to `curl`. The data in the file should follow the [InfluxDB line protocol syntax](/influxdb/v1/write_protocols/write_syntax/). Example of a properly-formatted file (`cpu_data.txt`): ```txt cpu_load_short,host=server02 value=0.67 cpu_load_short,host=server02,region=us-west value=0.55 1422568543702900257 cpu_load_short,direction=in,host=server01,region=us-west value=2.0 1422568543702900257 ``` Write the data in `cpu_data.txt` to the `mydb` database with: ```bash curl -i -XPOST 'http://localhost:8086/write?db=mydb' --data-binary @cpu_data.txt ``` > > > **Note:** If your data file has more than 5,000 points, it may be necessary to split that file into several files in order to write your data in batches to InfluxDB. > By default, the HTTP request times out after five seconds. > InfluxDB will still attempt to write the points after that time out but there will be no confirmation that they were successfully written. > > ### Schemaless Design InfluxDB is a schemaless database. You can add new measurements, tags, and fields at any time. Note that if you attempt to write data with a different type than previously used (for example, writing a string to a field that previously accepted integers), InfluxDB will reject those data. ### A note on REST InfluxDB uses HTTP solely as a convenient and widely supported data transfer protocol. Modern web APIs have settled on REST because it addresses a common need. As the number of endpoints grows the need for an organizing system becomes pressing. REST is the industry agreed style for organizing large numbers of endpoints. This consistency is good for those developing and consuming the API: everyone involved knows what to expect. REST, however, is a convention. InfluxDB makes do with three API endpoints. This simple, easy to understand system uses HTTP as a transfer method for [InfluxQL](/influxdb/v1/query_language/spec/). The InfluxDB API makes no attempt to be RESTful. ### HTTP response summary * 2xx: If your write request received `HTTP 204 No Content`, it was a success! * 4xx: InfluxDB could not understand the request. * 5xx: The system is overloaded or significantly impaired. #### Examples ##### Writing a float to a field that previously accepted booleans ```bash curl -i -XPOST 'http://localhost:8086/write?db=hamlet' --data-binary 'tobeornottobe booleanonly=true' curl -i -XPOST 'http://localhost:8086/write?db=hamlet' --data-binary 'tobeornottobe booleanonly=5' ``` returns: ```bash HTTP/1.1 400 Bad Request Content-Type: application/json Request-Id: [...] X-Influxdb-Version: 1.13.0 Date: Wed, 01 Mar 2017 19:38:01 GMT Content-Length: 150 {"error":"field type conflict: input field \"booleanonly\" on measurement \"tobeornottobe\" is type float, already exists as type boolean dropped=1"} ``` ##### Writing a point to a database that doesn’t exist ```bash curl -i -XPOST 'http://localhost:8086/write?db=atlantis' --data-binary 'liters value=10' ``` returns: ```bash HTTP/1.1 404 Not Found Content-Type: application/json Request-Id: [...] X-Influxdb-Version: 1.13.0 Date: Wed, 01 Mar 2017 19:38:35 GMT Content-Length: 45 {"error":"database not found: \"atlantis\""} ``` ### Next steps Now that you know how to write data with the InfluxDB API, discover how to query them with the [Querying data](/influxdb/v1/guides/querying_data/) guide! For more information about writing data with the InfluxDB API, please see the [InfluxDB API reference](/influxdb/v1/api/write/). --- # High availability with InfluxDB Source: https://docs.influxdata.com/influxdb/v1/high_availability/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxDB OSS 1.13 does **not** support clustering. For high availability or horizontal scaling of InfluxDB, use the commercial clustered offering, [InfluxDB Enterprise](/enterprise_influxdb/v1/). * For information about creating an InfluxDB Enterprise cluster, see[Install an InfluxDB Enterprise cluster](/enterprise_influxdb/v1/introduction/installation/). * To learn more about high availability clustering, see[Clustering in InfluxDB Enterprise](/enterprise_influxdb/v1/concepts/clustering/). #### Related * [Install an InfluxDB Enterprise v1 cluster](/enterprise_influxdb/v1/introduction/installation/) * [Clustering in InfluxDB Enterprise v1](/enterprise_influxdb/v1/concepts/clustering/) --- # Learn about InfluxDB OSS 1.x Source: https://docs.influxdata.com/influxdb/v1/introduction/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. To get up and running with InfluxDB OSS v1.13, complete the following: ## [Download InfluxDB OSS v1](/influxdb/v1/introduction/download/) Download the latest InfluxDB open source (OSS) release at the [InfluxData Downloads page](https://www.influxdata.com/downloads/). 1. Scroll to the bottom of the [downloads page](https://www.influxdata.com/downloads/) and click **2** to navigate to the second page of products. 2. Under **InfluxDB OSS 1.x**, select: * your **Platform** (operating system) and * the **Version** of InfluxDB you want to download. 3. Run the provided commands to download and install InfluxDB. ## [Get started with InfluxDB OSS v1](/influxdb/v1/introduction/get-started/) Get started with InfluxDB OSS v1.13. Learn how to create databases, write data, and query your time series data. ## [Install and run InfluxDB using Docker](/influxdb/v1/introduction/install/docker/) Install and run InfluxDB OSS v1.13 using Docker. Configure and operate InfluxDB in a Docker container. ## [Install InfluxDB OSS v1](/influxdb/v1/introduction/install/) Install, start, and configure InfluxDB OSS v1.13. --- # Download InfluxDB OSS v1 Source: https://docs.influxdata.com/influxdb/v1/introduction/download/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Download the latest InfluxDB open source (OSS) release at the [InfluxData Downloads page](https://www.influxdata.com/downloads/). 1. Scroll to the bottom of the [downloads page](https://www.influxdata.com/downloads/) and click **2** to navigate to the second page of products. 2. Under **InfluxDB OSS 1.x**, select: * your **Platform** (operating system) and * the **Version** of InfluxDB you want to download. 3. Run the provided commands to download and install InfluxDB. --- # Get started with InfluxDB OSS v1 Source: https://docs.influxdata.com/influxdb/v1/introduction/get-started/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. With InfluxDB [installed](/influxdb/v1/introduction/installation), you’re ready to start working with time series data. This guide uses the `influx` [command line interface](/influxdb/v1/tools/shell/) (CLI), which is included with InfluxDB and provides direct access to the database. The CLI communicates with InfluxDB through the HTTP API on port `8086`. > [!Tip] > **Docker users**: Access the CLI from your container using: > > ``` > docker exec -it influx > ``` > [!Note] > #### Directly access the API > > You can also interact with InfluxDB using the HTTP API directly. > See [Writing Data](/influxdb/v1/guides/writing_data/) and [Querying Data](/influxdb/v1/guides/querying_data/) for examples using `curl`. ## Creating a database After installing InfluxDB locally, the `influx` command is available from your terminal. Running `influx` starts the CLI and connects to your local InfluxDB instance (ensure InfluxDB is running with `service influxdb start` or `influxd`). To start the CLI and connect to the local InfluxDB instance, run the following command. The [`-precision` argument](/influxdb/v1/tools/shell/#influx-arguments) specifies the format and precision of any returned timestamps. ```bash $ influx -precision rfc3339 Connected to http://localhost:8086 version 1.13.0 InfluxDB shell 1.13.0 > ``` The `influx` CLI connects to port `localhost:8086` (the default). The timestamp precision `rfc3339` tells InfluxDB to return timestamps in [RFC3339 format](https://www.ietf.org/rfc/rfc3339.txt) (`YYYY-MM-DDTHH:MM:SS.nnnnnnnnnZ`). To view available options for customizing CLI connection parameters or other settings, run `influx --help` in your terminal. The command line is ready to take input in the form of the Influx Query Language (InfluxQL) statements. To exit the InfluxQL shell, type `exit` and hit return. A fresh install of InfluxDB has no databases (apart from the system `_internal`), so creating one is our first task. You can create a database with the `CREATE DATABASE ` InfluxQL statement, where `` is the name of the database you wish to create. Names of databases can contain any unicode character as long as the string is double-quoted. Names can also be left unquoted if they contain *only* ASCII letters, digits, or underscores and do not begin with a digit. Throughout this guide, we’ll use the database name `mydb`: ```sql CREATE DATABASE mydb ``` > > > **Note:** After hitting enter, a new prompt appears and nothing else is displayed. > In the CLI, this means the statement was executed and there were no errors to display. > There will always be an error displayed if something went wrong. > > Now that the `mydb` database is created, we’ll use the `SHOW DATABASES` statement to display all existing databases: ``` > SHOW DATABASES name: databases ## name _internal mydb > ``` > > > **Note:** The `_internal` database is created and used by InfluxDB to store internal runtime metrics. > Check it out later to get an interesting look at how InfluxDB is performing under the hood. > > Unlike `SHOW DATABASES`, most InfluxQL statements must operate against a specific database. You may explicitly name the database with each query, but the CLI provides a convenience statement, `USE `, which will automatically set the database for all future requests. For example: ```sql > USE mydb Using database mydb > ``` Now future commands will only be run against the `mydb` database. ## Writing and exploring data Now that we have a database, InfluxDB is ready to accept queries and writes. First, a short primer on the datastore. Data in InfluxDB is organized by “time series”, which contain a measured value, like “cpu\_load” or “temperature”. Time series have zero to many `points`, one for each discrete sample of the metric. Points consist of `time` (a timestamp), a `measurement` (“cpu\_load”, for example), at least one key-value `field` (the measured value itself, e.g. “value=0.64”, or “temperature=21.2”), and zero to many key-value `tags` containing any metadata about the value (e.g. “host=server01”, “region=EMEA”, “dc=Frankfurt”). Conceptually you can think of a `measurement` as an SQL table, where the primary index is always time.`tags` and `fields` are effectively columns in the table.`tags` are indexed, and `fields` are not. The difference is that, with InfluxDB, you can have millions of measurements, you don’t have to define schemas up-front, and null values aren’t stored. Points are written to InfluxDB using the InfluxDB line protocol, which follows the following format: ``` [,=...] =[,=...] [unix-nano-timestamp] ``` The following lines are all examples of points that can be written to InfluxDB: ``` cpu,host=serverA,region=us_west value=0.64 payment,device=mobile,product=Notepad,method=credit billed=33,licenses=3i 1434067467100293230 stock,symbol=AAPL bid=127.46,ask=127.48 temperature,machine=unit42,type=assembly external=25,internal=37 1434067467000000000 ``` > > > **Note:** For details on the InfluxDB line protocol, see [InfluxDB line protocol syntax](/influxdb/v1/write_protocols/line_protocol_reference/#line-protocol-syntax) page. > > To insert a single time series data point into InfluxDB using the CLI, enter `INSERT` followed by a point: ```sql INSERT cpu,host=serverA,region=us_west value=0.64 ``` A point with the measurement name of `cpu` and tags `host` and `region` has now been written to the database, with the measured `value` of `0.64`. Now we will query for the data we just wrote: ``` > SELECT "host", "region", "value" FROM "cpu" ## name: cpu time host region value 2015-10-21T19:28:07.580664347Z serverA us_west 0.64 > ``` > > > **Note:** We did not supply a timestamp when writing our point. > When no timestamp is supplied for a point, InfluxDB assigns the local current timestamp when the point is ingested. > That means your timestamp will be different. > > Let’s try storing another type of data, with two fields in the same measurement: ```sql INSERT temperature,machine=unit42,type=assembly external=25,internal=37 ``` To return all fields and tags with a query, you can use the `*` operator: ``` > SELECT * FROM "temperature" ## name: temperature time external internal machine type 2015-10-21T19:28:08.385013942Z 25 37 unit42 assembly > ``` **Warning:** Using `*` without a `LIMIT` clause on a large database can cause performance issues. You can use `Ctrl+C` to cancel a query that is taking too long to respond. InfluxQL has many [features and keywords](/influxdb/v1/query_language/spec/) that are not covered here, including support for Go-style regex. For example: ```sql > SELECT * FROM /.*/ LIMIT 1 -- > SELECT * FROM "cpu_load_short" -- > SELECT * FROM "cpu_load_short" WHERE "value" > 0.9 ``` ## Using the HTTP API You can also interact with InfluxDB using HTTP requests with tools like `curl`: ### Create a database ```bash curl -G http://localhost:8086/query --data-urlencode "q=CREATE DATABASE mydb" ``` ### Write data ```bash curl -i -XPOST 'http://localhost:8086/write?db=mydb' \ --data-binary 'cpu,host=serverA,region=us_west value=0.64' ``` ### Query data ```bash curl -G 'http://localhost:8086/query?pretty=true' \ --data-urlencode "db=mydb" \ --data-urlencode "q=SELECT * FROM cpu" ``` ## Next steps This is all you need to know to write data into InfluxDB and query it back. To learn more about the InfluxDB write protocol, check out the guide on [Writing Data](/influxdb/v1/guides/writing_data/). To further explore the query language, check out the guide on [Querying Data](/influxdb/v1/guides/querying_data/). For more information on InfluxDB concepts, check out the [Key Concepts](/influxdb/v1/concepts/key_concepts/) page. --- # Install InfluxDB OSS v1 Source: https://docs.influxdata.com/influxdb/v1/introduction/install/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Install, start, and configure InfluxDB open source (OSS) v1.13. ## InfluxDB OSS installation requirements Installation of the InfluxDB package may require `root` or administrator privileges in order to complete successfully. ### InfluxDB OSS networking ports By default, InfluxDB uses the following network ports: * TCP port `8086` is available for client-server communication using the InfluxDB API. * TCP port `8088` is available for the RPC service to perform back up and restore operations. * TCP port `2003` is available for the Graphite protocol (when enabled). In addition to the ports above, InfluxDB also offers multiple plugins that may require [custom ports](/influxdb/v1/administration/ports/). All port mappings can be modified through the [configuration file](/influxdb/v1/administration/config), which is located at `/etc/influxdb/influxdb.conf` for default installations. ### Network Time Protocol (NTP) InfluxDB uses a host’s local time in UTC to assign timestamps to data and for coordination purposes. Use the Network Time Protocol (NTP) to synchronize time between hosts; if hosts' clocks aren’t synchronized with NTP, the timestamps on the data written to InfluxDB can be inaccurate. ## Installing InfluxDB OSS For users who don’t want to install any software and are ready to use InfluxDB, you may want to check out our[managed hosted InfluxDB offering](https://cloud.influxdata.com). [Ubuntu & Debian](#ubuntu--debian)[Red Hat & CentOS](#red-hat--centos)[SLES & openSUSE](#sles--opensuse)[FreeBSD/PC-BSD](#freebsdpc-bsd)[macOS](#macos)[Docker](#docker) For instructions on how to install the Debian package from a file, see the[downloads page](https://influxdata.com/downloads/). Debian and Ubuntu users can install the latest stable version of InfluxDB using the`apt-get` package manager. For Ubuntu/Debian users, add the InfluxData repository with the following commands: [wget](#wget)[curl](#curl) ```sh # influxdata-archive.key GPG fingerprint: # Primary key fingerprint: 24C9 75CB A61A 024E E1B6 3178 7C3D 5715 9FC2 F927 # Subkey fingerprint: 9D53 9D90 D332 8DC7 D6C8 D3B9 D8FF 8E1F 7DF8 B07E wget -q https://repos.influxdata.com/influxdata-archive.key gpg --show-keys --with-fingerprint --with-colons ./influxdata-archive.key 2>&1 | grep -q '^fpr::*24C975CBA61A024EE1B631787C3D57159FC2F927:$' && cat influxdata-archive.key | gpg --dearmor | sudo tee /etc/apt/keyrings/influxdata-archive.gpg > /dev/null echo 'deb [signed-by=/etc/apt/keyrings/influxdata-archive.gpg] https://repos.influxdata.com/debian stable main' | sudo tee /etc/apt/sources.list.d/influxdata.list ``` ```sh # influxdata-archive.key GPG fingerprint: # Primary key fingerprint: 24C9 75CB A61A 024E E1B6 3178 7C3D 5715 9FC2 F927 # Subkey fingerprint: 9D53 9D90 D332 8DC7 D6C8 D3B9 D8FF 8E1F 7DF8 B07E curl --silent --location -O https://repos.influxdata.com/influxdata-archive.key gpg --show-keys --with-fingerprint --with-colons ./influxdata-archive.key 2>&1 | grep -q '^fpr::*24C975CBA61A024EE1B631787C3D57159FC2F927:$' && cat influxdata-archive.key | gpg --dearmor | sudo tee /etc/apt/keyrings/influxdata-archive.gpg > /dev/null echo 'deb [signed-by=/etc/apt/keyrings/influxdata-archive.gpg] https://repos.influxdata.com/debian stable main' | sudo tee /etc/apt/sources.list.d/influxdata.list ``` Then, install and start the InfluxDB service: ```bash sudo apt-get update && sudo apt-get install influxdb sudo service influxdb start ``` Or if your operating system is using systemd (Ubuntu 15.04+, Debian 8+): ```bash sudo apt-get update && sudo apt-get install influxdb sudo systemctl unmask influxdb.service sudo systemctl start influxdb ``` For instructions on how to install the RPM package from a file, please see the [downloads page](https://influxdata.com/downloads/). Red Hat and CentOS users can install the latest stable version of InfluxDB using the `yum` package manager: ```bash cat < [!Note] > ##### Multiple versions of InfluxDB with Homebrew > > Installing both InfluxDB 1.8 and InfluxDB 2.0 with Homebrew > can result in unexpected path and naming conflicts. > You can always run the desired version by specifying the full path: > > ``` > $ /usr/local/opt/influxdb/bin/influxd version > InfluxDB 1.13.0 (git: none) build_date: 2021-04-01T17:55:08Z > $ /usr/local/opt/influxdb@1/bin/influxd version > InfluxDB v (git: unknown unknown) > ``` Use Docker to run InfluxDB v1 in a container. For comprehensive Docker installation instructions, configuration options, and initialization features, see: **[Install and run with Docker ›](/influxdb/v1/introduction/install/docker/)** Quick start: ```bash # Pull the latest InfluxDB v1.x image docker pull influxdb:1.13.0 # Start InfluxDB with persistent storage docker run -p 8086:8086 \ -v $PWD/data:/var/lib/influxdb \ influxdb:1.13.0 ``` ### Verify the authenticity of downloaded binary (optional) For added security, follow these steps to verify the signature of your InfluxDB download with `gpg`. (Most operating systems include the `gpg` command by default. If `gpg` is not available, see the [GnuPG homepage](https://gnupg.org/download/) for installation instructions.) 1. Download and import InfluxData’s public key: ``` curl -s https://repos.influxdata.com/influxdata-archive.key | gpg --import ``` 2. Download the signature file for the release by adding `.asc` to the download URL. For example: ``` wget https://download.influxdata.com/influxdb/releases/influxdb-1.13.0-linux-amd64.tar.gz.asc ``` 3. Verify the signature with `gpg --verify`: ``` gpg --verify influxdb-1.13.0_linux_amd64.tar.gz.asc influxdb-1.13.0-linux-amd64.tar.gz ``` The output from this command should include the following: ``` gpg: Good signature from "InfluxData Package Signing Key " [unknown] ``` ## Configuring InfluxDB OSS The system has internal defaults for every configuration file setting. View the default configuration settings with the `influxd config` command. > [!Note] > **Note:** If InfluxDB is being deployed on a publicly accessible endpoint, we strongly recommend authentication be enabled. > Otherwise the data will be publicly available to any unauthenticated user. The default settings do **NOT** enable > authentication and authorization. Further, authentication and authorization should not be solely relied upon to prevent access > and protect data from malicious actors. If additional security or compliance features are desired, InfluxDB should be run > behind a third-party service. Review the [authentication and authorization](/influxdb/v1/administration/authentication_and_authorization/)settings. Most of the settings in the local configuration file (`/etc/influxdb/influxdb.conf`) are commented out; all commented-out settings will be determined by the internal defaults. Any uncommented settings in the local configuration file override the internal defaults. Note that the local configuration file does not need to include every configuration setting. Use one of the following methods to launch InfluxDB OSS v1 with your configuration file: * **CLI flag**: Pass the `-config` flag with the path to your configuration file: ``` influxd -config /etc/influxdb/influxdb.conf ``` * **Environment variable**: Set the `INFLUXDB_CONFIG_PATH` environment variable to the path of your configuration file and start the process–for example: [Linux/macOS (bash/zsh)](#linuxmacos-bashzsh)[Linux/macOS (persistent)](#linuxmacos-persistent)[Windows (PowerShell)](#windows-powershell)[Windows (CMD)](#windows-cmd) ``` # Set the environment variable for the current session export INFLUXDB_CONFIG_PATH=/etc/influxdb/influxdb.conf # Start InfluxDB influxd ``` ``` # Add to ~/.bashrc or ~/.zshrc for persistence echo 'export INFLUXDB_CONFIG_PATH=/etc/influxdb/influxdb.conf' >> ~/.bashrc source ~/.bashrc # Start InfluxDB influxd ``` ``` # Set the environment variable for the current session $env:INFLUXDB_CONFIG_PATH = "C:\Program Files\InfluxDB\influxdb.conf" # Start InfluxDB influxd ``` ``` REM Set the environment variable for the current session set INFLUXDB_CONFIG_PATH=C:\Program Files\InfluxDB\influxdb.conf REM Start InfluxDB influxd ``` InfluxDB first checks for the `-config` option and then for the environment variable. ### Configuring InfluxDB with Docker For detailed Docker configuration instructions including environment variables, configuration files, initialization options, and examples, see: **[Install and run with Docker ›](/influxdb/v1/introduction/install/docker/)** See the [Configuration](/influxdb/v1/administration/config/) documentation for more information. ### Data and WAL directory permissions Make sure the directories in which data and the [write ahead log](/influxdb/v1/concepts/glossary#wal-write-ahead-log) (WAL) are stored are writable for the user running the `influxd` service. > [!Important] > If the data and WAL directories are not writable, the `influxd` service will not start. The user running the `influxd` process should have the following permissions for directories in the [InfluxDB file system](/influxdb/v1//concepts/file-system-layout/): | Directory path |Permission| |----------------|----------| | `influxdb/` | 755 | |`influxdb/data/`| 755 | |`influxdb/meta/`| 755 | |`influxdb/wal/` | 700 | Information about `data` and `wal` directory paths is available in the [Data settings](/influxdb/v1/administration/config/#data-settings) section of the [Configuring InfluxDB](/influxdb/v1/administration/config/) documentation. ## Hosting InfluxDB OSS on AWS ### Hardware requirements for InfluxDB We recommend using two SSD volumes, using one for the `influxdb/wal` and the other for the `influxdb/data`. Depending on your load, each volume should have around 1k-3k provisioned IOPS. The `influxdb/data` volume should have more disk space with lower IOPS and the `influxdb/wal` volume should have less disk space with higher IOPS. Each machine should have a minimum of 8GB RAM. We’ve seen the best performance with the R4 class of machines, as they provide more memory than either of the C3/C4 class and the M4 class. ### Configuring InfluxDB OSS instances This example assumes that you are using two SSD volumes and that you have mounted them appropriately. This example also assumes that each of those volumes is mounted at `/mnt/influx` and `/mnt/db`. For more information on how to do that see the Amazon documentation on how to [Add a Volume to Your Instance](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-attaching-volume.html). ### Configuration file You’ll have to update the configuration file appropriately for each InfluxDB instance you have. ```toml ... [meta] dir = "/mnt/db/meta" ... ... [data] dir = "/mnt/db/data" ... wal-dir = "/mnt/influx/wal" ... ... [hinted-handoff] ... dir = "/mnt/db/hh" ... ``` ### Authentication and Authorization For all AWS deployments, we strongly recommend authentication be enabled. Without this, it is possible that your InfluxDB instance may be publicly available to any unauthenticated user. The default settings do **NOT** enable authentication and authorization. Further, authentication and authorization should not be solely relied upon to prevent access and protect data from malicious actors. If additional security or compliance features are desired, InfluxDB should be run behind additional services offered by AWS. Review the [authentication and authorization](/influxdb/v1/administration/authentication_and_authorization/) settings. ### InfluxDB OSS permissions When using non-standard directories for InfluxDB data and configurations, also be sure to set filesystem permissions correctly: ```bash chown influxdb:influxdb /mnt/influx chown influxdb:influxdb /mnt/db ``` For InfluxDB 1.7.6 or later, you must give owner permissions to the `init.sh` file. To do this, run the following script in your `influxdb` directory: ```sh if [ ! -f "$STDOUT" ]; then mkdir -p $(dirname $STDOUT) chown $USER:$GROUP $(dirname $STDOUT) fi if [ ! -f "$STDERR" ]; then mkdir -p $(dirname $STDERR) chown $USER:$GROUP $(dirname $STDERR) fi # Override init script variables with DEFAULT values ``` | Directory path | Permission | | --- | --- | | Directory path | Permission | | influxdb/ | 755 | | influxdb/data/ | 755 | | influxdb/meta/ | 755 | | influxdb/wal/ | 700 | --- # Install and run InfluxDB using Docker Source: https://docs.influxdata.com/influxdb/v1/introduction/install/docker/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Install and run InfluxDB OSS v1.13 using Docker containers. This guide covers Docker installation, configuration, and initialization options. * [Install and run InfluxDB](#install-and-run-influxdb) * [Pull the InfluxDB v1.x image](#pull-the-influxdb-v1x-image) * [Start InfluxDB](#start-influxdb) * [Configure InfluxDB](#configure-influxdb) * [Using environment variables](#using-environment-variables) * [Using a configuration file](#using-a-configuration-file) * [Initialize InfluxDB](#initialize-influxdb) * [Automatic initialization (for development)](#automatic-initialization-for-development) * [Custom initialization scripts](#custom-initialization-scripts) * [Access the InfluxDB CLI](#access-the-influxdb-cli) * [Next steps](#next-steps) ## Install and run InfluxDB ### Pull the InfluxDB v1.x image ```bash docker pull influxdb:1.13.0 ``` ### Volume permissions > [!Important] > The InfluxDB Docker container runs as user influxdb with UID/GID 1500. > When mounting volumes for persistent storage, ensure the mounted directory on the host is owned by UID/GID 1500, or InfluxDB may not have permission to write data. Set the correct ownership before starting the container: #### Create the data directory mkdir -p data #### Set ownership to UID/GID 1500 sudo chown -R 1500:1500 data ### Start InfluxDB > [!Important] > On MacOS due to issues related to docker desktop’s VM and local binds > you must use either a *delegated local directory* or a *named volume* when starting a container. [Linux/Windows](#linuxwindows)[MacOS (Docker Desktop)](#macos-docker-desktop) Start a basic InfluxDB container with persistent storage: ```bash docker run -p 8086:8086 \ -v $PWD/data:/var/lib/influxdb \ influxdb:1.13.0 ``` On MacOS, you must use either a *delegated local directory* or a *named volume* when starting an InfluxDB container. **Option A: Use a delegated local directory** ```bash docker run -p 8086:8086 \ -v $PWD/data-test2:/var/lib/influxdb:delegated \ influxdb:1.13.0 ``` **Option B: Use a named docker volume (still local, but managed by Docker Desktop)** ```bash docker run -d --name influxdb \ -p 8086:8086 \ -v influxdb-data:/var/lib/influxdb \ influxdb:1.13.0 ``` InfluxDB is now running and available at [http://localhost:8086](http://localhost:8086). ## Configure InfluxDB ### Using environment variables Configure InfluxDB settings using environment variables: ```bash docker run -p 8086:8086 \ -v $PWD/data:/var/lib/influxdb \ -e INFLUXDB_REPORTING_DISABLED=true \ -e INFLUXDB_HTTP_AUTH_ENABLED=true \ -e INFLUXDB_HTTP_LOG_ENABLED=true \ influxdb:1.13.0 ``` ### Using a configuration file Generate a default configuration file: ```bash docker run --rm influxdb:1.13.0 influxd config > influxdb.conf ``` Start InfluxDB with your custom configuration: ```bash docker run -p 8086:8086 \ -v $PWD/influxdb.conf:/etc/influxdb/influxdb.conf:ro \ -v $PWD/data:/var/lib/influxdb \ influxdb:1.13.0 ``` ## Initialize InfluxDB ### Automatic initialization (for development) > [!Warning] > Automatic initialization with InfluxDB v1 is not recommended for production. > Use this approach only for development and testing. Automatically create a database and admin user on first startup: ```bash docker run -p 8086:8086 \ -v $PWD/data:/var/lib/influxdb \ -e INFLUXDB_DB=mydb \ -e INFLUXDB_HTTP_AUTH_ENABLED=true \ -e INFLUXDB_ADMIN_USER=admin \ -e INFLUXDB_ADMIN_PASSWORD=supersecretpassword \ influxdb:1.13.0 ``` Environment variables for user creation: * `INFLUXDB_USER`: Create a user with no privileges * `INFLUXDB_USER_PASSWORD`: Password for the user * `INFLUXDB_READ_USER`: Create a user who can read from `INFLUXDB_DB` * `INFLUXDB_READ_USER_PASSWORD`: Password for the read user * `INFLUXDB_WRITE_USER`: Create a user who can write to `INFLUXDB_DB` * `INFLUXDB_WRITE_USER_PASSWORD`: Password for the write user ### Custom initialization scripts InfluxDB v1.x Docker containers support custom initialization scripts for testing scenarios: Create an initialization script (`init-scripts/setup.iql`): ```sql CREATE DATABASE sensors; CREATE DATABASE logs; CREATE USER "telegraf" WITH PASSWORD 'secret123'; GRANT WRITE ON "sensors" TO "telegraf"; CREATE USER "grafana" WITH PASSWORD 'secret456'; GRANT READ ON "sensors" TO "grafana"; GRANT READ ON "logs" TO "grafana"; CREATE RETENTION POLICY "one_week" ON "sensors" DURATION 1w REPLICATION 1 DEFAULT; ``` Run with initialization scripts: ```bash docker run -p 8086:8086 \ -v $PWD/data:/var/lib/influxdb \ -v $PWD/init-scripts:/docker-entrypoint-initdb.d \ influxdb:1.13.0 ``` Supported script types: * Shell scripts (`.sh`) * InfluxDB query language files (`.iql`) > [!Important] > Initialization scripts only run on first startup when the data directory is empty. > Scripts execute in alphabetical order based on filename. ## Access the InfluxDB CLI To access the InfluxDB command line interface from within the Docker container: ```bash docker exec -it influx ``` Replace `` with your InfluxDB container name or ID. ## Next steps Once you have InfluxDB running in Docker, see the [Get started guide](/influxdb/v1/introduction/get-started/) to: * Create databases * Write and query data * Learn InfluxQL basics #### Related * [Install InfluxDB OSS v1](/influxdb/v1/introduction/install/) * [Get started with InfluxDB OSS v1](/influxdb/v1/introduction/get-started/) * [Authentication and authorization in InfluxDB OSS v1](/influxdb/v1/administration/authentication_and_authorization/) * [Write data to InfluxDB OSS v1](/influxdb/v1/guides/write_data/) * [Query data in InfluxDB OSS v1](/influxdb/v1/guides/query_data/) * [Configure InfluxDB OSS v1](/influxdb/v1/administration/config/) --- # Influx Query Language (InfluxQL) Source: https://docs.influxdata.com/influxdb/v1/query_language/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. This section introduces InfluxQL, the InfluxDB SQL-like query language for working with data in InfluxDB databases. ## InfluxQL tutorial The first seven documents in this section provide a tutorial-style introduction to InfluxQL. Feel free to download the dataset provided in[Sample Data](/influxdb/v1/query_language/data_download/) and follow along with the documentation. #### Data exploration [Data exploration](/influxdb/v1/query_language/explore-data/) covers the query language basics for InfluxQL, including the[`SELECT` statement](/influxdb/v1/query_language/explore-data/#the-basic-select-statement),[`GROUP BY` clauses](/influxdb/v1/query_language/explore-data/#the-group-by-clause),[`INTO` clauses](/influxdb/v1/query_language/explore-data/#the-into-clause), and more. See Data Exploration to learn about[time syntax](/influxdb/v1/query_language/explore-data/#time-syntax) and[regular expressions](/influxdb/v1/query_language/explore-data/#regular-expressions) in queries. #### Schema exploration [Schema exploration](/influxdb/v1/query_language/explore-schema/) covers queries that are useful for viewing and exploring your[schema](/influxdb/v1/concepts/glossary/#schema). See Schema Exploration for syntax explanations and examples of InfluxQL’s `SHOW`queries. #### Database management [Database management](/influxdb/v1/query_language/manage-database/) covers InfluxQL for managing[databases](/influxdb/v1/concepts/glossary/#database) and[retention policies](/influxdb/v1/concepts/glossary/#retention-policy-rp) in InfluxDB. See Database Management for creating and dropping databases and retention policies as well as deleting and dropping data. #### InfluxQL functions Covers all [InfluxQL functions](/influxdb/v1/query_language/functions/). #### InfluxQL Continuous Queries [InfluxQL Continuous Queries](/influxdb/v1/query_language/continuous_queries/) covers the[basic syntax](/influxdb/v1/query_language/continuous_queries/#basic-syntax),[advanced syntax](/influxdb/v1/query_language/continuous_queries/#advanced-syntax), and[common use cases](/influxdb/v1/query_language/continuous_queries/#continuous-query-use-cases)for[Continuous Queries](/influxdb/v1/concepts/glossary/#continuous-query-cq). This page also describes how to[`SHOW`](/influxdb/v1/query_language/continuous_queries/#listing-continuous-queries) and[`DROP`](/influxdb/v1/query_language/continuous_queries/#deleting-continuous-queries)Continuous Queries. #### InfluxQL mathematical operators [InfluxQL mathematical operators](/influxdb/v1/query_language/math_operators/)covers the use of mathematical operators in InfluxQL. #### Authentication and authorization [Authentication and authorization](/influxdb/v1/administration/authentication_and_authorization/) covers how to[set up authentication](/influxdb/v1/administration/authentication_and_authorization/#set-up-authentication)and how to[authenticate requests](/influxdb/v1/administration/authentication_and_authorization/#authenticate-requests) in InfluxDB. This page also describes the different[user types](/influxdb/v1/administration/authentication_and_authorization/#user-types-and-privileges) and the InfluxQL for[managing database users](/influxdb/v1/administration/authentication_and_authorization/#user-management-commands). ## InfluxQL reference The [reference documentation for InfluxQL](/influxdb/v1/query_language/spec/). --- # InfluxQL Continuous Queries Source: https://docs.influxdata.com/influxdb/v1/query_language/continuous_queries/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. ## Introduction Continuous queries (CQ) are InfluxQL queries that run automatically and periodically on realtime data and store query results in a specified measurement. | [Basic Syntax](#basic-syntax) | [Advanced Syntax](#advanced-syntax) |[CQ Management](#continuous-query-management)| |-------------------------------------------------------------------|-------------------------------------------------------------------------|---------------------------------------------| | [Examples of Basic Syntax](#examples-of-basic-syntax) | [Examples of Advanced Syntax](#examples-of-advanced-syntax) | [CQ Use Cases](#continuous-query-use-cases) | |[Common Issues with Basic Syntax](#common-issues-with-basic-syntax)|[Common Issues with Advanced Syntax](#common-issues-with-advanced-syntax)| [Further information](#further-information) | ## Syntax ### Basic syntax ```sql CREATE CONTINUOUS QUERY ON BEGIN END ``` #### Description of basic syntax ##### The `cq_query` The `cq_query` requires a[function](/influxdb/v1/concepts/glossary/#function), an [`INTO` clause](/influxdb/v1/query_language/spec/#clauses), and a [`GROUP BY time()` clause](/influxdb/v1/query_language/spec/#clauses): ```sql SELECT INTO FROM [WHERE ] GROUP BY time()[,] ``` > > > **Note:** Notice that the `cq_query` does not require a time range in a `WHERE` clause. > InfluxDB automatically generates a time range for the `cq_query` when it executes the CQ. > Any user-specified time ranges in the `cq_query`’s `WHERE` clause will be ignored > by the system. > > ##### Schedule and coverage Continuous queries operate on real-time data. They use the local server’s timestamp, the `GROUP BY time()` interval, and InfluxDB database’s preset time boundaries to determine when to execute and what time range to cover in the query. CQs execute at the same interval as the `cq_query`’s `GROUP BY time()` interval, and they run at the start of the InfluxDB database’s preset time boundaries. If the `GROUP BY time()` interval is one hour, the CQ executes at the start of every hour. When the CQ executes, it runs a single query for the time range between[`now()`](/influxdb/v1/concepts/glossary/#now) and `now()` minus the`GROUP BY time()` interval. If the `GROUP BY time()` interval is one hour and the current time is 17:00, the query’s time range is between 16:00 and 16:59.999999999. #### Examples of basic syntax The examples below use the following sample data in the `transportation`database. The measurement `bus_data` stores 15-minute resolution data on the number of bus`passengers` and `complaints`: ``` ## name: bus_data time passengers complaints 2016-08-28T07:00:00Z 5 9 2016-08-28T07:15:00Z 8 9 2016-08-28T07:30:00Z 8 9 2016-08-28T07:45:00Z 7 9 2016-08-28T08:00:00Z 8 9 2016-08-28T08:15:00Z 15 7 2016-08-28T08:30:00Z 15 7 2016-08-28T08:45:00Z 17 7 2016-08-28T09:00:00Z 20 7 ``` ##### Automatically downsampling data Use a simple CQ to automatically downsample data from a single field and write the results to another measurement in the same database. ```sql CREATE CONTINUOUS QUERY "cq_basic" ON "transportation" BEGIN SELECT mean("passengers") INTO "average_passengers" FROM "bus_data" GROUP BY time(1h) END ``` `cq_basic` calculates the average hourly number of passengers from the`bus_data` measurement and stores the results in the `average_passengers`measurement in the `transportation` database. `cq_basic` executes at one-hour intervals, the same interval as the`GROUP BY time()` interval. Every hour, `cq_basic` runs a single query that covers the time range between`now()` and `now()` minus the `GROUP BY time()` interval, that is, the time range between `now()` and one hour prior to `now()`. Annotated log output on the morning of August 28, 2016: ```sql > At **8:00** `cq_basic` executes a query with the time range `time >= '7:00' AND time < '08:00'`. `cq_basic` writes one point to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T07:00:00Z 7 > At **9:00** `cq_basic` executes a query with the time range `time >= '8:00' AND time < '9:00'`. `cq_basic` writes one point to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T08:00:00Z 13.75 ``` Here are the results: ``` > SELECT * FROM "average_passengers" ## name: average_passengers time mean 2016-08-28T07:00:00Z 7 2016-08-28T08:00:00Z 13.75 ``` ##### Automatically downsampling data into another retention policy [Fully qualify](/influxdb/v1/query_language/explore-data/#the-basic-select-statement)the destination measurement to store the downsampled data in a non-`DEFAULT`[retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp) (RP). ```sql CREATE CONTINUOUS QUERY "cq_basic_rp" ON "transportation" BEGIN SELECT mean("passengers") INTO "transportation"."three_weeks"."average_passengers" FROM "bus_data" GROUP BY time(1h) END ``` `cq_basic_rp` calculates the average hourly number of passengers from the`bus_data` measurement and stores the results in the `transportation` database, the `three_weeks` RP, and the `average_passengers` measurement. `cq_basic_rp` executes at one-hour intervals, the same interval as the`GROUP BY time()` interval. Every hour, `cq_basic_rp` runs a single query that covers the time range between`now()` and `now()` minus the `GROUP BY time()` interval, that is, the time range between `now()` and one hour prior to `now()`. Annotated log output on the morning of August 28, 2016: ```sql > At **8:00** `cq_basic_rp` executes a query with the time range `time >= '7:00' AND time < '8:00'`. `cq_basic_rp` writes one point to the `three_weeks` RP and the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T07:00:00Z 7 > At **9:00** `cq_basic_rp` executes a query with the time range `time >= '8:00' AND time < '9:00'`. `cq_basic_rp` writes one point to the `three_weeks` RP and the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T08:00:00Z 13.75 ``` Here are the results: ``` > SELECT * FROM "transportation"."three_weeks"."average_passengers" ## name: average_passengers time mean 2016-08-28T07:00:00Z 7 2016-08-28T08:00:00Z 13.75 ``` `cq_basic_rp` uses CQs and retention policies to automatically downsample data and keep those downsampled data for an alternative length of time. See the [Downsampling and Data Retention](/influxdb/v1/guides/downsampling_and_retention/)guide for an in-depth discussion about this CQ use case. ##### Automatically downsampling a database with backreferencing Use a function with a wildcard (`*`) and `INTO` query’s[backreferencing syntax](/influxdb/v1/query_language/explore-data/#the-into-clause)to automatically downsample data from all measurements and numerical fields in a database. ```sql CREATE CONTINUOUS QUERY "cq_basic_br" ON "transportation" BEGIN SELECT mean(*) INTO "downsampled_transportation"."autogen".:MEASUREMENT FROM /.*/ GROUP BY time(30m),* END ``` `cq_basic_br` calculates the 30-minute average of `passengers` and `complaints`from every measurement in the `transportation` database (in this case, there’s only the`bus_data` measurement). It stores the results in the `downsampled_transportation` database. `cq_basic_br` executes at 30 minutes intervals, the same interval as the`GROUP BY time()` interval. Every 30 minutes, `cq_basic_br` runs a single query that covers the time range between `now()` and `now()` minus the `GROUP BY time()` interval, that is, the time range between `now()` and 30 minutes prior to `now()`. Annotated log output on the morning of August 28, 2016: ```sql > At **7:30**, `cq_basic_br` executes a query with the time range `time >= '7:00' AND time < '7:30'`. `cq_basic_br` writes two points to the `bus_data` measurement in the `downsampled_transportation` database: > name: bus_data -------------- time mean_complaints mean_passengers 2016-08-28T07:00:00Z 9 6.5 > At **8:00**, `cq_basic_br` executes a query with the time range `time >= '7:30' AND time < '8:00'`. `cq_basic_br` writes two points to the `bus_data` measurement in the `downsampled_transportation` database: > name: bus_data -------------- time mean_complaints mean_passengers 2016-08-28T07:30:00Z 9 7.5 > [...] > At **9:00**, `cq_basic_br` executes a query with the time range `time >= '8:30' AND time < '9:00'`. `cq_basic_br` writes two points to the `bus_data` measurement in the `downsampled_transportation` database: > name: bus_data -------------- time mean_complaints mean_passengers 2016-08-28T08:30:00Z 7 16 ``` Here are the results: ``` > SELECT * FROM "downsampled_transportation."autogen"."bus_data" ## name: bus_data time mean_complaints mean_passengers 2016-08-28T07:00:00Z 9 6.5 2016-08-28T07:30:00Z 9 7.5 2016-08-28T08:00:00Z 8 11.5 2016-08-28T08:30:00Z 7 16 ``` ##### Automatically downsampling data and configuring CQ time boundaries Use an[offset interval](/influxdb/v1/query_language/explore-data/#advanced-group-by-time-syntax)in the `GROUP BY time()` clause to alter both the CQ’s default execution time and preset time boundaries. ```sql CREATE CONTINUOUS QUERY "cq_basic_offset" ON "transportation" BEGIN SELECT mean("passengers") INTO "average_passengers" FROM "bus_data" GROUP BY time(1h,15m) END ``` `cq_basic_offset`calculates the average hourly number of passengers from the`bus_data` measurement and stores the results in the `average_passengers`measurement. `cq_basic_offset` executes at one-hour intervals, the same interval as the`GROUP BY time()` interval. The 15 minute offset interval forces the CQ to execute 15 minutes after the default execution time; `cq_basic_offset` executes at 8:15 instead of 8:00. Every hour, `cq_basic_offset` runs a single query that covers the time range between `now()` and `now()` minus the `GROUP BY time()` interval, that is, the time range between `now()` and one hour prior to `now()`. The 15 minute offset interval shifts forward the generated preset time boundaries in the CQ’s `WHERE` clause; `cq_basic_offset` queries between 7:15 and 8:14.999999999 instead of 7:00 and 7:59.999999999. Annotated log output on the morning of August 28, 2016: ``` > At **8:15** `cq_basic_offset` executes a query with the time range `time >= '7:15' AND time < '8:15'`. `cq_basic_offset` writes one point to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T07:15:00Z 7.75 > At **9:15** `cq_basic_offset` executes a query with the time range `time >= '8:15' AND time < '9:15'`. `cq_basic_offset` writes one point to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T08:15:00Z 16.75 ``` Here are the results: ``` > SELECT * FROM "average_passengers" ## name: average_passengers time mean 2016-08-28T07:15:00Z 7.75 2016-08-28T08:15:00Z 16.75 ``` Notice that the timestamps are for 7:15 and 8:15 instead of 7:00 and 8:00. #### Common issues with basic syntax ##### Handling time intervals with no data CQs do not write any results for a time interval if no data fall within that time range. Note that the basic syntax does not support using[`fill()`](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill)to change the value reported for intervals with no data. Basic syntax CQs ignore `fill()` if it’s included in the CQ query. A possible workaround is to use the[advanced CQ syntax](#advanced-syntax). ##### Resampling previous time intervals The basic CQ runs a single query that covers the time range between `now()`and `now()` minus the `GROUP BY time()` interval. See the [advanced syntax](#advanced-syntax) for how to configure the query’s time range. ##### Backfilling results for older data CQs operate on realtime data, that is, data with timestamps that occur relative to [`now()`](/influxdb/v1/concepts/glossary/#now). Use a basic[`INTO` query](/influxdb/v1/query_language/explore-data/#the-into-clause)to backfill results for data with older timestamps. ##### Missing tags in the CQ results By default, all[`INTO` queries](/influxdb/v1/query_language/explore-data/#the-into-clause)convert any tags in the source measurement to fields in the destination measurement. Include `GROUP BY *` in the CQ to preserve tags in the destination measurement. ### Advanced syntax ```txt CREATE CONTINUOUS QUERY ON RESAMPLE EVERY FOR BEGIN END ``` #### Description of advanced syntax ##### The `cq_query` See [Description of Basic Syntax](/influxdb/v1/query_language/continuous_queries/#description-of-basic-syntax). ##### Scheduling and coverage CQs operate on real-time data. With the advanced syntax, CQs use the local server’s timestamp, the information in the `RESAMPLE` clause, and the InfluxDB server’s preset time boundaries to determine when to execute and what time range to cover in the query. CQs execute at the same interval as the `EVERY` interval in the `RESAMPLE`clause, and they run at the start of InfluxDB’s preset time boundaries. If the `EVERY` interval is two hours, InfluxDB executes the CQ at the top of every other hour. When the CQ executes, it runs a single query for the time range between[`now()`](/influxdb/v1/concepts/glossary/#now) and `now()` minus the `FOR` interval in the `RESAMPLE` clause. If the `FOR` interval is two hours and the current time is 17:00, the query’s time range is between 15:00 and 16:59.999999999. Both the `EVERY` interval and the `FOR` interval accept[duration literals](/influxdb/v1/query_language/spec/#durations). The `RESAMPLE` clause works with either or both of the `EVERY` and `FOR` intervals configured. CQs default to the relevant[basic syntax behavior](/influxdb/v1/query_language/continuous_queries/#description-of-basic-syntax)if the `EVERY` interval or `FOR` interval is not provided (see the first issue in[Common Issues with Advanced Syntax](/influxdb/v1/query_language/continuous_queries/#common-issues-with-advanced-syntax)for an anomalous case). #### Examples of advanced syntax The examples below use the following sample data in the `transportation` database. The measurement `bus_data` stores 15-minute resolution data on the number of bus`passengers`: ``` ## name: bus_data time passengers 2016-08-28T06:30:00Z 2 2016-08-28T06:45:00Z 4 2016-08-28T07:00:00Z 5 2016-08-28T07:15:00Z 8 2016-08-28T07:30:00Z 8 2016-08-28T07:45:00Z 7 2016-08-28T08:00:00Z 8 2016-08-28T08:15:00Z 15 2016-08-28T08:30:00Z 15 2016-08-28T08:45:00Z 17 2016-08-28T09:00:00Z 20 ``` ##### Configuring execution intervals Use an `EVERY` interval in the `RESAMPLE` clause to specify the CQ’s execution interval. ```sql CREATE CONTINUOUS QUERY "cq_advanced_every" ON "transportation" RESAMPLE EVERY 30m BEGIN SELECT mean("passengers") INTO "average_passengers" FROM "bus_data" GROUP BY time(1h) END ``` `cq_advanced_every` calculates the one-hour average of `passengers`from the `bus_data` measurement and stores the results in the`average_passengers` measurement in the `transportation` database. `cq_advanced_every` executes at 30-minute intervals, the same interval as the`EVERY` interval. Every 30 minutes, `cq_advanced_every` runs a single query that covers the time range for the current time bucket, that is, the one-hour time bucket that intersects with `now()`. Annotated log output on the morning of August 28, 2016: ```sql > At **8:00**, `cq_advanced_every` executes a query with the time range `WHERE time >= '7:00' AND time < '8:00'`. `cq_advanced_every` writes one point to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T07:00:00Z 7 > At **8:30**, `cq_advanced_every` executes a query with the time range `WHERE time >= '8:00' AND time < '9:00'`. `cq_advanced_every` writes one point to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T08:00:00Z 12.6667 > At **9:00**, `cq_advanced_every` executes a query with the time range `WHERE time >= '8:00' AND time < '9:00'`. `cq_advanced_every` writes one point to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T08:00:00Z 13.75 ``` Here are the results: ``` > SELECT * FROM "average_passengers" ## name: average_passengers time mean 2016-08-28T07:00:00Z 7 2016-08-28T08:00:00Z 13.75 ``` Notice that `cq_advanced_every` calculates the result for the 8:00 time interval twice. First, it runs at 8:30 and calculates the average for every available data point between 8:00 and 9:00 (`8`,`15`, and `15`). Second, it runs at 9:00 and calculates the average for every available data point between 8:00 and 9:00 (`8`, `15`, `15`, and `17`). Because of the way InfluxDB[handles duplicate points](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-does-influxdb-handle-duplicate-points), the second result simply overwrites the first result. ##### Configuring time ranges for resampling Use a `FOR` interval in the `RESAMPLE` clause to specify the length of the CQ’s time range. ```sql CREATE CONTINUOUS QUERY "cq_advanced_for" ON "transportation" RESAMPLE FOR 1h BEGIN SELECT mean("passengers") INTO "average_passengers" FROM "bus_data" GROUP BY time(30m) END ``` `cq_advanced_for` calculates the 30-minute average of `passengers`from the `bus_data` measurement and stores the results in the `average_passengers`measurement in the `transportation` database. `cq_advanced_for` executes at 30-minute intervals, the same interval as the`GROUP BY time()` interval. Every 30 minutes, `cq_advanced_for` runs a single query that covers the time range between `now()` and `now()` minus the `FOR` interval, that is, the time range between `now()` and one hour prior to `now()`. Annotated log output on the morning of August 28, 2016: ```sql > At **8:00** `cq_advanced_for` executes a query with the time range `WHERE time >= '7:00' AND time < '8:00'`. `cq_advanced_for` writes two points to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T07:00:00Z 6.5 2016-08-28T07:30:00Z 7.5 > At **8:30** `cq_advanced_for` executes a query with the time range `WHERE time >= '7:30' AND time < '8:30'`. `cq_advanced_for` writes two points to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T07:30:00Z 7.5 2016-08-28T08:00:00Z 11.5 > At **9:00** `cq_advanced_for` executes a query with the time range `WHERE time >= '8:00' AND time < '9:00'`. `cq_advanced_for` writes two points to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T08:00:00Z 11.5 2016-08-28T08:30:00Z 16 ``` Notice that `cq_advanced_for` will calculate the result for every time interval twice. The CQ calculates the average for the 7:30 time interval at 8:00 and at 8:30, and it calculates the average for the 8:00 time interval at 8:30 and 9:00. Here are the results: ``` > SELECT * FROM "average_passengers" ## name: average_passengers time mean 2016-08-28T07:00:00Z 6.5 2016-08-28T07:30:00Z 7.5 2016-08-28T08:00:00Z 11.5 2016-08-28T08:30:00Z 16 ``` ##### Configuring execution intervals and CQ time ranges Use an `EVERY` interval and `FOR` interval in the `RESAMPLE` clause to specify the CQ’s execution interval and the length of the CQ’s time range. ```sql CREATE CONTINUOUS QUERY "cq_advanced_every_for" ON "transportation" RESAMPLE EVERY 1h FOR 90m BEGIN SELECT mean("passengers") INTO "average_passengers" FROM "bus_data" GROUP BY time(30m) END ``` `cq_advanced_every_for` calculates the 30-minute average of`passengers` from the `bus_data` measurement and stores the results in the`average_passengers` measurement in the `transportation` database. `cq_advanced_every_for` executes at one-hour intervals, the same interval as the`EVERY` interval. Every hour, `cq_advanced_every_for` runs a single query that covers the time range between `now()` and `now()` minus the `FOR` interval, that is, the time range between `now()` and 90 minutes prior to `now()`. Annotated log output on the morning of August 28, 2016: ```sql > At **8:00** `cq_advanced_every_for` executes a query with the time range `WHERE time >= '6:30' AND time < '8:00'`. `cq_advanced_every_for` writes three points to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T06:30:00Z 3 2016-08-28T07:00:00Z 6.5 2016-08-28T07:30:00Z 7.5 > At **9:00** `cq_advanced_every_for` executes a query with the time range `WHERE time >= '7:30' AND time < '9:00'`. `cq_advanced_every_for` writes three points to the `average_passengers` measurement: > name: average_passengers ------------------------ time mean 2016-08-28T07:30:00Z 7.5 2016-08-28T08:00:00Z 11.5 2016-08-28T08:30:00Z 16 ``` Notice that `cq_advanced_every_for` will calculate the result for every time interval twice. The CQ calculates the average for the 7:30 interval at 8:00 and 9:00. Here are the results: ``` > SELECT * FROM "average_passengers" ## name: average_passengers time mean 2016-08-28T06:30:00Z 3 2016-08-28T07:00:00Z 6.5 2016-08-28T07:30:00Z 7.5 2016-08-28T08:00:00Z 11.5 2016-08-28T08:30:00Z 16 ``` ##### Configuring CQ time ranges and filling empty results Use a `FOR` interval and `fill()` to change the value reported for time intervals with no data. Note that at least one data point must fall within the `FOR` interval for `fill()`to operate. If no data fall within the `FOR` interval the CQ writes no points to the destination measurement. ```sql CREATE CONTINUOUS QUERY "cq_advanced_for_fill" ON "transportation" RESAMPLE FOR 2h BEGIN SELECT mean("passengers") INTO "average_passengers" FROM "bus_data" GROUP BY time(1h) fill(1000) END ``` `cq_advanced_for_fill` calculates the one-hour average of `passengers` from the`bus_data` measurement and stores the results in the `average_passengers`measurement in the `transportation` database. Where possible, it writes the value `1000` for time intervals with no results. `cq_advanced_for_fill` executes at one-hour intervals, the same interval as the`GROUP BY time()` interval. Every hour, `cq_advanced_for_fill` runs a single query that covers the time range between `now()` and `now()` minus the `FOR` interval, that is, the time range between `now()` and two hours prior to `now()`. Annotated log output on the morning of August 28, 2016: ```sql > At **6:00**, `cq_advanced_for_fill` executes a query with the time range `WHERE time >= '4:00' AND time < '6:00'`. `cq_advanced_for_fill` writes nothing to `average_passengers`; `bus_data` has no data that fall within that time range. > At **7:00**, `cq_advanced_for_fill` executes a query with the time range `WHERE time >= '5:00' AND time < '7:00'`. `cq_advanced_for_fill` writes two points to `average_passengers`: > name: average_passengers ------------------------ time mean 2016-08-28T05:00:00Z 1000 <------ fill(1000) 2016-08-28T06:00:00Z 3 <------ average of 2 and 4 > [...] > At **11:00**, `cq_advanced_for_fill` executes a query with the time range `WHERE time >= '9:00' AND time < '11:00'`. `cq_advanced_for_fill` writes two points to `average_passengers`: > name: average_passengers ------------------------ 2016-08-28T09:00:00Z 20 <------ average of 20 2016-08-28T10:00:00Z 1000 <------ fill(1000) > ``` At **12:00**, `cq_advanced_for_fill` executes a query with the time range `WHERE time >= '10:00' AND time < '12:00'`.`cq_advanced_for_fill` writes nothing to `average_passengers`; `bus_data` has no data that fall within that time range. Here are the results: ``` > SELECT * FROM "average_passengers" ## name: average_passengers time mean 2016-08-28T05:00:00Z 1000 2016-08-28T06:00:00Z 3 2016-08-28T07:00:00Z 7 2016-08-28T08:00:00Z 13.75 2016-08-28T09:00:00Z 20 2016-08-28T10:00:00Z 1000 ``` > > > **Note:** `fill(previous)` doesn’t fill the result for a time interval if the > previous value is outside the query’s time range. > See [Frequently Asked Questions](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-does-fill-previous-return-empty-results)for more information. > > #### Common issues with advanced syntax ##### If the `EVERY` interval is greater than the `GROUP BY time()` interval If the `EVERY` interval is greater than the `GROUP BY time()` interval, the CQ executes at the same interval as the `EVERY` interval and runs a single query that covers the time range between `now()` and `now()` minus the `EVERY`interval (not between `now()` and `now()` minus the `GROUP BY time()` interval). For example, if the `GROUP BY time()` interval is `5m` and the `EVERY` interval is `10m`, the CQ executes every ten minutes. Every ten minutes, the CQ runs a single query that covers the time range between `now()` and `now()` minus the `EVERY` interval, that is, the time range between `now()` and ten minutes prior to `now()`. This behavior is intentional and prevents the CQ from missing data between execution times. ##### If the `FOR` interval is less than the execution interval If the `FOR` interval is less than the `GROUP BY time()` interval or, if specified, the `EVERY` interval, InfluxDB returns the following error: ```sql error parsing query: FOR duration must be >= GROUP BY time duration: must be a minimum of got ``` To avoid missing data between execution times, the `FOR` interval must be equal to or greater than the `GROUP BY time()` interval or, if specified, the `EVERY`interval. Currently, this is the intended behavior. GitHub Issue [#6963](https://github.com/influxdata/influxdb/issues/6963)outlines a feature request for CQs to support gaps in data coverage. ## Continuous query management Only admin users are allowed to work with CQs. For more on user privileges, see [Authentication and Authorization](/influxdb/v1/administration/authentication_and_authorization/#user-types-and-privileges). ### Listing continuous queries List every CQ on an InfluxDB instance with: ```sql SHOW CONTINUOUS QUERIES ``` `SHOW CONTINUOUS QUERIES` groups results by database. ##### Examples The output shows that the `telegraf` and `mydb` databases have CQs: ``` > SHOW CONTINUOUS QUERIES ## name: _internal name query ## name: telegraf name query idle_hands CREATE CONTINUOUS QUERY idle_hands ON telegraf BEGIN SELECT min(usage_idle) INTO telegraf.autogen.min_hourly_cpu FROM telegraf.autogen.cpu GROUP BY time(1h) END feeling_used CREATE CONTINUOUS QUERY feeling_used ON telegraf BEGIN SELECT mean(used) INTO downsampled_telegraf.autogen.:MEASUREMENT FROM telegraf.autogen./.*/ GROUP BY time(1h) END ## name: downsampled_telegraf name query ## name: mydb name query vampire CREATE CONTINUOUS QUERY vampire ON mydb BEGIN SELECT count(dracula) INTO mydb.autogen.all_of_them FROM mydb.autogen.one GROUP BY time(5m) END ``` ### Deleting continuous queries Delete a CQ from a specific database with: ```sql DROP CONTINUOUS QUERY ON ``` `DROP CONTINUOUS QUERY` returns an empty result. ##### Examples Drop the `idle_hands` CQ from the `telegraf` database: ```sql DROP CONTINUOUS QUERY "idle_hands" ON "telegraf" ``` ### Altering continuous queries CQs cannot be altered once they’re created. To change a CQ, you must `DROP` and re`CREATE` it with the updated settings. ### Continuous query statistics If `query-stats-enabled` is set to `true` in your `influxdb.conf` or using the `INFLUXDB_CONTINUOUS_QUERIES_QUERY_STATS_ENABLED` environment variable, data will be written to `_internal` with information about when continuous queries ran and their duration. Information about CQ configuration settings is available in the [Configuration](/influxdb/v1/administration/config/#continuous-queries-settings) documentation. > > > **Note:** `_internal` houses internal system data and is meant for internal use. > The structure of and data stored in `_internal` can change at any time. > Use of this data falls outside the scope of official InfluxData support. > > ## Continuous query use cases ### Downsampling and Data Retention Use CQs with InfluxDB database[retention policies](/influxdb/v1/concepts/glossary/#retention-policy-rp)(RPs) to mitigate storage concerns. Combine CQs and RPs to automatically downsample high precision data to a lower precision and remove the dispensable, high precision data from the database. See the[Downsampling and data retention](/influxdb/v1/guides/downsampling_and_retention/)guide for a detailed walkthrough of this common use case. ### Precalculating expensive queries Shorten query runtimes by pre-calculating expensive queries with CQs. Use a CQ to automatically downsample commonly queried, high precision data to a lower precision. Queries on lower precision data require fewer resources and return faster. **Tip:** Pre-calculate queries for your preferred graphing tool to accelerate the population of graphs and dashboards. ### Substituting for a `HAVING` clause InfluxQL does not support [`HAVING` clauses](https://en.wikipedia.org/wiki/Having_%28SQL%29). Get the same functionality by creating a CQ to aggregate the data and querying the CQ results to apply the `HAVING` clause. > > > **Note:** InfluxQL supports [subqueries](/influxdb/v1/query_language/explore-data/#subqueries) which also offer similar functionality to `HAVING` clauses. > See [Data Exploration](/influxdb/v1/query_language/explore-data/#subqueries) for more information. > > ##### Example InfluxDB does not accept the following query with a `HAVING` clause. The query calculates the average number of `bees` at `30` minute intervals and requests averages that are greater than `20`. ```sql SELECT mean("bees") FROM "farm" GROUP BY time(30m) HAVING mean("bees") > 20 ``` To get the same results: **1. Create a CQ** This step performs the `mean("bees")` part of the query above. Because this step creates CQ you only need to execute it once. The following CQ automatically calculates the average number of `bees` at`30` minutes intervals and writes those averages to the `mean_bees` field in the`aggregate_bees` measurement. ```sql CREATE CONTINUOUS QUERY "bee_cq" ON "mydb" BEGIN SELECT mean("bees") AS "mean_bees" INTO "aggregate_bees" FROM "farm" GROUP BY time(30m) END ``` **2. Query the CQ results** This step performs the `HAVING mean("bees") > 20` part of the query above. Query the data in the measurement `aggregate_bees` and request values of the `mean_bees` field that are greater than `20` in the `WHERE` clause: ```sql SELECT "mean_bees" FROM "aggregate_bees" WHERE "mean_bees" > 20 ``` ### Substituting for nested functions Some InfluxQL functions[support nesting](/influxdb/v1/troubleshooting/frequently-asked-questions/#which-influxql-functions-support-nesting)of other functions. Most do not. If your function does not support nesting, you can get the same functionality using a CQ to calculate the inner-most function. Then simply query the CQ results to calculate the outer-most function. > > > **Note:** InfluxQL supports [subqueries](/influxdb/v1/query_language/explore-data/#subqueries) which also offer the same functionality as nested functions. > See [Data Exploration](/influxdb/v1/query_language/explore-data/#subqueries) for more information. > > ##### Example InfluxDB does not accept the following query with a nested function. The query calculates the number of non-null values of `bees` at `30` minute intervals and the average of those counts: ```sql SELECT mean(count("bees")) FROM "farm" GROUP BY time(30m) ``` To get the same results: **1. Create a CQ** This step performs the `count("bees")` part of the nested function above. Because this step creates a CQ you only need to execute it once. The following CQ automatically calculates the number of non-null values of `bees` at `30` minute intervals and writes those counts to the `count_bees` field in the `aggregate_bees` measurement. ```sql CREATE CONTINUOUS QUERY "bee_cq" ON "mydb" BEGIN SELECT count("bees") AS "count_bees" INTO "aggregate_bees" FROM "farm" GROUP BY time(30m) END ``` **2. Query the CQ results** This step performs the `mean([...])` part of the nested function above. Query the data in the measurement `aggregate_bees` to calculate the average of the`count_bees` field: ```sql SELECT mean("count_bees") FROM "aggregate_bees" WHERE time >= AND time <= ``` ## Further information To see how to combine two InfluxDB features, CQs, and retention policies, to periodically downsample data and automatically expire the dispensable high precision data, see [Downsampling and data retention](/influxdb/v1/guides/downsampling_and_retention/). Kapacitor, InfluxData’s data processing engine, can do the same work as continuous queries in InfluxDB databases. To learn when to use Kapacitor instead of InfluxDB and how to perform the same CQ functionality with a TICKscript, see [examples of continuous queries in Kapacitor](/kapacitor/v1/guides/continuous_queries/). | Basic Syntax | Advanced Syntax | CQ Management | | --- | --- | --- | | Examples of Basic Syntax | Examples of Advanced Syntax | CQ Use Cases | | Common Issues with Basic Syntax | Common Issues with Advanced Syntax | Further information | --- # Explore data using InfluxQL Source: https://docs.influxdata.com/influxdb/v1/query_language/explore-data/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxQL is an SQL-like query language for interacting with data in InfluxDB. The following sections detail InfluxQL’s `SELECT` statement and useful query syntax for exploring your data. | **The Basics:** | **Configure Query Results:** | **General Tips on Query Syntax:** | |---------------------------------------------------|-----------------------------------------------------------------|-----------------------------------------------------------------| |[The SELECT statement](#the-basic-select-statement)| [ORDER BY time DESC](#order-by-time-desc) | [Time Syntax](#time-syntax) | | [The WHERE clause](#the-where-clause) | [The LIMIT and SLIMIT clauses](#the-limit-and-slimit-clauses) | [Regular Expressions](#regular-expressions) | | [The GROUP BY clause](#the-group-by-clause) |[The OFFSET and SOFFSET clauses](#the-offset-and-soffset-clauses)|[Data types and cast operations](#data-types-and-cast-operations)| | [The INTO clause](#the-into-clause) | [The Time Zone clause](#the-time-zone-clause) | [Merge behavior](#merge-behavior) | | [](#) | [](#) | [Multiple statements](#multiple-statements) | | [](#) | [](#) | [Subqueries](#subqueries) | ### Sample data This document uses publicly available data from the[National Oceanic and Atmospheric Administration’s (NOAA) Center for Operational Oceanographic Products and Services](http://tidesandcurrents.noaa.gov/stations.html?type=Water+Levels). See the [Sample Data](/influxdb/v1/query_language/data_download/) page to download the data and follow along with the example queries in the sections below. Start by logging into the Influx CLI: ```bash $ influx -precision rfc3339 -database NOAA_water_database Connected to http://localhost:8086 version 1.13.0 InfluxDB shell 1.13.0 > ``` Next, get acquainted with this subsample of the data in the `h2o_feet` measurement: name: h2o\_feet | time | level description | location |water\_level| |--------------------|--------------------|-------------|------------| |2015-08-18T00:00:00Z|between 6 and 9 feet|coyote\_creek| 8.12 | |2015-08-18T00:00:00Z| below 3 feet |santa\_monica| 2.064 | |2015-08-18T00:06:00Z|between 6 and 9 feet|coyote\_creek| 8.005 | |2015-08-18T00:06:00Z| below 3 feet |santa\_monica| 2.116 | |2015-08-18T00:12:00Z|between 6 and 9 feet|coyote\_creek| 7.887 | |2015-08-18T00:12:00Z| below 3 feet |santa\_monica| 2.028 | The data in the `h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement)occur at six-minute time intervals. The measurement has one [tag key](/influxdb/v1/concepts/glossary/#tag-key)(`location`) which has two [tag values](/influxdb/v1/concepts/glossary/#tag-value):`coyote_creek` and `santa_monica`. The measurement also has two [fields](/influxdb/v1/concepts/glossary/#field):`level description` stores string [field values](/influxdb/v1/concepts/glossary/#field-value)and `water_level` stores float field values. All of these data is in the `NOAA_water_database` [database](/influxdb/v1/concepts/glossary/#database). > > > **Disclaimer:** The `level description` field isn’t part of the original NOAA data - we snuck it in there for the sake of having a field key with a special character and string field values. > > ## The basic SELECT statement The `SELECT` statement queries data from a particular [measurement](/influxdb/v1/concepts/glossary/#measurement) or measurements. ### Syntax ```sql SELECT [,,] FROM [,] ``` The `SELECT` statement requires a `SELECT` clause and a `FROM` clause. #### `SELECT` clause The `SELECT` clause supports several formats for specifying data: `SELECT *` Returns all [fields](/influxdb/v1/concepts/glossary/#field) and [tags](/influxdb/v1/concepts/glossary/#tag). `SELECT ""` Returns a specific field. `SELECT "",""` Returns more than one field. `SELECT "",""` Returns a specific field and tag. The `SELECT` clause must specify at least one field when it includes a tag. `SELECT ""::field,""::tag` Returns a specific field and tag. The `::[field | tag]` syntax specifies the [identifier’s](/influxdb/v1/concepts/glossary/#identifier) type. Use this syntax to differentiate between field keys and tag keys that have the same name. Other supported features:[Arithmetic operations](/influxdb/v1/query_language/math_operators/),[Functions](/influxdb/v1/query_language/functions/),[Basic cast operations](#data-types-and-cast-operations),[Regular expressions](#regular-expressions) > > > **Note:** The SELECT statement cannot include an aggregate function **and** a non-aggregate function, field key, or tag key. For more information, see [error about mixing aggregate and non-aggregate queries](/influxdb/v1/troubleshooting/errors/#error-parsing-query-mixing-aggregate-and-non-aggregate-queries-is-not-supported). > > #### `FROM` clause The `FROM` clause supports several formats for specifying a [measurement(s)](/influxdb/v1/concepts/glossary/#measurement): `FROM ` Returns data from a single measurement. If you’re using the [CLI](/influxdb/v1/tools/shell/) InfluxDB queries the measurement in the[`USE`d](/influxdb/v1/tools/shell/#commands)[database](/influxdb/v1/concepts/glossary/#database) and the `DEFAULT` [retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp). If you’re using the [InfluxDB API](/influxdb/v1/api/) InfluxDB queries the measurement in the database specified in the [`db` query string parameter](/influxdb/v1/api/query/)and the `DEFAULT` retention policy. `FROM ,` Returns data from more than one measurement. `FROM ..` Returns data from a fully qualified measurement. Fully qualify a measurement by specifying its database and retention policy. `FROM ..` Returns data from a measurement in a user-specified [database](/influxdb/v1/concepts/glossary/#database) and the `DEFAULT`[retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp). Other supported features:[Regular Expressions](#regular-expressions) #### Quoting [Identifiers](/influxdb/v1/concepts/glossary/#identifier) **must** be double quoted if they contain characters other than `[A-z,0-9,_]`, if they begin with a digit, or if they are an [InfluxQL keyword](https://github.com/influxdata/influxql/blob/master/README.md#keywords). While not always necessary, we recommend that you double quote identifiers. > > > **Note:** The quoting syntax for queries differs from the [line protocol](/influxdb/v1/concepts/glossary/#influxdb-line-protocol). > Please review the [rules for single and double-quoting](/influxdb/v1/troubleshooting/frequently-asked-questions/#when-should-i-single-quote-and-when-should-i-double-quote-in-queries) in queries. > > ### Examples #### Select all fields and tags from a single measurement ``` > SELECT * FROM "h2o_feet" ## name: h2o_feet time level description location water_level 2015-08-18T00:00:00Z below 3 feet santa_monica 2.064 2015-08-18T00:00:00Z between 6 and 9 feet coyote_creek 8.12 [...] 2015-09-18T21:36:00Z between 3 and 6 feet santa_monica 5.066 2015-09-18T21:42:00Z between 3 and 6 feet santa_monica 4.938 ``` The query selects all [fields](/influxdb/v1/concepts/glossary/#field) and[tags](/influxdb/v1/concepts/glossary/#tag) from the `h2o_feet`[measurement](/influxdb/v1/concepts/glossary/#measurement). If you’re using the [CLI](/influxdb/v1/tools/shell/) be sure to enter`USE NOAA_water_database` before you run the query. The CLI queries the data in the `USE`d database and the`DEFAULT` [retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp). If you’re using the [InfluxDB API](/influxdb/v1/api/) be sure to set the`db` [query string parameter](/influxdb/v1/api/query/)to `NOAA_water_database`. If you do not set the `rp` query string parameter, the InfluxDB API automatically queries the database’s `DEFAULT` retention policy. #### Select specific tags and fields from a single measurement ``` > SELECT "level description","location","water_level" FROM "h2o_feet" ## name: h2o_feet time level description location water_level 2015-08-18T00:00:00Z below 3 feet santa_monica 2.064 2015-08-18T00:00:00Z between 6 and 9 feet coyote_creek 8.12 [...] 2015-09-18T21:36:00Z between 3 and 6 feet santa_monica 5.066 2015-09-18T21:42:00Z between 3 and 6 feet santa_monica 4.938 ``` The query selects the `level description` field, the `location` tag, and the`water_level` field. Note that the `SELECT` clause must specify at least one field when it includes a tag. #### Select specific tags and fields from a single measurement, and provide their identifier type ``` > SELECT "level description"::field,"location"::tag,"water_level"::field FROM "h2o_feet" ## name: h2o_feet time level description location water_level 2015-08-18T00:00:00Z below 3 feet santa_monica 2.064 2015-08-18T00:00:00Z between 6 and 9 feet coyote_creek 8.12 [...] 2015-09-18T21:36:00Z between 3 and 6 feet santa_monica 5.066 2015-09-18T21:42:00Z between 3 and 6 feet santa_monica 4.938 ``` The query selects the `level description` field, the `location` tag, and the`water_level` field from the `h2o_feet` measurement. The `::[field | tag]` syntax specifies if the[identifier](/influxdb/v1/concepts/glossary/#identifier) is a field or tag. Use `::[field | tag]` to differentiate between [an identical field key and tag key ](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-do-i-query-data-with-an-identical-tag-key-and-field-key). That syntax is not required for most use cases. #### Select all fields from a single measurement ``` > SELECT *::field FROM "h2o_feet" ## name: h2o_feet time level description water_level 2015-08-18T00:00:00Z below 3 feet 2.064 2015-08-18T00:00:00Z between 6 and 9 feet 8.12 [...] 2015-09-18T21:36:00Z between 3 and 6 feet 5.066 2015-09-18T21:42:00Z between 3 and 6 feet 4.938 ``` The query selects all fields from the `h2o_feet` measurement. The `SELECT` clause supports combining the `*` syntax with the `::` syntax. #### Select a specific field from a measurement and perform basic arithmetic ``` > SELECT ("water_level" * 2) + 4 FROM "h2o_feet" ## name: h2o_feet time water_level 2015-08-18T00:00:00Z 20.24 2015-08-18T00:00:00Z 8.128 [...] 2015-09-18T21:36:00Z 14.132 2015-09-18T21:42:00Z 13.876 ``` The query multiplies `water_level`’s field values by two and adds four to those values. Note that InfluxDB follows the standard order of operations. See [Mathematical Operators](/influxdb/v1/query_language/math_operators/)for more on supported operators. #### Select all data from more than one measurement ``` > SELECT * FROM "h2o_feet","h2o_pH" ## name: h2o_feet time level description location pH water_level 2015-08-18T00:00:00Z below 3 feet santa_monica 2.064 2015-08-18T00:00:00Z between 6 and 9 feet coyote_creek 8.12 [...] 2015-09-18T21:36:00Z between 3 and 6 feet santa_monica 5.066 2015-09-18T21:42:00Z between 3 and 6 feet santa_monica 4.938 ## name: h2o_pH time level description location pH water_level 2015-08-18T00:00:00Z santa_monica 6 2015-08-18T00:00:00Z coyote_creek 7 [...] 2015-09-18T21:36:00Z santa_monica 8 2015-09-18T21:42:00Z santa_monica 7 ``` The query selects all fields and tags from two measurements: `h2o_feet` and`h2o_pH`. Separate multiple measurements with a comma (`,`). #### Select all data from a fully qualified measurement ``` > SELECT * FROM "NOAA_water_database"."autogen"."h2o_feet" ## name: h2o_feet time level description location water_level 2015-08-18T00:00:00Z below 3 feet santa_monica 2.064 2015-08-18T00:00:00Z between 6 and 9 feet coyote_creek 8.12 [...] 2015-09-18T21:36:00Z between 3 and 6 feet santa_monica 5.066 2015-09-18T21:42:00Z between 3 and 6 feet santa_monica 4.938 ``` The query selects data in the `NOAA_water_database`, the `autogen` retention policy, and the measurement `h2o_feet`. In the CLI, fully qualify a measurement to query data in a database other than the `USE`d database and in a retention policy other than the`DEFAULT` retention policy. In the InfluxDB API, fully qualify a measurement in place of using the `db`and `rp` query string parameters if desired. #### Select all data from a measurement in a particular database ``` > SELECT * FROM "NOAA_water_database".."h2o_feet" ## name: h2o_feet time level description location water_level 2015-08-18T00:00:00Z below 3 feet santa_monica 2.064 2015-08-18T00:00:00Z between 6 and 9 feet coyote_creek 8.12 [...] 2015-09-18T21:36:00Z between 3 and 6 feet santa_monica 5.066 2015-09-18T21:42:00Z between 3 and 6 feet santa_monica 4.938 ``` The query selects data in the `NOAA_water_database`, the `DEFAULT` retention policy, and the `h2o_feet` measurement. The `..` indicates the `DEFAULT` retention policy for the specified database. In the CLI, specify the database to query data in a database other than the`USE`d database. In the InfluxDB API, specify the database in place of using the `db` query string parameter if desired. ### Common issues with the SELECT statement #### Selecting tag keys in the SELECT clause A query requires at least one [field key](/influxdb/v1/concepts/glossary/#field-key)in the `SELECT` clause to return data. If the `SELECT` clause only includes a single [tag key](/influxdb/v1/concepts/glossary/#tag-key) or several tag keys, the query returns an empty response. This behavior is a result of how the system stores data. ##### Example The following query returns no data because it specifies a single tag key (`location`) in the `SELECT` clause: ```sql SELECT "location" FROM "h2o_feet" ``` To return any data associated with the `location` tag key, the query’s `SELECT`clause must include at least one field key (`water_level`): ```sql > SELECT "water_level","location" FROM "h2o_feet" name: h2o_feet time water_level location ---- ----------- -------- 2015-08-18T00:00:00Z 8.12 coyote_creek 2015-08-18T00:00:00Z 2.064 santa_monica [...] 2015-09-18T21:36:00Z 5.066 santa_monica 2015-09-18T21:42:00Z 4.938 santa_monica ``` ## The `WHERE` clause The `WHERE` filters data based on[fields](/influxdb/v1/concepts/glossary/#field),[tags](/influxdb/v1/concepts/glossary/#tag), and/or[timestamps](/influxdb/v1/concepts/glossary/#timestamp). Tired of reading? Check out this InfluxQL Short: ### Syntax ``` SELECT_clause FROM_clause WHERE [(AND|OR) [...]] ``` The `WHERE` clause supports `conditional_expression`s on fields, tags, and timestamps. > > > **Note** InfluxDB does not support using OR in the WHERE clause to specify multiple time ranges. For example, InfluxDB returns an empty response for the following query: > > `> SELECT * FROM "absolutismus" WHERE time = '2016-07-31T20:07:00Z' OR time = '2016-07-31T23:07:17Z'` #### Fields ``` field_key ['string' | boolean | float | integer] ``` The `WHERE` clause supports comparisons against string, boolean, float, and integer [field values](/influxdb/v1/concepts/glossary/#field-value). Single quote string field values in the `WHERE` clause. Queries with unquoted string field values or double quoted string field values will not return any data and, in most cases,[will not return an error](#common-issues-with-the-where-clause). ##### Supported operators |Operator| Meaning | |--------|------------------------| | `=` | equal to | | `<>` | not equal to | | `!=` | not equal to | | `>` | greater than | | `>=` |greater than or equal to| | `<` | less than | | `<=` | less than or equal to | Other supported features:[Arithmetic Operations](/influxdb/v1/query_language/math_operators/),[Regular Expressions](#regular-expressions) #### Tags ```sql tag_key ['tag_value'] ``` Single quote [tag values](/influxdb/v1/concepts/glossary/#tag-value) in the `WHERE` clause. Queries with unquoted tag values or double quoted tag values will not return any data and, in most cases,[will not return an error](#common-issues-with-the-where-clause). ##### Supported operators |Operator| Meaning | |--------|------------| | `=` | equal to | | `<>` |not equal to| | `!=` |not equal to| Other supported features:[Regular Expressions](#regular-expressions) #### Timestamps For most `SELECT` statements, the default time range is between [`1677-09-21 00:12:43.145224194` and `2262-04-11T23:47:16.854775806Z` UTC](/influxdb/v1/troubleshooting/frequently-asked-questions/#what-are-the-minimum-and-maximum-timestamps-that-influxdb-can-store). For `SELECT` statements with a [`GROUP BY time()` clause](#group-by-time-intervals), the default time range is between `1677-09-21 00:12:43.145224194` UTC and [`now()`](/influxdb/v1/concepts/glossary/#now). The [Time Syntax](#time-syntax) section on this page details how to specify alternative time ranges in the `WHERE` clause. ### Examples #### Select data that have specific field key-values ``` > SELECT * FROM "h2o_feet" WHERE "water_level" > 8 ## name: h2o_feet time level description location water_level 2015-08-18T00:00:00Z between 6 and 9 feet coyote_creek 8.12 2015-08-18T00:06:00Z between 6 and 9 feet coyote_creek 8.005 [...] 2015-09-18T00:12:00Z between 6 and 9 feet coyote_creek 8.189 2015-09-18T00:18:00Z between 6 and 9 feet coyote_creek 8.084 ``` The query returns data from the `h2o_feet`[measurement](/influxdb/v1/concepts/glossary/#measurement) with[field values](/influxdb/v1/concepts/glossary/#field-value) of `water_level`that are greater than eight. #### Select data that have a specific string field key-value ``` > SELECT * FROM "h2o_feet" WHERE "level description" = 'below 3 feet' ## name: h2o_feet time level description location water_level 2015-08-18T00:00:00Z below 3 feet santa_monica 2.064 2015-08-18T00:06:00Z below 3 feet santa_monica 2.116 [...] 2015-09-18T14:06:00Z below 3 feet santa_monica 2.999 2015-09-18T14:36:00Z below 3 feet santa_monica 2.907 ``` The query returns data from the `h2o_feet` measurement with field values of`level description` that equal the `below 3 feet` string. InfluxQL requires single quotes around string field values in the `WHERE`clause. #### Select data that have a specific field key-value and perform basic arithmetic ``` > SELECT * FROM "h2o_feet" WHERE "water_level" + 2 > 11.9 ## name: h2o_feet time level description location water_level 2015-08-29T07:06:00Z at or greater than 9 feet coyote_creek 9.902 2015-08-29T07:12:00Z at or greater than 9 feet coyote_creek 9.938 2015-08-29T07:18:00Z at or greater than 9 feet coyote_creek 9.957 2015-08-29T07:24:00Z at or greater than 9 feet coyote_creek 9.964 2015-08-29T07:30:00Z at or greater than 9 feet coyote_creek 9.954 2015-08-29T07:36:00Z at or greater than 9 feet coyote_creek 9.941 2015-08-29T07:42:00Z at or greater than 9 feet coyote_creek 9.925 2015-08-29T07:48:00Z at or greater than 9 feet coyote_creek 9.902 2015-09-02T23:30:00Z at or greater than 9 feet coyote_creek 9.902 ``` The query returns data from the `h2o_feet` measurement with field values of`water_level` plus two that are greater than 11.9. Note that InfluxDB follows the standard order of operations See [Mathematical Operators](/influxdb/v1/query_language/math_operators/)for more on supported operators. #### Select data that have a specific tag key-value ``` > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' ## name: h2o_feet time water_level 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 [...] 2015-09-18T21:36:00Z 5.066 2015-09-18T21:42:00Z 4.938 ``` The query returns data from the `h2o_feet` measurement where the[tag key](/influxdb/v1/concepts/glossary/#tag-key) `location` is set to `santa_monica`. InfluxQL requires single quotes around tag values in the `WHERE` clause. #### Select data that have specific field key-values and tag key-values ``` > SELECT "water_level" FROM "h2o_feet" WHERE "location" <> 'santa_monica' AND (water_level < -0.59 OR water_level > 9.95) ## name: h2o_feet time water_level 2015-08-29T07:18:00Z 9.957 2015-08-29T07:24:00Z 9.964 2015-08-29T07:30:00Z 9.954 2015-08-29T14:30:00Z -0.61 2015-08-29T14:36:00Z -0.591 2015-08-30T15:18:00Z -0.594 ``` The query returns data from the `h2o_feet` measurement where the tag key`location` is not set to `santa_monica` and where the field values of`water_level` are either less than -0.59 or greater than 9.95. The `WHERE` clause supports the operators `AND` and `OR`, and supports separating logic with parentheses. #### Select data that have specific timestamps ```sql SELECT * FROM "h2o_feet" WHERE time > now() - 7d ``` The query returns data from the `h2o_feet` measurement that have [timestamps](/influxdb/v1/concepts/glossary/#timestamp)within the past seven days. The [Time Syntax](#time-syntax) section on this page offers in-depth information on supported time syntax in the `WHERE` clause. ### Common issues with the `WHERE` clause #### A `WHERE` clause query unexpectedly returns no data In most cases, this issue is the result of missing single quotes around[tag values](/influxdb/v1/concepts/glossary/#tag-value)or string [field values](/influxdb/v1/concepts/glossary/#field-value). Queries with unquoted or double quoted tag values or string field values will not return any data and, in most cases, will not return an error. The first two queries in the code block below attempt to specify the tag value`santa_monica` without any quotes and with double quotes. Those queries return no results. The third query single quotes `santa_monica` (this is the supported syntax) and returns the expected results. ``` > SELECT "water_level" FROM "h2o_feet" WHERE "location" = santa_monica > SELECT "water_level" FROM "h2o_feet" WHERE "location" = "santa_monica" > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' ## name: h2o_feet time water_level 2015-08-18T00:00:00Z 2.064 [...] 2015-09-18T21:42:00Z 4.938 ``` The first two queries in the code block below attempt to specify the string field value `at or greater than 9 feet` without any quotes and with double quotes. The first query returns an error because the string field value includes white spaces. The second query returns no results. The third query single quotes `at or greater than 9 feet` (this is the supported syntax) and returns the expected results. ``` > SELECT "level description" FROM "h2o_feet" WHERE "level description" = at or greater than 9 feet ERR: error parsing query: found than, expected ; at line 1, char 86 > SELECT "level description" FROM "h2o_feet" WHERE "level description" = "at or greater than 9 feet" > SELECT "level description" FROM "h2o_feet" WHERE "level description" = 'at or greater than 9 feet' ## name: h2o_feet time level description 2015-08-26T04:00:00Z at or greater than 9 feet [...] 2015-09-15T22:42:00Z at or greater than 9 feet ``` # The GROUP BY clause The `GROUP BY` clause groups query results by: * one or more specified [tags](/influxdb/v1/concepts/glossary/#tag) * specified time interval > > > **Note:** You cannot use `GROUP BY` to group fields. > > |[GROUP BY tags](#group-by-tags)| | | | |-------------------------------|-------------------------------------------|-------------------------------------------------|-----------------------------------------------------------------------| | **GROUP BY time intervals:** |[Basic Syntax](#basic-group-by-time-syntax)|[Advanced Syntax](#advanced-group-by-time-syntax)|[GROUP BY time intervals and fill()](#group-by-time-intervals-and-fill)| ## GROUP BY tags `GROUP BY ` groups query results by one or more specified [tags](/influxdb/v1/concepts/glossary/#tag). Watch InfluxQL short about `GROUP BY` with tags: #### Syntax ```sql SELECT_clause FROM_clause [WHERE_clause] GROUP BY [* | [,` Groups results by a specific tag `GROUP BY ,` Groups results by more than one tag. The order of the [tag keys](/influxdb/v1/concepts/glossary/#tag-key) is irrelevant. If the query includes a [`WHERE` clause](#the-where-clause) the `GROUP BY`clause must appear after the `WHERE` clause. Other supported features: [Regular Expressions](#regular-expressions) #### Examples ##### Group query results by a single tag ``` > SELECT MEAN("water_level") FROM "h2o_feet" GROUP BY "location" name: h2o_feet tags: location=coyote_creek time mean ---- ---- 1970-01-01T00:00:00Z 5.359342451341401 name: h2o_feet tags: location=santa_monica time mean ---- ---- 1970-01-01T00:00:00Z 3.530863470081006 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to calculate the average `water_level` for each[tag value](/influxdb/v1/concepts/glossary/#tag-value) of `location` in the `h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement). InfluxDB returns results in two [series](/influxdb/v1/concepts/glossary/#series): one for each tag value of `location`. > > > **Note:** In InfluxDB, [epoch 0](https://en.wikipedia.org/wiki/Unix_time) (`1970-01-01T00:00:00Z`) is often used as a null timestamp equivalent. > If you request a query that has no timestamp to return, such as an [aggregation function](/influxdb/v1/query_language/functions/) with an unbounded time range, InfluxDB returns epoch 0 as the timestamp. > > ##### Group query results by more than one tag ```sql > SELECT MEAN("index") FROM "h2o_quality" GROUP BY "location","randtag" name: h2o_quality tags: location=coyote_creek, randtag=1 time mean ---- ---- 1970-01-01T00:00:00Z 50.69033760186263 name: h2o_quality tags: location=coyote_creek, randtag=2 time mean ---- ---- 1970-01-01T00:00:00Z 49.661867544220485 name: h2o_quality tags: location=coyote_creek, randtag=3 time mean ---- ---- 1970-01-01T00:00:00Z 49.360939907550076 name: h2o_quality tags: location=santa_monica, randtag=1 time mean ---- ---- 1970-01-01T00:00:00Z 49.132712456344585 name: h2o_quality tags: location=santa_monica, randtag=2 time mean ---- ---- 1970-01-01T00:00:00Z 50.2937984496124 name: h2o_quality tags: location=santa_monica, randtag=3 time mean ---- ---- 1970-01-01T00:00:00Z 49.99919903884662 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/) to calculate the average `index` for each combination of the `location` [tag](/influxdb/v1/concepts/glossary/#tag) and the `randtag` tag in the`h2o_quality` measurement. Separate multiple tags with a comma in the `GROUP BY` clause. ##### Group query results by all tags ``` > SELECT MEAN("index") FROM "h2o_quality" GROUP BY * name: h2o_quality tags: location=coyote_creek, randtag=1 time mean ---- ---- 1970-01-01T00:00:00Z 50.55405446521169 name: h2o_quality tags: location=coyote_creek, randtag=2 time mean ---- ---- 1970-01-01T00:00:00Z 50.49958856271162 name: h2o_quality tags: location=coyote_creek, randtag=3 time mean ---- ---- 1970-01-01T00:00:00Z 49.5164137518956 name: h2o_quality tags: location=santa_monica, randtag=1 time mean ---- ---- 1970-01-01T00:00:00Z 50.43829082296367 name: h2o_quality tags: location=santa_monica, randtag=2 time mean ---- ---- 1970-01-01T00:00:00Z 52.0688508894012 name: h2o_quality tags: location=santa_monica, randtag=3 time mean ---- ---- 1970-01-01T00:00:00Z 49.29386362086556 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to calculate the average `index` for every possible[tag](/influxdb/v1/concepts/glossary/#tag) combination in the `h2o_quality`measurement. Note that the query results are identical to the results of the query in [Example 2](#examples-2)where we explicitly specified the `location` and `randtag` tag keys. This is because the `h2o_quality` measurement only has two tag keys. ## GROUP BY time intervals `GROUP BY time()` queries group query results by a user-specified time interval. ### Basic GROUP BY time() syntax #### Syntax ```sql SELECT () FROM_clause WHERE GROUP BY time(),[tag_key] [fill()] ``` Basic `GROUP BY time()` queries require an InfluxQL [function](/influxdb/v1/query_language/functions/)in the [`SELECT` clause](#the-basic-select-statement) and a time range in the[`WHERE` clause](#the-where-clause). Note that the `GROUP BY` clause must come after the `WHERE` clause. ##### `time(time_interval)` The `time_interval` in the `GROUP BY time()` clause is a[duration literal](/influxdb/v1/query_language/spec/#durations). It determines how InfluxDB groups query results over time. For example, a `time_interval` of `5m` groups query results into five-minute time groups across the time range specified in the [`WHERE` clause](#the-where-clause). ##### `fill()` `fill()` is optional. It changes the value reported for time intervals that have no data. See [GROUP BY time intervals and `fill()`](#group-by-time-intervals-and-fill)for more information. **Coverage:** Basic `GROUP BY time()` queries rely on the `time_interval` and on the InfluxDB database’s preset time boundaries to determine the raw data included in each time interval and the timestamps returned by the query. #### Examples of basic syntax The examples below use the following subsample of the sample data: ``` > SELECT "water_level","location" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' ## name: h2o_feet time water_level location 2015-08-18T00:00:00Z 8.12 coyote_creek 2015-08-18T00:00:00Z 2.064 santa_monica 2015-08-18T00:06:00Z 8.005 coyote_creek 2015-08-18T00:06:00Z 2.116 santa_monica 2015-08-18T00:12:00Z 7.887 coyote_creek 2015-08-18T00:12:00Z 2.028 santa_monica 2015-08-18T00:18:00Z 7.762 coyote_creek 2015-08-18T00:18:00Z 2.126 santa_monica 2015-08-18T00:24:00Z 7.635 coyote_creek 2015-08-18T00:24:00Z 2.041 santa_monica 2015-08-18T00:30:00Z 7.5 coyote_creek 2015-08-18T00:30:00Z 2.051 santa_monica ``` ##### Group query results into 12 minute intervals ``` > SELECT COUNT("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) ## name: h2o_feet time count 2015-08-18T00:00:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:24:00Z 2 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to count the number of `water_level` points with the [tag](/influxdb/v1/concepts/glossary/#tag)`location = coyote_creek` and it group results into 12 minute intervals. The result for each [timestamp](/influxdb/v1/concepts/glossary/#timestamp)represents a single 12 minute interval. The count for the first timestamp covers the raw data between `2015-08-18T00:00:00Z`and up to, but not including, `2015-08-18T00:12:00Z`. The count for the second timestamp covers the raw data between `2015-08-18T00:12:00Z`and up to, but not including, `2015-08-18T00:24:00Z.` ##### Group query results into 12 minutes intervals and by a tag key ```sql > SELECT COUNT("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m),"location" name: h2o_feet tags: location=coyote_creek time count ---- ----- 2015-08-18T00:00:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:24:00Z 2 name: h2o_feet tags: location=santa_monica time count ---- ----- 2015-08-18T00:00:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:24:00Z 2 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to count the number of `water_level` points. It groups results by the `location` tag and into 12 minute intervals. Note that the time interval and the tag key are separated by a comma in the`GROUP BY` clause. The query returns two [series](/influxdb/v1/concepts/glossary/#series) of results: one for each[tag value](/influxdb/v1/concepts/glossary/#tag-value) of the `location` tag. The result for each timestamp represents a single 12 minute interval. The count for the first timestamp covers the raw data between `2015-08-18T00:00:00Z`and up to, but not including, `2015-08-18T00:12:00Z`. The count for the second timestamp covers the raw data between `2015-08-18T00:12:00Z`and up to, but not including, `2015-08-18T00:24:00Z.` #### Common issues with basic syntax ##### Unexpected timestamps and values in query results With the basic syntax, InfluxDB relies on the `GROUP BY time()` interval and on the system’s preset time boundaries to determine the raw data included in each time interval and the timestamps returned by the query. In some cases, this can lead to unexpected results. **Example** Raw data: ``` > SELECT "water_level" FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:18:00Z' ## name: h2o_feet time water_level 2015-08-18T00:00:00Z 8.12 2015-08-18T00:06:00Z 8.005 2015-08-18T00:12:00Z 7.887 2015-08-18T00:18:00Z 7.762 ``` Query and results: The following query covers a 12-minute time range and groups results into 12-minute time intervals, but it returns **two** results: ```sql > SELECT COUNT("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:06:00Z' AND time < '2015-08-18T00:18:00Z' GROUP BY time(12m) name: h2o_feet time count ---- ----- 2015-08-18T00:00:00Z 1 <----- Note that this timestamp occurs before the start of the query's time range 2015-08-18T00:12:00Z 1 ``` Explanation: InfluxDB uses preset round-number time boundaries for `GROUP BY` intervals that are independent of any time conditions in the `WHERE` clause. When it calculates the results, all returned data must occur within the query’s explicit time range but the `GROUP BY` intervals will be based on the preset time boundaries. The table below shows the preset time boundary, the relevant `GROUP BY time()` interval, the points included, and the returned timestamp for each `GROUP BY time()`interval in the results. |Time Interval Number| Preset Time Boundary | `GROUP BY time()` Interval |Points Included| Returned Timestamp | |--------------------|--------------------------------------------------------------|--------------------------------------------------------------|---------------|----------------------| | 1 |`time >= 2015-08-18T00:00:00Z AND time < 2015-08-18T00:12:00Z`|`time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:12:00Z`| `8.005` |`2015-08-18T00:00:00Z`| | 2 |`time >= 2015-08-12T00:12:00Z AND time < 2015-08-18T00:24:00Z`|`time >= 2015-08-12T00:12:00Z AND time < 2015-08-18T00:18:00Z`| `7.887` |`2015-08-18T00:12:00Z`| The first preset 12-minute time boundary begins at `00:00` and ends just before`00:12`. Only one raw point (`8.005`) falls both within the query’s first `GROUP BY time()` interval and in that first time boundary. Note that while the returned timestamp occurs before the start of the query’s time range, the query result excludes data that occur before the query’s time range. The second preset 12-minute time boundary begins at `00:12` and ends just before`00:24`. Only one raw point (`7.887`) falls both within the query’s second `GROUP BY time()` interval and in that second time boundary. The [advanced `GROUP BY time()` syntax](#advanced-group-by-time-syntax) allows users to shift the start time of the InfluxDB database’s preset time boundaries.[Example 3](#examples-3)in the Advanced Syntax section continues with the query shown here; it shifts forward the preset time boundaries by six minutes such that InfluxDB returns: ```sql name: h2o_feet time count ---- ----- 2015-08-18T00:06:00Z 2 ``` ### Advanced GROUP BY time() syntax #### Syntax ```sql SELECT () FROM_clause WHERE GROUP BY time(,),[tag_key] [fill()] ``` Advanced `GROUP BY time()` queries require an InfluxQL [function](/influxdb/v1/query_language/functions/)in the [`SELECT` clause](#the-basic-select-statement) and a time range in the[`WHERE` clause](#the-where-clause). Note that the `GROUP BY` clause must come after the `WHERE` clause. ##### `time(time_interval,offset_interval)` See the [Basic GROUP BY time() Syntax](#basic-group-by-time-syntax)for details on the `time_interval`. The `offset_interval` is a[duration literal](/influxdb/v1/query_language/spec/#durations). It shifts forward or back the InfluxDB database’s preset time boundaries. The `offset_interval` can be positive or negative. ##### `fill()` `fill()` is optional. It changes the value reported for time intervals that have no data. See [GROUP BY time intervals and `fill()`](#group-by-time-intervals-and-fill)for more information. **Coverage:** Advanced `GROUP BY time()` queries rely on the `time_interval`, the `offset_interval`, and on the InfluxDB database’s preset time boundaries to determine the raw data included in each time interval and the timestamps returned by the query. #### Examples of advanced syntax The examples below use the following subsample of the sample data: ``` > SELECT "water_level" FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:54:00Z' ## name: h2o_feet time water_level 2015-08-18T00:00:00Z 8.12 2015-08-18T00:06:00Z 8.005 2015-08-18T00:12:00Z 7.887 2015-08-18T00:18:00Z 7.762 2015-08-18T00:24:00Z 7.635 2015-08-18T00:30:00Z 7.5 2015-08-18T00:36:00Z 7.372 2015-08-18T00:42:00Z 7.234 2015-08-18T00:48:00Z 7.11 2015-08-18T00:54:00Z 6.982 ``` ##### Group query results into 18 minute intervals and shift the preset time boundaries forward ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:06:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(18m,6m) name: h2o_feet time mean ---- ---- 2015-08-18T00:06:00Z 7.884666666666667 2015-08-18T00:24:00Z 7.502333333333333 2015-08-18T00:42:00Z 7.108666666666667 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to calculate the average `water_level`, grouping results into 18 minute time intervals, and offsetting the preset time boundaries by six minutes. The time boundaries and returned timestamps for the query **without** the `offset_interval` adhere to the InfluxDB database’s preset time boundaries. Let’s first examine the results without the offset: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:06:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(18m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 7.946 2015-08-18T00:18:00Z 7.6323333333333325 2015-08-18T00:36:00Z 7.238666666666667 2015-08-18T00:54:00Z 6.982 ``` The time boundaries and returned timestamps for the query **without** the`offset_interval` adhere to the InfluxDB database’s preset time boundaries: |Time Interval Number| Preset Time Boundary | `GROUP BY time()` Interval | Points Included | Returned Timestamp | |--------------------|--------------------------------------------------------------|--------------------------------------------------------------|----------------------|----------------------| | 1 |`time >= 2015-08-18T00:00:00Z AND time < 2015-08-18T00:18:00Z`|`time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:18:00Z`| `8.005`,`7.887` |`2015-08-18T00:00:00Z`| | 2 |`time >= 2015-08-18T00:18:00Z AND time < 2015-08-18T00:36:00Z`| \<— same |`7.762`,`7.635`,`7.5` |`2015-08-18T00:18:00Z`| | 3 |`time >= 2015-08-18T00:36:00Z AND time < 2015-08-18T00:54:00Z`| \<— same |`7.372`,`7.234`,`7.11`|`2015-08-18T00:36:00Z`| | 4 |`time >= 2015-08-18T00:54:00Z AND time < 2015-08-18T01:12:00Z`| `time = 2015-08-18T00:54:00Z` | `6.982` |`2015-08-18T00:54:00Z`| The first preset 18-minute time boundary begins at `00:00` and ends just before`00:18`. Two raw points (`8.005` and `7.887`) fall both within the first `GROUP BY time()` interval and in that first time boundary. Note that while the returned timestamp occurs before the start of the query’s time range, the query result excludes data that occur before the query’s time range. The second preset 18-minute time boundary begins at `00:18` and ends just before`00:36`. Three raw points (`7.762` and `7.635` and `7.5`) fall both within the second `GROUP BY time()` interval and in that second time boundary. In this case, the boundary time range and the interval’s time range are the same. The fourth preset 18-minute time boundary begins at `00:54` and ends just before`1:12:00`. One raw point (`6.982`) falls both within the fourth `GROUP BY time()` interval and in that fourth time boundary. The time boundaries and returned timestamps for the query **with** the`offset_interval` adhere to the offset time boundaries: |Time Interval Number| Offset Time Boundary |`GROUP BY time()` Interval| Points Included | Returned Timestamp | |--------------------|--------------------------------------------------------------|--------------------------|-----------------------|----------------------| | 1 |`time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:24:00Z`| \<— same |`8.005`,`7.887`,`7.762`|`2015-08-18T00:06:00Z`| | 2 |`time >= 2015-08-18T00:24:00Z AND time < 2015-08-18T00:42:00Z`| \<— same | `7.635`,`7.5`,`7.372` |`2015-08-18T00:24:00Z`| | 3 |`time >= 2015-08-18T00:42:00Z AND time < 2015-08-18T01:00:00Z`| \<— same |`7.234`,`7.11`,`6.982` |`2015-08-18T00:42:00Z`| | 4 |`time >= 2015-08-18T01:00:00Z AND time < 2015-08-18T01:18:00Z`| NA | NA | NA | The six-minute offset interval shifts forward the preset boundary’s time range such that the boundary time ranges and the relevant `GROUP BY time()` interval time ranges are always the same. With the offset, each interval performs the calculation on three points, and the timestamp returned matches both the start of the boundary time range and the start of the `GROUP BY time()` interval time range. Note that `offset_interval` forces the fourth time boundary to be outside the query’s time range so the query returns no results for that last interval. ##### Group query results into 12 minute intervals and shift the preset time boundaries back ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:06:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(18m,-12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:06:00Z 7.884666666666667 2015-08-18T00:24:00Z 7.502333333333333 2015-08-18T00:42:00Z 7.108666666666667 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to calculate the average `water_level`, grouping results into 18 minute time intervals, and offsetting the preset time boundaries by -12 minutes. > > > **Note:** The query in Example 2 returns the same results as the query in Example 1, but > the query in Example 2 uses a negative `offset_interval` instead of a positive`offset_interval`. > There are no performance differences between the two queries; feel free to choose the most > intuitive option when deciding between a positive and negative `offset_interval`. > > The time boundaries and returned timestamps for the query **without** the `offset_interval` adhere to InfluxDB database’s preset time boundaries. Let’s first examine the results without the offset: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:06:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(18m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 7.946 2015-08-18T00:18:00Z 7.6323333333333325 2015-08-18T00:36:00Z 7.238666666666667 2015-08-18T00:54:00Z 6.982 ``` The time boundaries and returned timestamps for the query **without** the`offset_interval` adhere to the InfluxDB database’s preset time boundaries: |Time Interval Number| Preset Time Boundary | `GROUP BY time()` Interval | Points Included | Returned Timestamp | |--------------------|--------------------------------------------------------------|--------------------------------------------------------------|----------------------|----------------------| | 1 |`time >= 2015-08-18T00:00:00Z AND time < 2015-08-18T00:18:00Z`|`time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:18:00Z`| `8.005`,`7.887` |`2015-08-18T00:00:00Z`| | 2 |`time >= 2015-08-18T00:18:00Z AND time < 2015-08-18T00:36:00Z`| \<— same |`7.762`,`7.635`,`7.5` |`2015-08-18T00:18:00Z`| | 3 |`time >= 2015-08-18T00:36:00Z AND time < 2015-08-18T00:54:00Z`| \<— same |`7.372`,`7.234`,`7.11`|`2015-08-18T00:36:00Z`| | 4 |`time >= 2015-08-18T00:54:00Z AND time < 2015-08-18T01:12:00Z`| `time = 2015-08-18T00:54:00Z` | `6.982` |`2015-08-18T00:54:00Z`| The first preset 18-minute time boundary begins at `00:00` and ends just before`00:18`. Two raw points (`8.005` and `7.887`) fall both within the first `GROUP BY time()` interval and in that first time boundary. Note that while the returned timestamp occurs before the start of the query’s time range, the query result excludes data that occur before the query’s time range. The second preset 18-minute time boundary begins at `00:18` and ends just before`00:36`. Three raw points (`7.762` and `7.635` and `7.5`) fall both within the second `GROUP BY time()` interval and in that second time boundary. In this case, the boundary time range and the interval’s time range are the same. The fourth preset 18-minute time boundary begins at `00:54` and ends just before`1:12:00`. One raw point (`6.982`) falls both within the fourth `GROUP BY time()` interval and in that fourth time boundary. The time boundaries and returned timestamps for the query **with** the`offset_interval` adhere to the offset time boundaries: |Time Interval Number| Offset Time Boundary |`GROUP BY time()` Interval| Points Included | Returned Timestamp | |--------------------|--------------------------------------------------------------|--------------------------|-----------------------|----------------------| | 1 |`time >= 2015-08-17T23:48:00Z AND time < 2015-08-18T00:06:00Z`| NA | NA | NA | | 2 |`time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:24:00Z`| \<— same |`8.005`,`7.887`,`7.762`|`2015-08-18T00:06:00Z`| | 3 |`time >= 2015-08-18T00:24:00Z AND time < 2015-08-18T00:42:00Z`| \<— same | `7.635`,`7.5`,`7.372` |`2015-08-18T00:24:00Z`| | 4 |`time >= 2015-08-18T00:42:00Z AND time < 2015-08-18T01:00:00Z`| \<— same |`7.234`,`7.11`,`6.982` |`2015-08-18T00:42:00Z`| The negative 12-minute offset interval shifts back the preset boundary’s time range such that the boundary time ranges and the relevant `GROUP BY time()` interval time ranges are always the same. With the offset, each interval performs the calculation on three points, and the timestamp returned matches both the start of the boundary time range and the start of the `GROUP BY time()` interval time range. Note that `offset_interval` forces the first time boundary to be outside the query’s time range so the query returns no results for that first interval. ##### Group query results into 12 minute intervals and shift the preset time boundaries forward This example is a continuation of the scenario outlined in [Common Issues with Basic Syntax](#common-issues-with-basic-syntax). ```sql > SELECT COUNT("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:06:00Z' AND time < '2015-08-18T00:18:00Z' GROUP BY time(12m,6m) name: h2o_feet time count ---- ----- 2015-08-18T00:06:00Z 2 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to count the number of `water_level` points, grouping results into 12 minute time intervals, and offsetting the preset time boundaries by six minutes. The time boundaries and returned timestamps for the query **without** the `offset_interval` adhere to InfluxDB database’s preset time boundaries. Let’s first examine the results without the offset: ```sql > SELECT COUNT("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-08-18T00:06:00Z' AND time < '2015-08-18T00:18:00Z' GROUP BY time(12m) name: h2o_feet time count ---- ----- 2015-08-18T00:00:00Z 1 2015-08-18T00:12:00Z 1 ``` The time boundaries and returned timestamps for the query **without** the`offset_interval` adhere to InfluxDB database’s preset time boundaries: |Time Interval Number| Preset Time Boundary | `GROUP BY time()` Interval |Points Included| Returned Timestamp | |--------------------|--------------------------------------------------------------|--------------------------------------------------------------|---------------|----------------------| | 1 |`time >= 2015-08-18T00:00:00Z AND time < 2015-08-18T00:12:00Z`|`time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:12:00Z`| `8.005` |`2015-08-18T00:00:00Z`| | 2 |`time >= 2015-08-12T00:12:00Z AND time < 2015-08-18T00:24:00Z`|`time >= 2015-08-12T00:12:00Z AND time < 2015-08-18T00:18:00Z`| `7.887` |`2015-08-18T00:12:00Z`| The first preset 12-minute time boundary begins at `00:00` and ends just before`00:12`. Only one raw point (`8.005`) falls both within the query’s first `GROUP BY time()` interval and in that first time boundary. Note that while the returned timestamp occurs before the start of the query’s time range, the query result excludes data that occur before the query’s time range. The second preset 12-minute time boundary begins at `00:12` and ends just before`00:24`. Only one raw point (`7.887`) falls both within the query’s second `GROUP BY time()` interval and in that second time boundary. The time boundaries and returned timestamps for the query **with** the`offset_interval` adhere to the offset time boundaries: |Time Interval Number| Offset Time Boundary |`GROUP BY time()` Interval|Points Included| Returned Timestamp | |--------------------|--------------------------------------------------------------|--------------------------|---------------|----------------------| | 1 |`time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:18:00Z`| \<— same |`8.005`,`7.887`|`2015-08-18T00:06:00Z`| | 2 |`time >= 2015-08-18T00:18:00Z AND time < 2015-08-18T00:30:00Z`| NA | NA | NA | The six-minute offset interval shifts forward the preset boundary’s time range such that the preset boundary time range and the relevant `GROUP BY time()` interval time range are the same. With the offset, the query returns a single result, and the timestamp returned matches both the start of the boundary time range and the start of the `GROUP BY time()` interval time range. Note that `offset_interval` forces the second time boundary to be outside the query’s time range so the query returns no results for that second interval. ## `GROUP BY` time intervals and `fill()` `fill()` changes the value reported for time intervals that have no data. #### Syntax ```sql SELECT () FROM_clause WHERE GROUP BY time(time_interval,[)] ``` By default, a `GROUP BY time()` interval with no data reports `null` as its value in the output column.`fill()` changes the value reported for time intervals that have no data. Note that `fill()` must go at the end of the `GROUP BY` clause if you’re`GROUP(ing) BY` several things (for example, both [tags](/influxdb/v1/concepts/glossary/#tag) and a time interval). ##### fill\_option Any numerical value Reports the given numerical value for time intervals with no data. `linear` Reports the results of [linear interpolation](https://en.wikipedia.org/wiki/Linear_interpolation) for time intervals with no data. `none` Reports no timestamp and no value for time intervals with no data. `null` Reports null for time intervals with no data but returns a timestamp. This is the same as the default behavior. `previous` Reports the value from the previous time interval for time intervals with no data. #### Examples [Example 1: fill(100)](#example-1-fill100)[Example 2: fill(linear)](#example-2-filllinear)[Example 3: fill(none)](#example-3-fillnone)[Example 4: fill(null)](#example-4-fillnull)[Example 5: fill(previous)](#example-5-fillprevious) Without `fill(100)`: ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-09-18T16:00:00Z' AND time <= '2015-09-18T16:42:00Z' GROUP BY time(12m) ## name: h2o_feet time max 2015-09-18T16:00:00Z 3.599 2015-09-18T16:12:00Z 3.402 2015-09-18T16:24:00Z 3.235 2015-09-18T16:36:00Z ``` With `fill(100)`: ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-09-18T16:00:00Z' AND time <= '2015-09-18T16:42:00Z' GROUP BY time(12m) fill(100) ## name: h2o_feet time max 2015-09-18T16:00:00Z 3.599 2015-09-18T16:12:00Z 3.402 2015-09-18T16:24:00Z 3.235 2015-09-18T16:36:00Z 100 ``` `fill(100)` changes the value reported for the time interval with no data to `100`. Without `fill(linear)`: ```sql > SELECT MEAN("tadpoles") FROM "pond" WHERE time >= '2016-11-11T21:00:00Z' AND time <= '2016-11-11T22:06:00Z' GROUP BY time(12m) name: pond time mean ---- ---- 2016-11-11T21:00:00Z 1 2016-11-11T21:12:00Z 2016-11-11T21:24:00Z 3 2016-11-11T21:36:00Z 2016-11-11T21:48:00Z 2016-11-11T22:00:00Z 6 ``` With `fill(linear)`: ```sql > SELECT MEAN("tadpoles") FROM "pond" WHERE time >= '2016-11-11T21:00:00Z' AND time <= '2016-11-11T22:06:00Z' GROUP BY time(12m) fill(linear) name: pond time mean ---- ---- 2016-11-11T21:00:00Z 1 2016-11-11T21:12:00Z 2 2016-11-11T21:24:00Z 3 2016-11-11T21:36:00Z 4 2016-11-11T21:48:00Z 5 2016-11-11T22:00:00Z 6 ``` `fill(linear)` changes the value reported for the time interval with no data to the results of [linear interpolation](https://en.wikipedia.org/wiki/Linear_interpolation). > > > **Note:** The data in Example 2 are not in `NOAA_water_database`. > We had to create a dataset with less regular data to work with `fill(linear)`. > > Without `fill(none)`: ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-09-18T16:00:00Z' AND time <= '2015-09-18T16:42:00Z' GROUP BY time(12m) ## name: h2o_feet time max 2015-09-18T16:00:00Z 3.599 2015-09-18T16:12:00Z 3.402 2015-09-18T16:24:00Z 3.235 2015-09-18T16:36:00Z ``` With `fill(none)`: ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-09-18T16:00:00Z' AND time <= '2015-09-18T16:42:00Z' GROUP BY time(12m) fill(none) ## name: h2o_feet time max 2015-09-18T16:00:00Z 3.599 2015-09-18T16:12:00Z 3.402 2015-09-18T16:24:00Z 3.235 ``` `fill(none)` reports no value and no timestamp for the time interval with no data. Without `fill(null)`: ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-09-18T16:00:00Z' AND time <= '2015-09-18T16:42:00Z' GROUP BY time(12m) ## name: h2o_feet time max 2015-09-18T16:00:00Z 3.599 2015-09-18T16:12:00Z 3.402 2015-09-18T16:24:00Z 3.235 2015-09-18T16:36:00Z ``` With `fill(null)`: ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-09-18T16:00:00Z' AND time <= '2015-09-18T16:42:00Z' GROUP BY time(12m) fill(null) ## name: h2o_feet time max 2015-09-18T16:00:00Z 3.599 2015-09-18T16:12:00Z 3.402 2015-09-18T16:24:00Z 3.235 2015-09-18T16:36:00Z ``` `fill(null)` reports `null` as the value for the time interval with no data. That result matches the result of the query without `fill(null)`. Without `fill(previous)`: ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-09-18T16:00:00Z' AND time <= '2015-09-18T16:42:00Z' GROUP BY time(12m) ## name: h2o_feet time max 2015-09-18T16:00:00Z 3.599 2015-09-18T16:12:00Z 3.402 2015-09-18T16:24:00Z 3.235 2015-09-18T16:36:00Z ``` With `fill(previous)`: ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE "location"='coyote_creek' AND time >= '2015-09-18T16:00:00Z' AND time <= '2015-09-18T16:42:00Z' GROUP BY time(12m) fill(previous) ## name: h2o_feet time max 2015-09-18T16:00:00Z 3.599 2015-09-18T16:12:00Z 3.402 2015-09-18T16:24:00Z 3.235 2015-09-18T16:36:00Z 3.235 ``` `fill(previous)` changes the value reported for the time interval with no data to `3.235`, the value from the previous time interval. #### Common issues with `fill()` ##### Queries with `fill()` when no data fall within the query’s time range Currently, queries ignore `fill()` if no data fall within the query’s time range. This is the expected behavior. An open[feature request](https://github.com/influxdata/influxdb/issues/6967) on GitHub proposes that `fill()` should force a return of values even if the query’s time range covers no data. **Example** The following query returns no data because `water_level` has no points within the query’s time range. Note that `fill(800)` has no effect on the query results. ```sql SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location" = 'coyote_creek' AND time >= '2015-09-18T22:00:00Z' AND time <= '2015-09-18T22:18:00Z' GROUP BY time(12m) fill(800) ``` ##### Queries with `fill(previous)` when the previous result falls outside the query’s time range `fill(previous)` doesn’t fill the result for a time interval if the previous value is outside the query’s time range. **Example** The following query covers the time range between `2015-09-18T16:24:00Z` and `2015-09-18T16:54:00Z`. Note that `fill(previous)` fills the result for `2015-09-18T16:36:00Z` with the result from `2015-09-18T16:24:00Z`. ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE location = 'coyote_creek' AND time >= '2015-09-18T16:24:00Z' AND time <= '2015-09-18T16:54:00Z' GROUP BY time(12m) fill(previous) ## name: h2o_feet time max 2015-09-18T16:24:00Z 3.235 2015-09-18T16:36:00Z 3.235 2015-09-18T16:48:00Z 4 ``` The next query shortens the time range in the previous query. It now covers the time between `2015-09-18T16:36:00Z` and `2015-09-18T16:54:00Z`. Note that `fill(previous)` doesn’t fill the result for `2015-09-18T16:36:00Z` with the result from `2015-09-18T16:24:00Z`; the result for `2015-09-18T16:24:00Z` is outside the query’s shorter time range. ``` > SELECT MAX("water_level") FROM "h2o_feet" WHERE location = 'coyote_creek' AND time >= '2015-09-18T16:36:00Z' AND time <= '2015-09-18T16:54:00Z' GROUP BY time(12m) fill(previous) ## name: h2o_feet time max 2015-09-18T16:36:00Z 2015-09-18T16:48:00Z 4 ``` ##### `fill(linear)` when the previous or following result falls outside the query’s time range `fill(linear)` doesn’t fill the result for a time interval with no data if the previous result or the following result is outside the query’s time range. **Example** The following query covers the time range between `2016-11-11T21:24:00Z` and`2016-11-11T22:06:00Z`. Note that `fill(linear)` fills the results for the`2016-11-11T21:36:00Z` time interval and the `2016-11-11T21:48:00Z` time interval using the values from the `2016-11-11T21:24:00Z` time interval and the`2016-11-11T22:00:00Z` time interval. ```sql > SELECT MEAN("tadpoles") FROM "pond" WHERE time > '2016-11-11T21:24:00Z' AND time <= '2016-11-11T22:06:00Z' GROUP BY time(12m) fill(linear) name: pond time mean ---- ---- 2016-11-11T21:24:00Z 3 2016-11-11T21:36:00Z 4 2016-11-11T21:48:00Z 5 2016-11-11T22:00:00Z 6 ``` The next query shortens the time range in the previous query. It now covers the time between `2016-11-11T21:36:00Z` and `2016-11-11T22:06:00Z`. Note that `fill()` previous doesn’t fill the results for the `2016-11-11T21:36:00Z`time interval and the `2016-11-11T21:48:00Z` time interval; the result for`2016-11-11T21:24:00Z` is outside the query’s shorter time range and InfluxDB cannot perform the linear interpolation. ```sql > SELECT MEAN("tadpoles") FROM "pond" WHERE time >= '2016-11-11T21:36:00Z' AND time <= '2016-11-11T22:06:00Z' GROUP BY time(12m) fill(linear) name: pond time mean ---- ---- 2016-11-11T21:36:00Z 2016-11-11T21:48:00Z 2016-11-11T22:00:00Z 6 ``` > > > **Note:** The data in Issue 3 are not in `NOAA_water_database`. > We had to create a dataset with less regular data to work with `fill(linear)`. > > # The INTO clause The `INTO` clause writes query results to a user-specified [measurement](/influxdb/v1/concepts/glossary/#measurement). ### Syntax ```sql SELECT_clause INTO FROM_clause [WHERE_clause] [GROUP_BY_clause] ``` The `INTO` clause supports several formats for specifying a [measurement](/influxdb/v1/concepts/glossary/#measurement): `INTO ` Writes data to the specified measurement. If you’re using the [CLI](/influxdb/v1/tools/shell/) InfluxDB writes the data to the measurement in the[`USE`d](/influxdb/v1/tools/shell/#commands)[database](/influxdb/v1/concepts/glossary/#database) and the `DEFAULT` [retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp). If you’re using the [InfluxDB API](/influxdb/v1/api/) InfluxDB writes the data to the measurement in the database specified in the [`db` query string parameter](/influxdb/v1/api/query/)and the `DEFAULT` retention policy. `INTO ..` Writes data to a fully qualified measurement. Fully qualify a measurement by specifying its database and retention policy. `INTO ..` Writes data to a measurement in a user-specified database and the `DEFAULT`retention policy. `INTO ..:MEASUREMENT FROM //` Writes data to all measurements in the user-specified database and retention policy that match the [regular expression](#regular-expressions) in the `FROM` clause.`:MEASUREMENT` is a backreference to each measurement matched in the `FROM` clause. ### Examples #### Rename a database ```sql > SELECT * INTO "copy_NOAA_water_database"."autogen".:MEASUREMENT FROM "NOAA_water_database"."autogen"./.*/ GROUP BY * name: result time written ---- ------- 0 76290 ``` Directly renaming a database in InfluxDB is not possible, so a common use for the `INTO` clause is to move data from one database to another. The query above writes all data in the `NOAA_water_database` and `autogen` retention policy to the `copy_NOAA_water_database` database and the `autogen` retention policy. The [backreference](#examples-5) syntax (`:MEASUREMENT`) maintains the source measurement names in the destination database. Note that both the `copy_NOAA_water_database` database and its `autogen` retention policy must exist prior to running the `INTO` query. See [Database Management](/influxdb/v1/query_language/manage-database/)for how to manage databases and retention policies. The `GROUP BY *` clause [preserves tags](#missing-data) in the source database as tags in the destination database. The following query does not maintain the series context for tags; tags will be stored as fields in the destination database (`copy_NOAA_water_database`): ```sql SELECT * INTO "copy_NOAA_water_database"."autogen".:MEASUREMENT FROM "NOAA_water_database"."autogen"./.*/ ``` When moving large amounts of data, to avoid running out of memory, sequentially run `INTO` queries for different measurements and time boundaries. Use the [`WHERE` clause](#time-syntax) to define time boundaries for each query. > [!Note] > `INTO` queries without time boundaries fail with the error: `ERR: no data received`. ##### Move large amounts of data with sequential queries ```sql SELECT * INTO .. FROM .. WHERE time > now() - 100w AND time < now() - 90w GROUP BY * SELECT * INTO .. FROM ..} WHERE time > now() - 90w AND < now() - 80w GROUP BY * SELECT * INTO .. FROM .. WHERE time > now() - 80w AND time < now() - 70w GROUP BY * ``` #### Write the results of a query to a measurement ``` > SELECT "water_level" INTO "h2o_feet_copy_1" FROM "h2o_feet" WHERE "location" = 'coyote_creek' ## name: result time written 1970-01-01T00:00:00Z 7604 > SELECT * FROM "h2o_feet_copy_1" ## name: h2o_feet_copy_1 time water_level 2015-08-18T00:00:00Z 8.12 [...] 2015-09-18T16:48:00Z 4 ``` The query writes its results a new [measurement](/influxdb/v1/concepts/glossary/#measurement): `h2o_feet_copy_1`. If you’re using the [CLI](/influxdb/v1/tools/shell/), InfluxDB writes the data to the `USE`d [database](/influxdb/v1/concepts/glossary/#database) and the `DEFAULT` [retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp). If you’re using the [InfluxDB API](/influxdb/v1/api/), InfluxDB writes the data to the database and retention policy specified in the `db` and `rp`[query string parameters](/influxdb/v1/api/query/). If you do not set the `rp` query string parameter, the InfluxDB API automatically writes the data to the database’s `DEFAULT` retention policy. The response shows the number of points (`7605`) that InfluxDB writes to `h2o_feet_copy_1`. The timestamp in the response is meaningless; InfluxDB uses epoch 0 (`1970-01-01T00:00:00Z`) as a null timestamp equivalent. #### Write the results of a query to a fully qualified measurement ``` > SELECT "water_level" INTO "where_else"."autogen"."h2o_feet_copy_2" FROM "h2o_feet" WHERE "location" = 'coyote_creek' ## name: result time written 1970-01-01T00:00:00Z 7604 > SELECT * FROM "where_else"."autogen"."h2o_feet_copy_2" ## name: h2o_feet_copy_2 time water_level 2015-08-18T00:00:00Z 8.12 [...] 2015-09-18T16:48:00Z 4 ``` The query writes its results to a new measurement: `h2o_feet_copy_2`. InfluxDB writes the data to the `where_else` database and to the `autogen`retention policy. Note that both `where_else` and `autogen` must exist prior to running the `INTO`query. See [Database Management](/influxdb/v1/query_language/manage-database/)for how to manage databases and retention policies. The response shows the number of points (`7605`) that InfluxDB writes to `h2o_feet_copy_2`. The timestamp in the response is meaningless; InfluxDB uses epoch 0 (`1970-01-01T00:00:00Z`) as a null timestamp equivalent. #### Write aggregated results to a measurement (downsampling) ``` > SELECT MEAN("water_level") INTO "all_my_averages" FROM "h2o_feet" WHERE "location" = 'coyote_creek' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) ## name: result time written 1970-01-01T00:00:00Z 3 > SELECT * FROM "all_my_averages" ## name: all_my_averages time mean 2015-08-18T00:00:00Z 8.0625 2015-08-18T00:12:00Z 7.8245 2015-08-18T00:24:00Z 7.5675 ``` The query aggregates data using an InfluxQL [function](/influxdb/v1/query_language/functions) and a [`GROUP BY time()` clause](#group-by-time-intervals). It also writes its results to the `all_my_averages` measurement. The response shows the number of points (`3`) that InfluxDB writes to `all_my_averages`. The timestamp in the response is meaningless; InfluxDB uses epoch 0 (`1970-01-01T00:00:00Z`) as a null timestamp equivalent. The query is an example of downsampling: taking higher precision data, aggregating those data to a lower precision, and storing the lower precision data in the database. Downsampling is a common use case for the `INTO` clause. #### Write aggregated results for more than one measurement to a different database (downsampling with backreferencing) ```sql > SELECT MEAN(*) INTO "where_else"."autogen".:MEASUREMENT FROM /.*/ WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:06:00Z' GROUP BY time(12m) name: result time written ---- ------- 1970-01-01T00:00:00Z 5 > SELECT * FROM "where_else"."autogen"./.*/ name: average_temperature time mean_degrees mean_index mean_pH mean_water_level ---- ------------ ---------- ------- ---------------- 2015-08-18T00:00:00Z 78.5 name: h2o_feet time mean_degrees mean_index mean_pH mean_water_level ---- ------------ ---------- ------- ---------------- 2015-08-18T00:00:00Z 5.07625 name: h2o_pH time mean_degrees mean_index mean_pH mean_water_level ---- ------------ ---------- ------- ---------------- 2015-08-18T00:00:00Z 6.75 name: h2o_quality time mean_degrees mean_index mean_pH mean_water_level ---- ------------ ---------- ------- ---------------- 2015-08-18T00:00:00Z 51.75 name: h2o_temperature time mean_degrees mean_index mean_pH mean_water_level ---- ------------ ---------- ------- ---------------- 2015-08-18T00:00:00Z 63.75 ``` The query aggregates data using an InfluxQL [function](/influxdb/v1/query_language/functions) and a [`GROUP BY time()` clause](#group-by-time-intervals). It aggregates data in every measurement that matches the [regular expression](#regular-expressions)in the `FROM` clause and writes the results to measurements with the same name in the`where_else` database and the `autogen` retention policy. Note that both `where_else` and `autogen` must exist prior to running the `INTO`query. See [Database management](/influxdb/v1/query_language/manage-database/)for how to manage databases and retention policies. The response shows the number of points (`5`) that InfluxDB writes to the `where_else`database and the `autogen` retention policy. The timestamp in the response is meaningless; InfluxDB uses epoch 0 (`1970-01-01T00:00:00Z`) as a null timestamp equivalent. The query is an example of downsampling with backreferencing. It takes higher precision data from more than one measurement, aggregates those data to a lower precision, and stores the lower precision data in the database. Downsampling with backreferencing is a common use case for the `INTO` clause. ### Common issues with the `INTO` clause #### Missing data If an `INTO` query includes a [tag key](/influxdb/v1/concepts/glossary#tag-key) in the [`SELECT` clause](#the-basic-select-statement), the query converts [tags](/influxdb/v1/concepts/glossary#tag) in the current measurement to [fields](/influxdb/v1/concepts/glossary#field) in the destination measurement. This can cause InfluxDB to overwrite [points](/influxdb/v1/concepts/glossary#point) that were previously differentiated by a [tag value](/influxdb/v1/concepts/glossary#tag-value). Note that this behavior does not apply to queries that use the [`TOP()`](/influxdb/v1/query_language/functions/#top) or [`BOTTOM()`](/influxdb/v1/query_language/functions/#bottom) functions. The[Frequently Asked Questions](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-are-my-into-queries-missing-data)document describes that behavior in detail. To preserve tags in the current measurement as tags in the destination measurement,[`GROUP BY` the relevant tag key](#group-by-tags) or `GROUP BY *` in the `INTO` query. #### Automating queries with the `INTO` clause The `INTO` clause section in this document shows how to manually implement queries with an `INTO` clause. See the [Continuous Queries](/influxdb/v1/query_language/continuous_queries/)documentation for how to automate `INTO` clause queries on realtime data. Among [other uses](/influxdb/v1/query_language/continuous_queries/#continuous-query-use-cases), Continuous Queries automate the downsampling process. ## ORDER BY time DESC By default, InfluxDB returns results in ascending time order; the first [point](/influxdb/v1/concepts/glossary/#point)returned has the oldest [timestamp](/influxdb/v1/concepts/glossary/#timestamp) and the last point returned has the most recent timestamp.`ORDER BY time DESC` reverses that order such that InfluxDB returns the points with the most recent timestamps first. ### Syntax ```sql SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] ORDER BY time DESC ``` `ORDER by time DESC` must appear after the [`GROUP BY` clause](#the-group-by-clause)if the query includes a `GROUP BY` clause.`ORDER by time DESC` must appear after the [`WHERE` clause](#the-where-clause)if the query includes a `WHERE` clause and no `GROUP BY` clause. ### Examples #### Return the newest points first ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' ORDER BY time DESC name: h2o_feet time water_level ---- ----------- 2015-09-18T21:42:00Z 4.938 2015-09-18T21:36:00Z 5.066 [...] 2015-08-18T00:06:00Z 2.116 2015-08-18T00:00:00Z 2.064 ``` The query returns the points with the most recent timestamps from the`h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement) first. Without `ORDER by time DESC`, the query would return `2015-08-18T00:00:00Z`first and `2015-09-18T21:42:00Z` last. #### Return the newest points first and include a GROUP BY time() clause ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:42:00Z' GROUP BY time(12m) ORDER BY time DESC name: h2o_feet time mean ---- ---- 2015-08-18T00:36:00Z 4.6825 2015-08-18T00:24:00Z 4.80675 2015-08-18T00:12:00Z 4.950749999999999 2015-08-18T00:00:00Z 5.07625 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions)and a time interval in the [GROUP BY clause](#group-by-time-intervals)to calculate the average `water_level` for each twelve-minute interval in the query’s time range.`ORDER BY time DESC` returns the most recent 12-minute time intervals first. Without `ORDER BY time DESC`, the query would return`2015-08-18T00:00:00Z` first and `2015-08-18T00:36:00Z` last. # The LIMIT and SLIMIT clauses `LIMIT` and `SLIMIT` limit the number of[points](/influxdb/v1/concepts/glossary/#point) and the number of[series](/influxdb/v1/concepts/glossary/#series) returned per query. ## The LIMIT clause `LIMIT ` returns the first `N` [points](/influxdb/v1/concepts/glossary/#point) from the specified [measurement](/influxdb/v1/concepts/glossary/#measurement). ### Syntax ```sql SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] LIMIT ``` `N` specifies the number of [points](/influxdb/v1/concepts/glossary/#point) to return from the specified [measurement](/influxdb/v1/concepts/glossary/#measurement). If `N` is greater than the number of points in a measurement, InfluxDB returns all points from that series. Note that the `LIMIT` clause must appear in the order outlined in the syntax above. ### Examples #### Limit the number of points returned ```sql > SELECT "water_level","location" FROM "h2o_feet" LIMIT 3 name: h2o_feet time water_level location ---- ----------- -------- 2015-08-18T00:00:00Z 8.12 coyote_creek 2015-08-18T00:00:00Z 2.064 santa_monica 2015-08-18T00:06:00Z 8.005 coyote_creek ``` The query returns the three oldest [points](/influxdb/v1/concepts/glossary/#point) (determined by timestamp) from the `h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement). #### Limit the number points returned and include a GROUP BY clause ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:42:00Z' GROUP BY *,time(12m) LIMIT 2 name: h2o_feet tags: location=coyote_creek time mean ---- ---- 2015-08-18T00:00:00Z 8.0625 2015-08-18T00:12:00Z 7.8245 name: h2o_feet tags: location=santa_monica time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions)and a [GROUP BY clause](#group-by-time-intervals)to calculate the average `water_level` for each [tag](/influxdb/v1/concepts/glossary/#tag) and for each twelve-minute interval in the query’s time range.`LIMIT 2` requests the two oldest twelve-minute averages (determined by timestamp). Note that without `LIMIT 2`, the query would return four points per [series](/influxdb/v1/concepts/glossary/#series); one for each twelve-minute interval in the query’s time range. ## The `SLIMIT` clause `SLIMIT ` returns every [point](/influxdb/v1/concepts/glossary/#point) from \ [series](/influxdb/v1/concepts/glossary/#series) in the specified [measurement](/influxdb/v1/concepts/glossary/#measurement). ### Syntax ```sql SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] GROUP BY *[,time()] [ORDER_BY_clause] SLIMIT ``` `N` specifies the number of [series](/influxdb/v1/concepts/glossary/#series) to return from the specified [measurement](/influxdb/v1/concepts/glossary/#measurement). If `N` is greater than the number of series in a measurement, InfluxDB returns all series from that measurement. There is an [ongoing issue](https://github.com/influxdata/influxdb/issues/7571) that requires queries with `SLIMIT` to include `GROUP BY *`. Note that the `SLIMIT` clause must appear in the order outlined in the syntax above. ### Examples #### Limit the number of series returned ```sql > SELECT "water_level" FROM "h2o_feet" GROUP BY * SLIMIT 1 name: h2o_feet tags: location=coyote_creek time water_level ---- ----- 2015-08-18T00:00:00Z 8.12 2015-08-18T00:06:00Z 8.005 2015-08-18T00:12:00Z 7.887 [...] 2015-09-18T16:12:00Z 3.402 2015-09-18T16:18:00Z 3.314 2015-09-18T16:24:00Z 3.235 ``` The query returns all `water_level` [points](/influxdb/v1/concepts/glossary/#point) from one of the [series](/influxdb/v1/concepts/glossary/#series) associated with the `h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement). #### Limit the number of series returned and include a GROUP BY time() clause ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:42:00Z' GROUP BY *,time(12m) SLIMIT 1 name: h2o_feet tags: location=coyote_creek time mean ---- ---- 2015-08-18T00:00:00Z 8.0625 2015-08-18T00:12:00Z 7.8245 2015-08-18T00:24:00Z 7.5675 2015-08-18T00:36:00Z 7.303 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions)and a time interval in the [GROUP BY clause](#group-by-time-intervals)to calculate the average `water_level` for each twelve-minute interval in the query’s time range.`SLIMIT 1` requests a single series associated with the `h2o_feet` measurement. Note that without `SLIMIT 1`, the query would return results for the two series associated with the `h2o_feet` measurement: `location=coyote_creek` and`location=santa_monica`. ## LIMIT and SLIMIT `LIMIT ` followed by `SLIMIT ` returns the first \ [points](/influxdb/v1/concepts/glossary/#point) from \ [series](/influxdb/v1/concepts/glossary/#series) in the specified measurement. ### Syntax ```sql SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] GROUP BY *[,time()] [ORDER_BY_clause] LIMIT SLIMIT ``` `N1` specifies the number of [points](/influxdb/v1/concepts/glossary/#point) to return per [measurement](/influxdb/v1/concepts/glossary/#measurement). If `N1` is greater than the number of points in a measurement, InfluxDB returns all points from that measurement. `N2` specifies the number of series to return from the specified [measurement](/influxdb/v1/concepts/glossary/#measurement). If `N2` is greater than the number of series in a measurement, InfluxDB returns all series from that measurement. There is an [ongoing issue](https://github.com/influxdata/influxdb/issues/7571) that requires queries with `LIMIT` and `SLIMIT` to include `GROUP BY *`. Note that the `LIMIT` and `SLIMIT` clauses must appear in the order outlined in the syntax above. ### Examples #### Limit the number of points and series returned ```sql > SELECT "water_level" FROM "h2o_feet" GROUP BY * LIMIT 3 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time water_level ---- ----------- 2015-08-18T00:00:00Z 8.12 2015-08-18T00:06:00Z 8.005 2015-08-18T00:12:00Z 7.887 ``` The query returns the three oldest [points](/influxdb/v1/concepts/glossary/#point) (determined by timestamp) from one of the [series](/influxdb/v1/concepts/glossary/#series) associated with the[measurement](/influxdb/v1/concepts/glossary/#measurement) `h2o_feet`. #### Limit the number of points and series returned and include a GROUP BY time() clause ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:42:00Z' GROUP BY *,time(12m) LIMIT 2 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time mean ---- ---- 2015-08-18T00:00:00Z 8.0625 2015-08-18T00:12:00Z 7.8245 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions)and a time interval in the [GROUP BY clause](#group-by-time-intervals)to calculate the average `water_level` for each twelve-minute interval in the query’s time range.`LIMIT 2` requests the two oldest twelve-minute averages (determined by timestamp) and `SLIMIT 1` requests a single series associated with the `h2o_feet` measurement. Note that without `LIMIT 2 SLIMIT 1`, the query would return four points for each of the two series associated with the `h2o_feet` measurement. ## The OFFSET and SOFFSET clauses `OFFSET` and `SOFFSET` paginates [points](/influxdb/v1/concepts/glossary/#point) and [series](/influxdb/v1/concepts/glossary/#series) returned. |[The OFFSET clause](#the-offset-clause)|[The SOFFSET clause](#the-soffset-clause)| |---------------------------------------|-----------------------------------------| ## The `OFFSET` clause `OFFSET ` paginates `N` [points](/influxdb/v1/concepts/glossary/#point) in the query results. ### Syntax ```sql SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] LIMIT_clause OFFSET [SLIMIT_clause] ``` `N` specifies the number of [points](/influxdb/v1/concepts/glossary/#point) to paginate. The `OFFSET` clause requires a [`LIMIT` clause](#the-limit-clause). Using the `OFFSET` clause without a `LIMIT` clause can cause [inconsistent query results](https://github.com/influxdata/influxdb/issues/7577). > > > **Note:** InfluxDB returns no results if the `WHERE` clause includes a time > range and the `OFFSET` clause would cause InfluxDB to return points with > timestamps outside of that time range. > > ### Examples #### Paginate points ```sql > SELECT "water_level","location" FROM "h2o_feet" LIMIT 3 OFFSET 3 name: h2o_feet time water_level location ---- ----------- -------- 2015-08-18T00:06:00Z 2.116 santa_monica 2015-08-18T00:12:00Z 7.887 coyote_creek 2015-08-18T00:12:00Z 2.028 santa_monica ``` The query returns the fourth, fifth, and sixth [points](/influxdb/v1/concepts/glossary/#point) from the `h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement). If the query did not include `OFFSET 3`, it would return the first, second, and third points from that measurement. #### Paginate points and include several clauses ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:42:00Z' GROUP BY *,time(12m) ORDER BY time DESC LIMIT 2 OFFSET 2 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time mean ---- ---- 2015-08-18T00:12:00Z 7.8245 2015-08-18T00:00:00Z 8.0625 ``` This example is pretty involved, so here’s the clause-by-clause breakdown: The [`SELECT` clause](#the-basic-select-statement) specifies an InfluxQL [function](/influxdb/v1/query_language/functions). The [`FROM` clause](#the-basic-select-statement) specifies a single measurement. The [`WHERE` clause](#the-where-clause) specifies the time range for the query. The [`GROUP BY` clause](#the-group-by-clause) groups results by all tags (`*`) and into 12-minute intervals. The [`ORDER BY time DESC` clause](#order-by-time-desc) returns results in descending timestamp order. The [`LIMIT 2` clause](#the-limit-clause) limits the number of points returned to two. The `OFFSET 2` clause excludes the first two averages from the query results. The [`SLIMIT 1` clause](#the-slimit-clause) limits the number of series returned to one. Without `OFFSET 2`, the query would return the first two averages of the query results: ```sql name: h2o_feet tags: location=coyote_creek time mean ---- ---- 2015-08-18T00:36:00Z 7.303 2015-08-18T00:24:00Z 7.5675 ``` ## The `SOFFSET` clause `SOFFSET ` paginates `N` [series](/influxdb/v1/concepts/glossary/#series) in the query results. ### Syntax ```sql SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] GROUP BY *[,time(time_interval)] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] SLIMIT_clause SOFFSET ``` `N` specifies the number of [series](/influxdb/v1/concepts/glossary/#series) to paginate. The `SOFFSET` clause requires an [`SLIMIT` clause](#the-slimit-clause). Using the `SOFFSET` clause without an `SLIMIT` clause can cause [inconsistent query results](https://github.com/influxdata/influxdb/issues/7578). There is an [ongoing issue](https://github.com/influxdata/influxdb/issues/7571) that requires queries with `SLIMIT` to include `GROUP BY *`. > > > **Note:** InfluxDB returns no results if the `SOFFSET` clause paginates > through more than the total number of series. > > ### Examples #### Paginate series ```sql > SELECT "water_level" FROM "h2o_feet" GROUP BY * SLIMIT 1 SOFFSET 1 name: h2o_feet tags: location=santa_monica time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 [...] 2015-09-18T21:36:00Z 5.066 2015-09-18T21:42:00Z 4.938 ``` The query returns data for the [series](/influxdb/v1/concepts/glossary/#series) associated with the `h2o_feet`[measurement](/influxdb/v1/concepts/glossary/#measurement) and the `location = santa_monica` [tag](/influxdb/v1/concepts/glossary/#tag). Without `SOFFSET 1`, the query returns data for the series associated with the`h2o_feet` measurement and the `location = coyote_creek` tag. #### Paginate series and include all clauses ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:42:00Z' GROUP BY *,time(12m) ORDER BY time DESC LIMIT 2 OFFSET 2 SLIMIT 1 SOFFSET 1 name: h2o_feet tags: location=santa_monica time mean ---- ---- 2015-08-18T00:12:00Z 2.077 2015-08-18T00:00:00Z 2.09 ``` This example is pretty involved, so here’s the clause-by-clause breakdown: The [`SELECT` clause](#the-basic-select-statement) specifies an InfluxQL [function](/influxdb/v1/query_language/functions). The [`FROM` clause](#the-basic-select-statement) specifies a single measurement. The [`WHERE` clause](#the-where-clause) specifies the time range for the query. The [`GROUP BY` clause](#the-group-by-clause) groups results by all tags (`*`) and into 12-minute intervals. The [`ORDER BY time DESC` clause](#order-by-time-desc) returns results in descending timestamp order. The [`LIMIT 2` clause](#the-limit-clause) limits the number of points returned to two. The [`OFFSET 2` clause](#the-offset-clause) excludes the first two averages from the query results. The [`SLIMIT 1` clause](#the-slimit-clause) limits the number of series returned to one. The `SOFFSET 1` clause paginates the series returned. Without `SOFFSET 1`, the query would return the results for a different series: ```sql name: h2o_feet tags: location=coyote_creek time mean ---- ---- 2015-08-18T00:12:00Z 7.8245 2015-08-18T00:00:00Z 8.0625 ``` ## The Time Zone clause The `tz()` clause returns the UTC offset for the specified timezone. ### Syntax ```sql SELECT_clause [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] tz('') ``` By default, InfluxDB stores and returns timestamps in UTC. The `tz()` clause includes the UTC offset or, if applicable, the UTC Daylight Savings Time (DST) offset to the query’s returned timestamps. The returned timestamps must be in [RFC3339 format](/influxdb/v1/query_language/explore-data/#configuring-the-returned-timestamps) for the UTC offset or UTC DST to appear. The `time_zone` parameter follows the TZ syntax in the [Internet Assigned Numbers Authority time zone database](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List) and it requires single quotes. ### Examples #### Return the UTC offset for Chicago’s time zone ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:18:00Z' tz('America/Chicago') name: h2o_feet time water_level ---- ----------- 2015-08-17T19:00:00-05:00 2.064 2015-08-17T19:06:00-05:00 2.116 2015-08-17T19:12:00-05:00 2.028 2015-08-17T19:18:00-05:00 2.126 ``` The query results include the UTC offset (`-05:00`) for the `America/Chicago` time zone in the timestamps. ## Time syntax For most `SELECT` statements, the default time range is between [`1677-09-21 00:12:43.145224194` and `2262-04-11T23:47:16.854775806Z` UTC](/influxdb/v1/troubleshooting/frequently-asked-questions/#what-are-the-minimum-and-maximum-timestamps-that-influxdb-can-store). For `SELECT` statements with a [`GROUP BY time()` clause](#group-by-time-intervals), the default time range is between `1677-09-21 00:12:43.145224194` UTC and [`now()`](/influxdb/v1/concepts/glossary/#now). The following sections detail how to specify alternative time ranges in the `SELECT`statement’s [`WHERE` clause](#the-where-clause). |[Absolute time](#absolute-time)|[Relative time](#relative-time)|[Common issues with time syntax](#common-issues-with-time-syntax)| |-------------------------------|-------------------------------|-----------------------------------------------------------------| Tired of reading? Check out this InfluxQL Short: ## Absolute time Specify absolute time with date-time strings and epoch time. ### Syntax ```sql SELECT_clause FROM_clause WHERE time ['' | '' | ] [AND ['' | '' | ] [...]] ``` #### Supported operators |Operator| Meaning | |--------|------------------------| | `=` | equal to | | `<>` | not equal to | | `!=` | not equal to | | `>` | greater than | | `>=` |greater than or equal to| | `<` | less than | | `<=` | less than or equal to | Currently, InfluxDB does not support using `OR` with absolute time in the `WHERE`clause. See the [Frequently Asked Questions](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-is-my-query-with-a-where-or-time-clause-returning-empty-results)document and the [GitHub Issue](https://github.com/influxdata/influxdb/issues/7530)for more information. #### `rfc3339_date_time_string` ```sql 'YYYY-MM-DDTHH:MM:SS.nnnnnnnnnZ' ``` `.nnnnnnnnn` is optional and is set to `.000000000` if not included. The [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) date-time string requires single quotes. #### `rfc3339_like_date_time_string` ```sql 'YYYY-MM-DD HH:MM:SS.nnnnnnnnn' ``` `HH:MM:SS.nnnnnnnnn.nnnnnnnnn` is optional and is set to `00:00:00.000000000` if not included. The RFC3339-like date-time string requires single quotes. #### `epoch_time` Epoch time is the amount of time that has elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970. By default, InfluxDB assumes that all epoch timestamps are in nanoseconds. Include a [duration literal](/influxdb/v1/query_language/spec/#durations)at the end of the epoch timestamp to indicate a precision other than nanoseconds. #### Basic arithmetic All timestamp formats support basic arithmetic. Add (`+`) or subtract (`-`) a time from a timestamp with a [duration literal](/influxdb/v1/query_language/spec/#durations). Note that InfluxQL requires a whitespace between the `+` or `-` and the duration literal. ### Examples #### Specify a time range with RFC3339 date-time strings ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00.000000000Z' AND time <= '2015-08-18T00:12:00Z' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 ``` The query returns data with timestamps between August 18, 2015 at 00:00:00.000000000 and August 18, 2015 at 00:12:00. The nanosecond specification in the first timestamp (`.000000000`) is optional. Note that the single quotes around the RFC3339 date-time strings are required. #### Specify a time range with RFC3339-like date-time strings ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18' AND time <= '2015-08-18 00:12:00' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 ``` The query returns data with timestamps between August 18, 2015 at 00:00:00 and August 18, 2015 at 00:12:00. The first date-time string does not include a time; InfluxDB assumes the time is 00:00:00. Note that the single quotes around the RFC3339-like date-time strings are required. #### Specify a time range with epoch timestamps ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= 1439856000000000000 AND time <= 1439856720000000000 name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 ``` The query returns data with timestamps that occur between August 18, 2015 at 00:00:00 and August 18, 2015 at 00:12:00. By default InfluxDB assumes epoch timestamps are in nanoseconds. #### Specify a time range with second-precision epoch timestamps ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= 1439856000s AND time <= 1439856720s name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 ``` The query returns data with timestamps that occur between August 18, 2015 at 00:00:00 and August 18, 2015 at 00:12:00. The `s` [duration literal](/influxdb/v1/query_language/spec/#durations) at the end of the epoch timestamps indicate that the epoch timestamps are in seconds. #### Perform basic arithmetic on an RFC3339-like date-time string ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time > '2015-09-18T21:24:00Z' + 6m name: h2o_feet time water_level ---- ----------- 2015-09-18T21:36:00Z 5.066 2015-09-18T21:42:00Z 4.938 ``` The query returns data with timestamps that occur at least six minutes after September 18, 2015 at 21:24:00. Note that the whitespace between the `+` and `6m` is required. #### Perform basic arithmetic on an epoch timestamp ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time > 24043524m - 6m name: h2o_feet time water_level ---- ----------- 2015-09-18T21:24:00Z 5.013 2015-09-18T21:30:00Z 5.01 2015-09-18T21:36:00Z 5.066 2015-09-18T21:42:00Z 4.938 ``` The query returns data with timestamps that occur at least six minutes before September 18, 2015 at 21:24:00. Note that the whitespace between the `-` and `6m` is required. ## Relative time Use [`now()`](/influxdb/v1/concepts/glossary/#now) to query data with [timestamps](/influxdb/v1/concepts/glossary/#timestamp) relative to the server’s current timestamp. ### Syntax ```sql SELECT_clause FROM_clause WHERE time now() [[ - | + ] ] [(AND|OR) now() [...]] ``` `now()` is the Unix time of the server at the time the query is executed on that server. The whitespace between `-` or `+` and the [duration literal](/influxdb/v1/query_language/spec/#durations) is required. #### Supported operators |Operator| Meaning | |--------|------------------------| | `=` | equal to | | `<>` | not equal to | | `!=` | not equal to | | `>` | greater than | | `>=` |greater than or equal to| | `<` | less than | | `<=` | less than or equal to | #### `duration_literal` * microseconds: `u` or `µ` * milliseconds: `ms` * seconds`s` * minutes`m` * hours:`h` * days:`d` * weeks:`w` ### Examples #### Specify a time range with relative time ```sql SELECT "water_level" FROM "h2o_feet" WHERE time > now() - 1h ``` The query returns data with timestamps that occur within the past hour. The whitespace between `-` and `1h` is required. #### Specify a time range with absolute time and relative time ```sql > SELECT "level description" FROM "h2o_feet" WHERE time > '2015-09-18T21:18:00Z' AND time < now() + 1000d name: h2o_feet time level description ---- ----------------- 2015-09-18T21:24:00Z between 3 and 6 feet 2015-09-18T21:30:00Z between 3 and 6 feet 2015-09-18T21:36:00Z between 3 and 6 feet 2015-09-18T21:42:00Z between 3 and 6 feet ``` The query returns data with timestamps that occur between September 18, 2015 at 21:18:00 and 1000 days from `now()`. The whitespace between `+` and `1000d` is required. ## Common issues with time syntax ### Using `OR` to select time multiple time intervals InfluxDB does not support using the `OR` operator in the `WHERE` clause to specify multiple time intervals. For more information, see [Frequently asked questions](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-is-my-query-with-a-where-or-time-clause-returning-empty-results). ### Querying data that occur after `now()` with a `GROUP BY time()` clause Most `SELECT` statements have a default time range between [`1677-09-21 00:12:43.145224194` and `2262-04-11T23:47:16.854775806Z` UTC](/influxdb/v1/troubleshooting/frequently-asked-questions/#what-are-the-minimum-and-maximum-timestamps-that-influxdb-can-store). For `SELECT` statements with a [`GROUP BY time()` clause](#group-by-time-intervals), the default time range is between `1677-09-21 00:12:43.145224194` UTC and [`now()`](/influxdb/v1/concepts/glossary/#now). To query data with timestamps that occur after `now()`, `SELECT` statements with a `GROUP BY time()` clause must provide an alternative upper bound in the`WHERE` clause. #### Example Use the [CLI](/influxdb/v1/tools/shell/) to write a point to the `NOAA_water_database` that occurs after `now()`: ```sql INSERT h2o_feet,location=santa_monica water_level=3.1 1587074400000000000 ``` Run a `GROUP BY time()` query that covers data with timestamps between`2015-09-18T21:30:00Z` and `now()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location"='santa_monica' AND time >= '2015-09-18T21:30:00Z' GROUP BY time(12m) fill(none) name: h2o_feet time mean ---- ---- 2015-09-18T21:24:00Z 5.01 2015-09-18T21:36:00Z 5.002 ``` Run a `GROUP BY time()` query that covers data with timestamps between`2015-09-18T21:30:00Z` and 180 weeks from `now()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location"='santa_monica' AND time >= '2015-09-18T21:30:00Z' AND time <= now() + 180w GROUP BY time(12m) fill(none) name: h2o_feet time mean ---- ---- 2015-09-18T21:24:00Z 5.01 2015-09-18T21:36:00Z 5.002 2020-04-16T22:00:00Z 3.1 ``` Note that the `WHERE` clause must provide an alternative **upper** bound to override the default `now()` upper bound. The following query merely resets the lower bound to `now()` such that the query’s time range is between`now()` and `now()`: ```sql SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location"='santa_monica' AND time >= now() GROUP BY time(12m) fill(none) ``` ### Configuring the returned timestamps The [CLI](/influxdb/v1/tools/shell/) returns timestamps in nanosecond epoch format by default. Specify alternative formats with the[`precision ` command](/influxdb/v1/tools/shell/#influx-commands). The [InfluxDB API](/influxdb/v1/api/) returns timestamps in [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) format by default. Specify alternative formats with the[`epoch` query string parameter](/influxdb/v1/api/query/). ## Regular expressions InfluxQL supports using regular expressions when specifying: * [field keys](/influxdb/v1/concepts/glossary/#field-key) and [tag keys](/influxdb/v1/concepts/glossary/#tag-key) in the [`SELECT` clause](#the-basic-select-statement) * [measurements](/influxdb/v1/concepts/glossary/#measurement) in the [`FROM` clause](#the-basic-select-statement) * [tag values](/influxdb/v1/concepts/glossary/#tag-value) and string [field values](/influxdb/v1/concepts/glossary/#field-value) in the [`WHERE` clause](#the-where-clause). * [tag keys](/influxdb/v1/concepts/glossary/#tag-key) in the [`GROUP BY` clause](#group-by-tags) Currently, InfluxQL does not support using regular expressions to match non-string field values in the`WHERE` clause,[databases](/influxdb/v1/concepts/glossary/#database), and[retention polices](/influxdb/v1/concepts/glossary/#retention-policy-rp). > > > **Note:** Regular expression comparisons are more computationally intensive than exact > string comparisons; queries with regular expressions are not as performant > as those without. > > ### Syntax ```sql SELECT // FROM // WHERE [ // | //] GROUP BY // ``` Regular expressions are surrounded by `/` characters and use[Golang’s regular expression syntax](http://golang.org/pkg/regexp/syntax/). #### Supported operators `=~` matches against`!~` doesn’t match against ### Examples #### Use a regular expression to specify field keys and tag keys in the SELECT clause ```sql > SELECT /l/ FROM "h2o_feet" LIMIT 1 name: h2o_feet time level description location water_level ---- ----------------- -------- ----------- 2015-08-18T00:00:00Z between 6 and 9 feet coyote_creek 8.12 ``` The query selects all [field keys](/influxdb/v1/concepts/glossary/#field-key)and [tag keys](/influxdb/v1/concepts/glossary/#tag-key) that include an `l`. Note that the regular expression in the `SELECT` clause must match at least one field key in order to return results for a tag key that matches the regular expression. Currently, there is no syntax to distinguish between regular expressions for field keys and regular expressions for tag keys in the `SELECT` clause. The syntax `//::[field | tag]` is not supported. #### Use a regular expression to specify measurements in the FROM clause ```sql > SELECT MEAN("degrees") FROM /temperature/ name: average_temperature time mean ---- ---- 1970-01-01T00:00:00Z 79.98472932232272 name: h2o_temperature time mean ---- ---- 1970-01-01T00:00:00Z 64.98872722506226 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to calculate the average `degrees` for every [measurement](/influxdb/v1/concepts/glossary#measurement) in the `NOAA_water_database`[database](/influxdb/v1/concepts/glossary#database) that contains the word `temperature`. #### Use a regular expression to specify tag values in the WHERE clause ```sql > SELECT MEAN(water_level) FROM "h2o_feet" WHERE "location" =~ /[m]/ AND "water_level" > 3 name: h2o_feet time mean ---- ---- 1970-01-01T00:00:00Z 4.47155532049926 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to calculate the average `water_level` where the [tag value](/influxdb/v1/concepts/glossary#tag-value) of `location`includes an `m` and `water_level` is greater than three. #### Use a regular expression to specify a tag with no value in the WHERE clause ```sql SELECT * FROM "h2o_feet" WHERE "location" !~ /./ ``` The query selects all data from the `h2o_feet` measurement where the `location`[tag](/influxdb/v1/concepts/glossary#tag) has no value. Every data [point](/influxdb/v1/concepts/glossary#point) in the `NOAA_water_database` has a tag value for `location`. It’s possible to perform this same query without a regular expression. See the[Frequently Asked Questions](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-do-i-select-data-with-a-tag-that-has-no-value)document for more information. #### Use a regular expression to specify a tag with a value in the WHERE clause ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location" =~ /./ name: h2o_feet time mean ---- ---- 1970-01-01T00:00:00Z 4.442107025822523 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to calculate the average `water_level` across all data that have a tag value for`location`. #### Use a regular expression to specify a field value in the WHERE clause ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica' AND "level description" =~ /between/ name: h2o_feet time mean ---- ---- 1970-01-01T00:00:00Z 4.47155532049926 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to calculate the average `water_level` for all data where the field value of`level description` includes the word `between`. #### Use a regular expression to specify tag keys in the GROUP BY clause ```sql > SELECT FIRST("index") FROM "h2o_quality" GROUP BY /l/ name: h2o_quality tags: location=coyote_creek time first ---- ----- 2015-08-18T00:00:00Z 41 name: h2o_quality tags: location=santa_monica time first ---- ----- 2015-08-18T00:00:00Z 99 ``` The query uses an InfluxQL [function](/influxdb/v1/query_language/functions/)to select the first value of `index` for every tag that includes the letter `l`in its tag key. ## Data types and cast operations The [`SELECT` clause](#the-basic-select-statement) supports specifying a [field’s](/influxdb/v1/concepts/glossary/#field) type and basic cast operations with the `::` syntax. |[Data Types](#data-types)|[Cast Operations](#cast-operations)| |-------------------------|-----------------------------------| ## Data types [Field values](/influxdb/v1/concepts/glossary/#field-value) can be floats, integers, strings, or booleans. The `::` syntax allows users to specify the field’s type in a query. > > > **Note:** Generally, it is not necessary to specify the field value > type in the [`SELECT` clause](#the-basic-select-statement). > In most cases, InfluxDB rejects any writes that attempt to write a [field value](/influxdb/v1/concepts/glossary/#field-value)to a field that previously accepted field values of a different type. > > It is possible for field value types to differ across [shard groups](/influxdb/v1/concepts/glossary/#shard-group). In these cases, it may be necessary to specify the field value type in the`SELECT` clause. Please see the[Frequently Asked Questions](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-does-influxdb-handle-field-type-discrepancies-across-shards)document for more information on how InfluxDB handles field value type discrepancies. ### Syntax ```sql SELECT_clause :: FROM_clause ``` `type` can be `float`, `integer`, `string`, or `boolean`. In most cases, InfluxDB returns no data if the `field_key` does not store data of the specified`type`. See [Cast Operations](#cast-operations) for more information. ### Example ``` > SELECT "water_level"::float FROM "h2o_feet" LIMIT 4 ## name: h2o_feet time water_level 2015-08-18T00:00:00Z 8.12 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 8.005 2015-08-18T00:06:00Z 2.116 ``` The query returns values of the `water_level` field key that are floats. ## Cast operations The `::` syntax allows users to perform basic cast operations in queries. Currently, InfluxDB supports casting [field values](/influxdb/v1/concepts/glossary/#field-value) from integers to floats or from floats to integers. ### Syntax ```sql SELECT_clause :: FROM_clause ``` `type` can be `float` or `integer`. InfluxDB returns no data if the query attempts to cast an integer or float to a string or boolean. ### Examples #### Cast float field values to integers ``` > SELECT "water_level"::integer FROM "h2o_feet" LIMIT 4 ## name: h2o_feet time water_level 2015-08-18T00:00:00Z 8 2015-08-18T00:00:00Z 2 2015-08-18T00:06:00Z 8 2015-08-18T00:06:00Z 2 ``` The query returns the integer form of `water_level`’s float [field values](/influxdb/v1/concepts/glossary/#field-value). #### Cast float field values to strings (this functionality is not supported) ```sql SELECT "water_level"::string FROM "h2o_feet" LIMIT 4 ``` The query returns no data as casting a float field value to a string is not yet supported. ## Merge behavior In InfluxDB, queries merge [series](/influxdb/v1/concepts/glossary/#series)automatically. ### Example The `h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement) in the `NOAA_water_database` is part of two [series](/influxdb/v1/concepts/glossary/#series). The first series is made up of the `h2o_feet` measurement and the `location = coyote_creek` [tag](/influxdb/v1/concepts/glossary/#tag). The second series is made of up the `h2o_feet` measurement and the `location = santa_monica` tag. The following query automatically merges those two series when it calculates the [average](/influxdb/v1/query_language/functions/#mean) `water_level`: ``` > SELECT MEAN("water_level") FROM "h2o_feet" ## name: h2o_feet time mean 1970-01-01T00:00:00Z 4.442107025822521 ``` If you want the average `water_level` for the first series only, specify the relevant tag in the [`WHERE` clause](#the-where-clause): ``` > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location" = 'coyote_creek' ## name: h2o_feet time mean 1970-01-01T00:00:00Z 5.359342451341401 ``` If you want the average `water_level` for each individual series, include a [`GROUP BY` clause](#group-by-tags): ```sql > SELECT MEAN("water_level") FROM "h2o_feet" GROUP BY "location" name: h2o_feet tags: location=coyote_creek time mean ---- ---- 1970-01-01T00:00:00Z 5.359342451341401 name: h2o_feet tags: location=santa_monica time mean ---- ---- 1970-01-01T00:00:00Z 3.530863470081006 ``` ## Multiple statements Separate multiple [`SELECT` statements](#the-basic-select-statement) in a query with a semicolon (`;`). ### Examples [Example 1: CLI](#example-1-cli)[Example 2: InfluxDB API](#example-2-influxdb-api) In the InfluxDB [CLI](/influxdb/v1/tools/shell/): ```sql > SELECT MEAN("water_level") FROM "h2o_feet"; SELECT "water_level" FROM "h2o_feet" LIMIT 2 name: h2o_feet time mean ---- ---- 1970-01-01T00:00:00Z 4.442107025822522 name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 8.12 2015-08-18T00:00:00Z 2.064 ``` With the [InfluxDB API](/influxdb/v1/api/): ```json { "results": [ { "statement_id": 0, "series": [ { "name": "h2o_feet", "columns": [ "time", "mean" ], "values": [ [ "1970-01-01T00:00:00Z", 4.442107025822522 ] ] } ] }, { "statement_id": 1, "series": [ { "name": "h2o_feet", "columns": [ "time", "water_level" ], "values": [ [ "2015-08-18T00:00:00Z", 8.12 ], [ "2015-08-18T00:00:00Z", 2.064 ] ] } ] } ] } ``` ## Subqueries A subquery is a query that is nested in the `FROM` clause of another query. Use a subquery to apply a query as a condition in the enclosing query. Subqueries offer functionality similar to nested functions and SQL[`HAVING` clauses](https://en.wikipedia.org/wiki/Having_%28SQL%29). ### Syntax ```sql SELECT_clause FROM ( SELECT_statement ) [...] ``` InfluxDB performs the subquery first and the main query second. The main query surrounds the subquery and requires at least the [`SELECT` clause](#the-basic-select-statement) and the [`FROM` clause](#the-basic-select-statement). The main query supports all clauses listed in this document. The subquery appears in the main query’s `FROM` clause, and it requires surrounding parentheses. The subquery supports all clauses listed in this document. InfluxQL supports multiple nested subqueries per main query. Sample syntax for multiple subqueries: ```sql SELECT_clause FROM ( SELECT_clause FROM ( SELECT_statement ) [...] ) [...] ``` > [!Note] > #### Improve performance of time-bound subqueries > > To improve the performance of InfluxQL queries with time-bound subqueries, > apply the `WHERE time` clause to the outer query instead of the inner query. > For example, the following queries return the same results, but **the query with > time bounds on the outer query is more performant than the query with time > bounds on the inner query**: > > ##### Time bounds on the outer query (recommended) > > ``` > SELECT inner_value AS value FROM (SELECT raw_value as inner_value) > WHERE time >= '2020-07-19T21:00:00Z' > AND time <= '2020-07-20T22:00:00Z' > ``` > > ##### Time bounds on the inner query > > ``` > SELECT inner_value AS value FROM ( > SELECT raw_value as inner_value > WHERE time >= '2020-07-19T21:00:00Z' > AND time <= '2020-07-20T22:00:00Z' > ) > ``` ### Examples #### Calculate the [`SUM()`](/influxdb/v1/query_language/functions/#sum) of several [`MAX()`](/influxdb/v1/query_language/functions/#max) values ```sql > SELECT SUM("max") FROM (SELECT MAX("water_level") FROM "h2o_feet" GROUP BY "location") name: h2o_feet time sum ---- --- 1970-01-01T00:00:00Z 17.169 ``` The query returns the sum of the maximum `water_level` values across every tag value of `location`. InfluxDB first performs the subquery; it calculates the maximum value of `water_level` for each tag value of `location`: ```sql > SELECT MAX("water_level") FROM "h2o_feet" GROUP BY "location" name: h2o_feet tags: location=coyote_creek time max ---- --- 2015-08-29T07:24:00Z 9.964 name: h2o_feet tags: location=santa_monica time max ---- --- 2015-08-29T03:54:00Z 7.205 ``` Next, InfluxDB performs the main query and calculates the sum of those maximum values: `9.964` + `7.205` = `17.169`. Notice that the main query specifies `max`, not `water_level`, as the field key in the `SUM()` function. #### Calculate the [`MEAN()`](/influxdb/v1/query_language/functions/#mean) difference between two fields ```sql > SELECT MEAN("difference") FROM (SELECT "cats" - "dogs" AS "difference" FROM "pet_daycare") name: pet_daycare time mean ---- ---- 1970-01-01T00:00:00Z 1.75 ``` The query returns the average of the differences between the number of `cats` and `dogs` in the `pet_daycare` measurement. InfluxDB first performs the subquery. The subquery calculates the difference between the values in the `cats` field and the values in the `dogs` field, and it names the output column `difference`: ```sql > SELECT "cats" - "dogs" AS "difference" FROM "pet_daycare" name: pet_daycare time difference ---- ---------- 2017-01-20T00:55:56Z -1 2017-01-21T00:55:56Z -49 2017-01-22T00:55:56Z 66 2017-01-23T00:55:56Z -9 ``` Next, InfluxDB performs the main query and calculates the average of those differences. Notice that the main query specifies `difference` as the field key in the `MEAN()` function. #### Calculate several [`MEAN()`](/influxdb/v1/query_language/functions/#mean) values and place a condition on those mean values ```sql > SELECT "all_the_means" FROM (SELECT MEAN("water_level") AS "all_the_means" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) ) WHERE "all_the_means" > 5 name: h2o_feet time all_the_means ---- ------------- 2015-08-18T00:00:00Z 5.07625 ``` The query returns all mean values of the `water_level` field that are greater than five. InfluxDB first performs the subquery. The subquery calculates `MEAN()` values of `water_level` from `2015-08-18T00:00:00Z` through `2015-08-18T00:30:00Z` and groups the results into 12-minute intervals. It also names the output column `all_the_means`: ```sql > SELECT MEAN("water_level") AS "all_the_means" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) name: h2o_feet time all_the_means ---- ------------- 2015-08-18T00:00:00Z 5.07625 2015-08-18T00:12:00Z 4.950749999999999 2015-08-18T00:24:00Z 4.80675 ``` Next, InfluxDB performs the main query and returns only those mean values that are greater than five. Notice that the main query specifies `all_the_means` as the field key in the `SELECT` clause. #### Calculate the [`SUM()`](/influxdb/v1/query_language/functions/#sum) of several [`DERIVATIVE()`](/influxdb/v1/query_language/functions/#derivative) values ```sql > SELECT SUM("water_level_derivative") AS "sum_derivative" FROM (SELECT DERIVATIVE(MEAN("water_level")) AS "water_level_derivative" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m),"location") GROUP BY "location" name: h2o_feet tags: location=coyote_creek time sum_derivative ---- -------------- 1970-01-01T00:00:00Z -0.4950000000000001 name: h2o_feet tags: location=santa_monica time sum_derivative ---- -------------- 1970-01-01T00:00:00Z -0.043999999999999595 ``` The query returns the sum of the derivative of average `water_level` values for each tag value of `location`. InfluxDB first performs the subquery. The subquery calculates the derivative of average `water_level` values taken at 12-minute intervals. It performs that calculation for each tag value of `location` and names the output column `water_level_derivative`: ```sql > SELECT DERIVATIVE(MEAN("water_level")) AS "water_level_derivative" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m),"location" name: h2o_feet tags: location=coyote_creek time water_level_derivative ---- ---------------------- 2015-08-18T00:12:00Z -0.23800000000000043 2015-08-18T00:24:00Z -0.2569999999999997 name: h2o_feet tags: location=santa_monica time water_level_derivative ---- ---------------------- 2015-08-18T00:12:00Z -0.0129999999999999 2015-08-18T00:24:00Z -0.030999999999999694 ``` Next, InfluxDB performs the main query and calculates the sum of the `water_level_derivative` values for each tag value of `location`. Notice that the main query specifies `water_level_derivative`, not `water_level` or `derivative`, as the field key in the `SUM()` function. ### Common issues with subqueries #### Multiple SELECT statements in a subquery InfluxQL supports multiple nested subqueries per main query: ```sql SELECT_clause FROM ( SELECT_clause FROM ( SELECT_statement ) [...] ) [...] ------------------ ---------------- Subquery 1 Subquery 2 ``` InfluxQL does not support multiple [`SELECT` statements](#the-basic-select-statement) per subquery: ```sql SELECT_clause FROM (SELECT_statement; SELECT_statement) [...] ``` The system returns a parsing error if a subquery includes multiple `SELECT` statements. | The Basics: | Configure Query Results: | General Tips on Query Syntax: | | --- | --- | --- | | The SELECT statement | ORDER BY time DESC | Time Syntax | | The WHERE clause | The LIMIT and SLIMIT clauses | Regular Expressions | | The GROUP BY clause | The OFFSET and SOFFSET clauses | Data types and cast operations | | The INTO clause | The Time Zone clause | Merge behavior | | | | Multiple statements | | | | Subqueries | | time | level description | location | water_level | | --- | --- | --- | --- | | time | level description | location | water_level | | 2015-08-18T00:00:00Z | between 6 and 9 feet | coyote_creek | 8.12 | | 2015-08-18T00:00:00Z | below 3 feet | santa_monica | 2.064 | | 2015-08-18T00:06:00Z | between 6 and 9 feet | coyote_creek | 8.005 | | 2015-08-18T00:06:00Z | below 3 feet | santa_monica | 2.116 | | 2015-08-18T00:12:00Z | between 6 and 9 feet | coyote_creek | 7.887 | | 2015-08-18T00:12:00Z | below 3 feet | santa_monica | 2.028 | | Operator | Meaning | | --- | --- | | Operator | Meaning | | = | equal to | | <> | not equal to | | != | not equal to | | > | greater than | | >= | greater than or equal to | | < | less than | | <= | less than or equal to | | Operator | Meaning | | --- | --- | | Operator | Meaning | | = | equal to | | <> | not equal to | | != | not equal to | | GROUP BY tags | | | | | --- | --- | --- | --- | | GROUP BY time intervals: | Basic Syntax | Advanced Syntax | GROUP BY time intervals and fill() | | Time Interval Number | Preset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | --- | --- | --- | --- | --- | | Time Interval Number | Preset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | 1 | time >= 2015-08-18T00:00:00Z AND time < 2015-08-18T00:12:00Z | time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:12:00Z | 8.005 | 2015-08-18T00:00:00Z | | 2 | time >= 2015-08-12T00:12:00Z AND time < 2015-08-18T00:24:00Z | time >= 2015-08-12T00:12:00Z AND time < 2015-08-18T00:18:00Z | 7.887 | 2015-08-18T00:12:00Z | | Time Interval Number | Preset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | --- | --- | --- | --- | --- | | Time Interval Number | Preset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | 1 | time >= 2015-08-18T00:00:00Z AND time < 2015-08-18T00:18:00Z | time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:18:00Z | 8.005 , 7.887 | 2015-08-18T00:00:00Z | | 2 | time >= 2015-08-18T00:18:00Z AND time < 2015-08-18T00:36:00Z | <— same | 7.762 , 7.635 , 7.5 | 2015-08-18T00:18:00Z | | 3 | time >= 2015-08-18T00:36:00Z AND time < 2015-08-18T00:54:00Z | <— same | 7.372 , 7.234 , 7.11 | 2015-08-18T00:36:00Z | | 4 | time >= 2015-08-18T00:54:00Z AND time < 2015-08-18T01:12:00Z | time = 2015-08-18T00:54:00Z | 6.982 | 2015-08-18T00:54:00Z | | Time Interval Number | Offset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | --- | --- | --- | --- | --- | | Time Interval Number | Offset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | 1 | time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:24:00Z | <— same | 8.005 , 7.887 , 7.762 | 2015-08-18T00:06:00Z | | 2 | time >= 2015-08-18T00:24:00Z AND time < 2015-08-18T00:42:00Z | <— same | 7.635 , 7.5 , 7.372 | 2015-08-18T00:24:00Z | | 3 | time >= 2015-08-18T00:42:00Z AND time < 2015-08-18T01:00:00Z | <— same | 7.234 , 7.11 , 6.982 | 2015-08-18T00:42:00Z | | 4 | time >= 2015-08-18T01:00:00Z AND time < 2015-08-18T01:18:00Z | NA | NA | NA | | Time Interval Number | Preset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | --- | --- | --- | --- | --- | | Time Interval Number | Preset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | 1 | time >= 2015-08-18T00:00:00Z AND time < 2015-08-18T00:18:00Z | time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:18:00Z | 8.005 , 7.887 | 2015-08-18T00:00:00Z | | 2 | time >= 2015-08-18T00:18:00Z AND time < 2015-08-18T00:36:00Z | <— same | 7.762 , 7.635 , 7.5 | 2015-08-18T00:18:00Z | | 3 | time >= 2015-08-18T00:36:00Z AND time < 2015-08-18T00:54:00Z | <— same | 7.372 , 7.234 , 7.11 | 2015-08-18T00:36:00Z | | 4 | time >= 2015-08-18T00:54:00Z AND time < 2015-08-18T01:12:00Z | time = 2015-08-18T00:54:00Z | 6.982 | 2015-08-18T00:54:00Z | | Time Interval Number | Offset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | --- | --- | --- | --- | --- | | Time Interval Number | Offset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | 1 | time >= 2015-08-17T23:48:00Z AND time < 2015-08-18T00:06:00Z | NA | NA | NA | | 2 | time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:24:00Z | <— same | 8.005 , 7.887 , 7.762 | 2015-08-18T00:06:00Z | | 3 | time >= 2015-08-18T00:24:00Z AND time < 2015-08-18T00:42:00Z | <— same | 7.635 , 7.5 , 7.372 | 2015-08-18T00:24:00Z | | 4 | time >= 2015-08-18T00:42:00Z AND time < 2015-08-18T01:00:00Z | <— same | 7.234 , 7.11 , 6.982 | 2015-08-18T00:42:00Z | | Time Interval Number | Preset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | --- | --- | --- | --- | --- | | Time Interval Number | Preset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | 1 | time >= 2015-08-18T00:00:00Z AND time < 2015-08-18T00:12:00Z | time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:12:00Z | 8.005 | 2015-08-18T00:00:00Z | | 2 | time >= 2015-08-12T00:12:00Z AND time < 2015-08-18T00:24:00Z | time >= 2015-08-12T00:12:00Z AND time < 2015-08-18T00:18:00Z | 7.887 | 2015-08-18T00:12:00Z | | Time Interval Number | Offset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | --- | --- | --- | --- | --- | | Time Interval Number | Offset Time Boundary | GROUP BY time() Interval | Points Included | Returned Timestamp | | 1 | time >= 2015-08-18T00:06:00Z AND time < 2015-08-18T00:18:00Z | <— same | 8.005 , 7.887 | 2015-08-18T00:06:00Z | | 2 | time >= 2015-08-18T00:18:00Z AND time < 2015-08-18T00:30:00Z | NA | NA | NA | | The OFFSET clause | The SOFFSET clause | | --- | --- | | Absolute time | Relative time | Common issues with time syntax | | --- | --- | --- | | Operator | Meaning | | --- | --- | | Operator | Meaning | | = | equal to | | <> | not equal to | | != | not equal to | | > | greater than | | >= | greater than or equal to | | < | less than | | <= | less than or equal to | | Operator | Meaning | | --- | --- | | Operator | Meaning | | = | equal to | | <> | not equal to | | != | not equal to | | > | greater than | | >= | greater than or equal to | | < | less than | | <= | less than or equal to | | Data Types | Cast Operations | | --- | --- | --- # Explore your schema using InfluxQL Source: https://docs.influxdata.com/influxdb/v1/query_language/explore-schema/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxQL is an SQL-like query language for interacting with data in InfluxDB. The following sections cover useful query syntax for exploring your [schema](/influxdb/v1/concepts/glossary/#schema). | [SHOW DATABASES](#show-databases) |[SHOW RETENTION POLICIES](#show-retention-policies)| [SHOW SERIES](#show-series) | | |---------------------------------------|---------------------------------------------------|-----------------------------------------------------------|---| |[SHOW MEASUREMENTS](#show-measurements)| [SHOW TAG KEYS](#show-tag-keys) | [SHOW TAG VALUES](#show-tag-values) | | | [SHOW FIELD KEYS](#show-field-keys) | [SHOW CARDINALITY](#show-cardinality) |[Filter meta queries by time](#filter-meta-queries-by-time)| | **Sample data** The data used in this document are available for download on the [Sample Data](/influxdb/v1/query_language/data_download/) page. Before proceeding, login to the Influx CLI. ```bash $ influx -precision rfc3339 Connected to http://localhost:8086 version 1.13.0 InfluxDB shell 1.13.0 > ``` ## `SHOW DATABASES` Returns a list of all [databases](/influxdb/v1/concepts/glossary/#database) on your instance. ### Syntax ```sql SHOW DATABASES ``` ### Examples #### Run a `SHOW DATABASES` query ``` > SHOW DATABASES name: databases ## name NOAA_water_database _internal ``` The query returns database names in a tabular format. This InfluxDB instance has two databases: `NOAA_water_database` and `_internal`. ## `SHOW RETENTION POLICIES` Returns a list of [retention policies](/influxdb/v1/concepts/glossary/#retention-policy-rp) for the specified [database](/influxdb/v1/concepts/glossary/#database). ### Syntax ```sql SHOW RETENTION POLICIES [ON ] ``` ### Description of syntax `ON ` is optional. If the query does not include `ON `, you must specify the database with `USE ` in the [CLI](/influxdb/v1/tools/shell/) or with the `db` query string parameter in the [InfluxDB API](/influxdb/v1/api/query/) request. ### Examples #### Run a `SHOW RETENTION POLICIES` query with the `ON` clause ```sql > SHOW RETENTION POLICIES ON NOAA_water_database name duration shardGroupDuration replicaN default ---- -------- ------------------ -------- ------- autogen 0s 168h0m0s 1 true ``` The query returns the list of retention policies in the `NOAA_water_database`database in tabular format. The database has one retention policy called `autogen`. The `autogen` retention policy has an infinite [duration](/influxdb/v1/concepts/glossary/#duration), a seven-day [shard group duration](/influxdb/v1/concepts/glossary/#shard-group), a [replication factor](/influxdb/v1/concepts/glossary/#replication-factor-rf)of one, and it is the `DEFAULT` retention policy for the database. #### Run a `SHOW RETENTION POLICIES` query without the `ON` clause [CLI](#cli)[InfluxDB API](#influxdb-api) Specify the database with `USE ` ```sql > USE NOAA_water_database Using database NOAA_water_database > SHOW RETENTION POLICIES name duration shardGroupDuration replicaN default ---- -------- ------------------ -------- ------- autogen 0s 168h0m0s 1 true ``` Specify the database with the `db` query string parameter: ```bash ~# curl -G "http://localhost:8086/query?db=NOAA_water_database&pretty=true" --data-urlencode "q=SHOW RETENTION POLICIES" { "results": [ { "statement_id": 0, "series": [ { "columns": [ "name", "duration", "shardGroupDuration", "replicaN", "default" ], "values": [ [ "autogen", "0s", "168h0m0s", 1, true ] ] } ] } ] } ``` ## `SHOW SERIES` Returns a list of [series](/influxdb/v1/concepts/glossary/#series) for the specified [database](/influxdb/v1/concepts/glossary/#database). ### Syntax ```sql SHOW SERIES [ON ] [FROM_clause] [WHERE [ '' | ]] [LIMIT_clause] [OFFSET_clause] ``` ### Description of syntax `ON ` is optional. If the query does not include `ON `, you must specify the database with `USE ` in the [CLI](/influxdb/v1/tools/shell/) or with the `db` query string parameter in the [InfluxDB API](/influxdb/v1/api/query/) request.`SHOW SERIES` only returns series in the database’s default retention policy, and fails if there is no default retention policy. The `FROM`, `WHERE`, `LIMIT`, and `OFFSET` clauses are optional. The `WHERE` clause supports tag comparisons; field comparisons are not valid for the `SHOW SERIES` query. Supported operators in the `WHERE` clause:`=` equal to`<>` not equal to`!=` not equal to`=~` matches against`!~` doesn’t match against See the Data Exploration page for documentation on the[`FROM` clause](/influxdb/v1/query_language/explore-data/#the-basic-select-statement),[`LIMIT` clause](/influxdb/v1/query_language/explore-data/#the-limit-clause),[`OFFSET` clause](/influxdb/v1/query_language/explore-data/#the-offset-clause), and on [Regular Expressions in Queries](/influxdb/v1/query_language/explore-data/#regular-expressions). ### Examples #### Run a `SHOW SERIES` query with the `ON` clause ``` // Returns series for all shards in the database and default retention policy > SHOW SERIES ON NOAA_water_database ## key average_temperature,location=coyote_creek average_temperature,location=santa_monica h2o_feet,location=coyote_creek h2o_feet,location=santa_monica h2o_pH,location=coyote_creek h2o_pH,location=santa_monica h2o_quality,location=coyote_creek,randtag=1 h2o_quality,location=coyote_creek,randtag=2 h2o_quality,location=coyote_creek,randtag=3 h2o_quality,location=santa_monica,randtag=1 h2o_quality,location=santa_monica,randtag=2 h2o_quality,location=santa_monica,randtag=3 h2o_temperature,location=coyote_creek h2o_temperature,location=santa_monica ``` The query’s output is similar to the [line protocol](/influxdb/v1/concepts/glossary/#influxdb-line-protocol) format. Everything before the first comma is the [measurement](/influxdb/v1/concepts/glossary/#measurement) name. Everything after the first comma is either a [tag key](/influxdb/v1/concepts/glossary/#tag-key) or a [tag value](/influxdb/v1/concepts/glossary/#tag-value). The `NOAA_water_database` has five different measurements and 14 different series. #### Run a `SHOW SERIES` query without the `ON` clause [CLI](#cli-2)[InfluxDB API](#influxdb-api-2) Specify the database with `USE ` ``` > USE NOAA_water_database Using database NOAA_water_database > SHOW SERIES ## key average_temperature,location=coyote_creek average_temperature,location=santa_monica h2o_feet,location=coyote_creek h2o_feet,location=santa_monica h2o_pH,location=coyote_creek h2o_pH,location=santa_monica h2o_quality,location=coyote_creek,randtag=1 h2o_quality,location=coyote_creek,randtag=2 h2o_quality,location=coyote_creek,randtag=3 h2o_quality,location=santa_monica,randtag=1 h2o_quality,location=santa_monica,randtag=2 h2o_quality,location=santa_monica,randtag=3 h2o_temperature,location=coyote_creek h2o_temperature,location=santa_monica ``` Specify the database with the `db` query string parameter: ```bash ~# curl -G "http://localhost:8086/query?db=NOAA_water_database&pretty=true" --data-urlencode "q=SHOW SERIES" { "results": [ { "statement_id": 0, "series": [ { "columns": [ "key" ], "values": [ [ "average_temperature,location=coyote_creek" ], [ "average_temperature,location=santa_monica" ], [ "h2o_feet,location=coyote_creek" ], [ "h2o_feet,location=santa_monica" ], [ "h2o_pH,location=coyote_creek" ], [ "h2o_pH,location=santa_monica" ], [ "h2o_quality,location=coyote_creek,randtag=1" ], [ "h2o_quality,location=coyote_creek,randtag=2" ], [ "h2o_quality,location=coyote_creek,randtag=3" ], [ "h2o_quality,location=santa_monica,randtag=1" ], [ "h2o_quality,location=santa_monica,randtag=2" ], [ "h2o_quality,location=santa_monica,randtag=3" ], [ "h2o_temperature,location=coyote_creek" ], [ "h2o_temperature,location=santa_monica" ] ] } ] } ] } ``` #### Run a `SHOW SERIES` query with several clauses ``` > SHOW SERIES ON NOAA_water_database FROM "h2o_quality" WHERE "location" = 'coyote_creek' LIMIT 2 ## key h2o_quality,location=coyote_creek,randtag=1 h2o_quality,location=coyote_creek,randtag=2 ``` The query returns all series in the `NOAA_water_database` database that are associated with the `h2o_quality` measurement and the tag `location = coyote_creek`. The `LIMIT` clause limits the number of series returned to two. #### Run a `SHOW SERIES` query limited by time Limit series returned within a specified shard group duration. ``` // Returns all series in the current shard. > SHOW SERIES ON NOAA_water_database WHERE time > now() - 1m ## key average_temperature,location=coyote_creek h2o_feet,location=coyote_creek h2o_pH,location=coyote_creek h2o_quality,location=coyote_creek,randtag=1 h2o_quality,location=coyote_creek,randtag=2 h2o_quality,location=coyote_creek,randtag=3 h2o_temperature,location=coyote_creek ``` The query above returns all series in the `NOAA_water_database` database in the current shard group. The `WHERE` clause limits results to series in the shard group that contain a timestamp in the last minute. Note, if a shard group duration is 7 days, results returned may be up to 7 days old. ``` // Returns all series in shard groups that contain a timestamp in the last 28 days. > SHOW SERIES ON NOAA_water_database WHERE time > now() - 28d ## key average_temperature,location=coyote_creek average_temperature,location=santa_monica h2o_feet,location=coyote_creek h2o_feet,location=santa_monica h2o_pH,location=coyote_creek h2o_pH,location=santa_monica h2o_quality,location=coyote_creek,randtag=1 h2o_quality,location=coyote_creek,randtag=2 h2o_quality,location=coyote_creek,randtag=3 h2o_quality,location=santa_monica,randtag=1 h2o_quality,location=santa_monica,randtag=2 h2o_quality,location=santa_monica,randtag=3 h2o_temperature,location=coyote_creek h2o_temperature,location=santa_monica ``` Note, if the specified shard group duration is 7 days, the query above returns series for the last 3 or 4 shards. ## `SHOW MEASUREMENTS` Returns a list of [measurements](/influxdb/v1/concepts/glossary/#measurement)for the specified [database](/influxdb/v1/concepts/glossary/#database). ### Syntax ```sql SHOW MEASUREMENTS [ON ] [WITH MEASUREMENT ['' | ]] [WHERE ['' | ]] [LIMIT_clause] [OFFSET_clause] ``` ### Description of Syntax `ON ` is optional. If the query does not include `ON `, you must specify the database with `USE ` in the [CLI](/influxdb/v1/tools/shell/) or with the `db` query string parameter in the [InfluxDB API](/influxdb/v1/api/query/) request. The `WITH`, `WHERE`, `LIMIT` and `OFFSET` clauses are optional. The `WHERE` clause supports tag comparisons; field comparisons are not valid for the `SHOW MEASUREMENTS` query. Supported operators in the `WHERE` clause:`=` equal to`<>` not equal to`!=` not equal to`=~` matches against`!~` doesn’t match against See the Data Exploration page for documentation on the[`LIMIT` clause](/influxdb/v1/query_language/explore-data/#the-limit-clause),[`OFFSET` clause](/influxdb/v1/query_language/explore-data/#the-offset-clause), and on [Regular expressions in queries](/influxdb/v1/query_language/explore-data/#regular-expressions). ### Examples #### Run a `SHOW MEASUREMENTS` query with the `ON` clause ``` > SHOW MEASUREMENTS ON NOAA_water_database name: measurements ## name average_temperature h2o_feet h2o_pH h2o_quality h2o_temperature ``` The query returns the list of measurements in the `NOAA_water_database`database. The database has five measurements: `average_temperature`, `h2o_feet`,`h2o_pH`, `h2o_quality`, and `h2o_temperature`. #### Run a `SHOW MEASUREMENTS` query without the `ON` clause [CLI](#cli-3)[InfluxDB API](#influxdb-api-3) Specify the database with `USE ` ``` > USE NOAA_water_database Using database NOAA_water_database > SHOW MEASUREMENTS name: measurements ## name average_temperature h2o_feet h2o_pH h2o_quality h2o_temperature ``` Specify the database with the `db` query string parameter: ``` ~# curl -G "http://localhost:8086/query?db=NOAA_water_database&pretty=true" --data-urlencode "q=SHOW MEASUREMENTS" { { "results": [ { "statement_id": 0, "series": [ { "name": "measurements", "columns": [ "name" ], "values": [ [ "average_temperature" ], [ "h2o_feet" ], [ "h2o_pH" ], [ "h2o_quality" ], [ "h2o_temperature" ] ] } ] } ] } ``` #### Run a `SHOW MEASUREMENTS` query with several clauses (i) ``` > SHOW MEASUREMENTS ON NOAA_water_database WITH MEASUREMENT =~ /h2o.*/ LIMIT 2 OFFSET 1 name: measurements ## name h2o_pH h2o_quality ``` The query returns the measurements in the `NOAA_water_database` database that start with `h2o`. The `LIMIT` and `OFFSET` clauses limit the number of measurement names returned to two and offset the results by one, skipping the `h2o_feet` measurement. #### Run a `SHOW MEASUREMENTS` query with several clauses (ii) ``` > SHOW MEASUREMENTS ON NOAA_water_database WITH MEASUREMENT =~ /h2o.*/ WHERE "randtag" =~ /\d/ name: measurements ## name h2o_quality ``` The query returns all measurements in the `NOAA_water_database` that start with `h2o` and have values for the tag key `randtag` that include an integer. ## `SHOW TAG KEYS` Returns a list of [tag keys](/influxdb/v1/concepts/glossary/#tag-key)associated with the specified [database](/influxdb/v1/concepts/glossary/#database). ### Syntax ```sql SHOW TAG KEYS [ON ] [FROM_clause] [WHERE ['' | ]] [LIMIT_clause] [OFFSET_clause] ``` ### Description of syntax `ON ` is optional. If the query does not include `ON `, you must specify the database with `USE ` in the [CLI](/influxdb/v1/tools/shell/) or with the `db` query string parameter in the [InfluxDB API](/influxdb/v1/api/query/) request. The `FROM` clause and the `WHERE` clause are optional. The `WHERE` clause supports tag comparisons; field comparisons are not valid for the `SHOW TAG KEYS` query. Supported operators in the `WHERE` clause:`=` equal to`<>` not equal to`!=` not equal to`=~` matches against`!~` doesn’t match against See the Data Exploration page for documentation on the[`FROM` clause](/influxdb/v1/query_language/explore-data/#the-basic-select-statement),[`LIMIT` clause](/influxdb/v1/query_language/explore-data/#the-limit-clause),[`OFFSET` clause](/influxdb/v1/query_language/explore-data/#the-offset-clause), and on [Regular Expressions in Queries](/influxdb/v1/query_language/explore-data/#regular-expressions). ### Examples #### Run a `SHOW TAG KEYS` query with the `ON` clause ``` > SHOW TAG KEYS ON "NOAA_water_database" name: average_temperature ## tagKey location name: h2o_feet ## tagKey location name: h2o_pH ## tagKey location name: h2o_quality ## tagKey location randtag name: h2o_temperature ## tagKey location ``` The query returns the list of tag keys in the `NOAA_water_database` database. The output groups tag keys by measurement name; it shows that every measurement has the `location` tag key and that the`h2o_quality` measurement has an additional `randtag` tag key. #### Run a `SHOW TAG KEYS` query without the `ON` clause [CLI](#cli-4)[InfluxDB API](#influxdb-api-4) Specify the database with `USE ` ``` > USE NOAA_water_database Using database NOAA_water_database > SHOW TAG KEYS name: average_temperature ## tagKey location name: h2o_feet ## tagKey location name: h2o_pH ## tagKey location name: h2o_quality ## tagKey location randtag name: h2o_temperature ## tagKey location ``` Specify the database with the `db` query string parameter: ```sql ~# curl -G "http://localhost:8086/query?db=NOAA_water_database&pretty=true" --data-urlencode "q=SHOW TAG KEYS" { "results": [ { "statement_id": 0, "series": [ { "name": "average_temperature", "columns": [ "tagKey" ], "values": [ [ "location" ] ] }, { "name": "h2o_feet", "columns": [ "tagKey" ], "values": [ [ "location" ] ] }, { "name": "h2o_pH", "columns": [ "tagKey" ], "values": [ [ "location" ] ] }, { "name": "h2o_quality", "columns": [ "tagKey" ], "values": [ [ "location" ], [ "randtag" ] ] }, { "name": "h2o_temperature", "columns": [ "tagKey" ], "values": [ [ "location" ] ] } ] } ] } ``` #### Run a `SHOW TAG KEYS` query with several clauses ``` > SHOW TAG KEYS ON "NOAA_water_database" FROM "h2o_quality" LIMIT 1 OFFSET 1 name: h2o_quality ## tagKey randtag ``` The query returns tag keys from the `h2o_quality` measurement in the`NOAA_water_database` database. The `LIMIT` and `OFFSET` clauses limit the number of tag keys returned to one and offsets the results by one. ## `SHOW TAG VALUES` Returns the list of [tag values](/influxdb/v1/concepts/glossary/#tag-value)for the specified [tag key(s)](/influxdb/v1/concepts/glossary/#tag-key) in the database. ### Syntax ```sql SHOW TAG VALUES [ON ][FROM_clause] WITH KEY [ [ "" | ] | [IN (""," ['' | ]] [LIMIT_clause] [OFFSET_clause] ``` ### Description of syntax `ON ` is optional. If the query does not include `ON `, you must specify the database with `USE ` in the [CLI](/influxdb/v1/tools/shell/) or with the `db` query string parameter in the [InfluxDB API](/influxdb/v1/api/query/) request. The `WITH` clause is required. It supports specifying a single tag key, a regular expression, and multiple tag keys. The `FROM`, `WHERE`, `LIMIT`, and `OFFSET` clauses are optional. The `WHERE` clause supports tag comparisons; field comparisons are not valid for the `SHOW TAG KEYS` query. Supported operators in the `WITH` and `WHERE` clauses:`=` equal to`<>` not equal to`!=` not equal to`=~` matches against`!~` doesn’t match against See the Data Exploration page for documentation on the[`FROM` clause](/influxdb/v1/query_language/explore-data/#the-basic-select-statement),[`LIMIT` clause](/influxdb/v1/query_language/explore-data/#the-limit-clause),[`OFFSET` clause](/influxdb/v1/query_language/explore-data/#the-offset-clause), and on [Regular Expressions in Queries](/influxdb/v1/query_language/explore-data/#regular-expressions). ### Examples #### Run a `SHOW TAG VALUES` query with the `ON` clause ```sql > SHOW TAG VALUES ON "NOAA_water_database" WITH KEY = "randtag" name: h2o_quality key value --- ----- randtag 1 randtag 2 randtag 3 ``` The query returns all tag values of the `randtag` tag key in the `NOAA_water_database`database.`SHOW TAG VALUES` groups query results by measurement name. #### Run a `SHOW TAG VALUES` query without the `ON` clause [CLI](#cli-5)[InfluxDB API](#influxdb-api-5) Specify the database with `USE ` ```sql > USE NOAA_water_database Using database NOAA_water_database > SHOW TAG VALUES WITH KEY = "randtag" name: h2o_quality key value --- ----- randtag 1 randtag 2 randtag 3 ``` Specify the database with the `db` query string parameter: ```bash ~# curl -G "http://localhost:8086/query?db=NOAA_water_database&pretty=true" --data-urlencode 'q=SHOW TAG VALUES WITH KEY = "randtag"' { "results": [ { "statement_id": 0, "series": [ { "name": "h2o_quality", "columns": [ "key", "value" ], "values": [ [ "randtag", "1" ], [ "randtag", "2" ], [ "randtag", "3" ] ] } ] } ] } ``` #### Run a `SHOW TAG VALUES` query with several clauses ```sql > SHOW TAG VALUES ON "NOAA_water_database" WITH KEY IN ("location","randtag") WHERE "randtag" =~ /./ LIMIT 3 name: h2o_quality key value --- ----- location coyote_creek location santa_monica randtag 1 ``` The query returns the tag values of the tag keys `location` and `randtag` for all measurements in the `NOAA_water_database` database where `randtag` has tag values. The `LIMIT` clause limits the number of tag values returned to three. ## `SHOW FIELD KEYS` Returns the [field keys](/influxdb/v1/concepts/glossary/#field-key) and the[data type](/influxdb/v1/write_protocols/line_protocol_reference/#data-types) of their[field values](/influxdb/v1/concepts/glossary/#field-value). ### Syntax ```sql SHOW FIELD KEYS [ON ] [FROM ] ``` ### Description of syntax `ON ` is optional. If the query does not include `ON `, you must specify the database with `USE ` in the [CLI](/influxdb/v1/tools/shell/) or with the `db` query string parameter in the [InfluxDB API](/influxdb/v1/api/query/) request. The `FROM` clause is also optional. See the Data Exploration page for documentation on the[`FROM` clause](/influxdb/v1/query_language/explore-data/#the-basic-select-statement). > > > **Note:** A field’s data type [can differ](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-does-influxdb-handle-field-type-discrepancies-across-shards) across[shards](/influxdb/v1/concepts/glossary/#shard). > If your field has more than one type, `SHOW FIELD KEYS` returns the type that > occurs first in the following list: float, integer, string, boolean. > > ### Examples #### Run a `SHOW FIELD KEYS` query with the `ON` clause ```sql > SHOW FIELD KEYS ON "NOAA_water_database" name: average_temperature fieldKey fieldType -------- --------- degrees float name: h2o_feet fieldKey fieldType -------- --------- level description string water_level float name: h2o_pH fieldKey fieldType -------- --------- pH float name: h2o_quality fieldKey fieldType -------- --------- index float name: h2o_temperature fieldKey fieldType -------- --------- degrees float ``` The query returns the field keys and field value data types for each measurement in the `NOAA_water_database` database. #### Run a `SHOW FIELD KEYS` query without the `ON` clause [CLI](#cli-6)[InfluxDB API](#influxdb-api-6) Specify the database with `USE ` ```sql > USE NOAA_water_database Using database NOAA_water_database > SHOW FIELD KEYS name: average_temperature fieldKey fieldType -------- --------- degrees float name: h2o_feet fieldKey fieldType -------- --------- level description string water_level float name: h2o_pH fieldKey fieldType -------- --------- pH float name: h2o_quality fieldKey fieldType -------- --------- index float name: h2o_temperature fieldKey fieldType -------- --------- degrees float ``` Specify the database with the `db` query string parameter: ```bash ~# curl -G "http://localhost:8086/query?db=NOAA_water_database&pretty=true" --data-urlencode 'q=SHOW FIELD KEYS' { "results": [ { "statement_id": 0, "series": [ { "name": "average_temperature", "columns": [ "fieldKey", "fieldType" ], "values": [ [ "degrees", "float" ] ] }, { "name": "h2o_feet", "columns": [ "fieldKey", "fieldType" ], "values": [ [ "level description", "string" ], [ "water_level", "float" ] ] }, { "name": "h2o_pH", "columns": [ "fieldKey", "fieldType" ], "values": [ [ "pH", "float" ] ] }, { "name": "h2o_quality", "columns": [ "fieldKey", "fieldType" ], "values": [ [ "index", "float" ] ] }, { "name": "h2o_temperature", "columns": [ "fieldKey", "fieldType" ], "values": [ [ "degrees", "float" ] ] } ] } ] } ``` #### Run a `SHOW FIELD KEYS` query with the `FROM` clause ```sql > SHOW FIELD KEYS ON "NOAA_water_database" FROM "h2o_feet" name: h2o_feet fieldKey fieldType -------- --------- level description string water_level float ``` The query returns the fields keys and field value data types for the `h2o_feet`measurement in the `NOAA_water_database` database. ### Common Issues with `SHOW FIELD KEYS` #### SHOW FIELD KEYS and field type discrepancies Field value[data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types)cannot differ within a [shard](/influxdb/v1/concepts/glossary/#shard) but they can differ across shards.`SHOW FIELD KEYS` returns every data type, across every shard, associated with the field key. ##### Example The `all_the_types` field stores four different data types: ```sql > SHOW FIELD KEYS name: mymeas fieldKey fieldType -------- --------- all_the_types integer all_the_types float all_the_types string all_the_types boolean ``` Note that `SHOW FIELD KEYS` handles field type discrepancies differently from`SELECT` statements. For more information, see the[How does InfluxDB handle field type discrepancies across shards?](/influxdb/v1/troubleshooting/frequently-asked-questions/#how-does-influxdb-handle-field-type-discrepancies-across-shards). ## `SHOW CARDINALITY` `SHOW CARDINALITY` refers to the group of commands used to estimate or count exactly the cardinality of measurements, series, tag keys, tag key values, and field keys. For more information on the `SHOW CARDINALITY` commands, see the [InfluxQL reference entry](/enterprise_influxdb/v1/query_language/spec/#show-cardinality). ### `SHOW FIELD KEY CARDINALITY` ```sql -- show estimated cardinality of the field key set of current database SHOW FIELD KEY CARDINALITY -- show exact cardinality on field key set of specified database SHOW FIELD KEY EXACT CARDINALITY ON mydb ``` ### `SHOW MEASUREMENT KEY CARDINALITY` ```sql -- show estimated cardinality of measurement set on current database SHOW MEASUREMENT CARDINALITY -- show exact cardinality of measurement set on specified database SHOW MEASUREMENT EXACT CARDINALITY ON mydb ``` ### `SHOW SERIES CARDINALITY` ```sql -- show estimated cardinality of the series on current database SHOW SERIES CARDINALITY -- show estimated cardinality of the series on specified database SHOW SERIES CARDINALITY ON mydb -- show exact series cardinality SHOW SERIES EXACT CARDINALITY -- show series cardinality of the series on specified database SHOW SERIES EXACT CARDINALITY ON mydb ``` ### `SHOW TAG KEY CARDINALITY` ```sql -- show estimated tag key cardinality SHOW TAG KEY CARDINALITY -- show exact tag key cardinality SHOW TAG KEY EXACT CARDINALITY ``` ### `SHOW TAG VALUES CARDINALITY` ```sql -- show estimated tag key values cardinality for a specified tag key SHOW TAG VALUES CARDINALITY WITH KEY = "myTagKey" -- show estimated tag key values cardinality for a specified tag key SHOW TAG VALUES CARDINALITY WITH KEY = "myTagKey" -- show exact tag key values cardinality for a specified tag key SHOW TAG VALUES EXACT CARDINALITY WITH KEY = "myTagKey" -- show exact tag key values cardinality for a specified tag key SHOW TAG VALUES EXACT CARDINALITY WITH KEY = "myTagKey" ``` ## Filter meta queries by time When you filter meta queries by time, you may see results outside of your specified time. Meta query results are filtered at the shard level, so results can be approximately as granular as your shard group duration. If your time filter spans multiple shards, you’ll get results from all shards with points in the specified time range. To review your shards and timestamps on points in the shard, run `SHOW SHARDS`. To learn more about shards and their duration, see [recommended shard groups durations](/influxdb/v1/concepts/schema_and_data_layout/#shard-group-duration-recommendations). The example below shows how to filter `SHOW TAG KEYS` by approximately one hour using a 1h shard group duration. To filter other meta data, replace `SHOW TAG KEYS` with `SHOW TAG VALUES`, `SHOW SERIES`, `SHOW FIELD KEYS`, and so on. > > > **Note:** `SHOW MEASUREMENTS` cannot be filtered by time. > > #### Example filtering `SHOW TAG KEYS` by time 1. Specify a shard duration on a new database or [alter an existing shard duration](/influxdb/v1/query_language/manage-database/#modify-retention-policies-with-alter-retention-policy). To specify a 1h shard duration when creating a new database, run the following command: ``` > CREATE database mydb with duration 7d REPLICATION 1 SHARD DURATION 1h name myRP; ``` > > > **Note:** The minimum shard duration is 1h. > > 2. Verify the shard duration has the correct time interval (precision) by running the `SHOW SHARDS` command. The example below shows a shard duration with an hour precision. ``` > SHOW SHARDS name: mydb id database retention_policy shard_group start_time end_time expiry_time owners -- -------- ---------------- ----------- ---------- -------- ----------- ------ > precision h ``` 3. (Optional) Insert sample tag keys. This step is for demonstration purposes. If you already have tag keys (or other meta data) to search for, skip this step. ``` // Insert a sample tag called "test_key" into the "test" measurement, and then check the timestamp: > INSERT test,test_key=hello value=1 > select * from test name: test time test_key value ---- -------- ----- 434820 hello 1 // Add new tag keys with timestamps one, two, and three hours earlier: > INSERT test,test_key_1=hello value=1 434819 > INSERT test,test_key_2=hello value=1 434819 > INSERT test,test_key_3_=hello value=1 434818 > INSERT test,test_key_4=hello value=1 434817 > INSERT test,test_key_5_=hello value=1 434817 ``` 4. To find tag keys within a shard duration, run one of the following commands: `SHOW TAG KEYS ON database-name ` OR `SELECT * FROM measurement ` The examples below use test data from step 3. ``` //Using data from Step 3, show tag keys between now and an hour ago > SHOW TAG KEYS ON mydb where time > now() -1h and time < now() name: test tagKey ------ test_key test_key_1 test_key_2 // Find tag keys between one and two hours ago > SHOW TAG KEYS ON mydb where > time > now() -2h and time < now()-1h name: test tagKey ------ test_key_1 test_key_2 test_key_3 // Find tag keys between two and three hours ago > SHOW TAG KEYS ON mydb where > time > now() -3h and time < now()-2h name: test tagKey ------ test_key_3 test_key_4 test_key_5 // For a specified measurement, find tag keys in a given shard by specifying the time boundaries of the shard > SELECT * FROM test WHERE time >= '2019-08-09T00:00:00Z' and time < '2019-08-09T10:00:00Z' name: test time test_key_4 test_key_5 value ---- ------------ ------------ ----- 434817 hello 1 434817 hello 1 // For a specified database, find tag keys in a given shard by specifying the time boundaries of the shard > SHOW TAG KEYS ON mydb WHERE time >= '2019-08-09T00:00:00Z' and time < '2019-08-09T10:00:00Z' name: test tagKey ------ test_key_4 test_key_5 ``` | SHOW DATABASES | SHOW RETENTION POLICIES | SHOW SERIES | | --- | --- | --- | | SHOW MEASUREMENTS | SHOW TAG KEYS | SHOW TAG VALUES | | SHOW FIELD KEYS | SHOW CARDINALITY | Filter meta queries by time | | --- # InfluxQL functions Source: https://docs.influxdata.com/influxdb/v1/query_language/functions/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. Aggregate, select, transform, and predict data with InfluxQL functions. #### Content * [Aggregations](#aggregations) * [COUNT()](#count) * [DISTINCT()](#distinct) * [INTEGRAL()](#integral) * [MEAN()](#mean) * [MEDIAN()](#median) * [MODE()](#mode) * [SPREAD()](#spread) * [STDDEV()](#stddev) * [SUM()](#sum) * [Selectors](#selectors) * [BOTTOM()](#bottom) * [FIRST()](#first) * [LAST()](#last) * [MAX()](#max) * [MIN()](#min) * [PERCENTILE()](#percentile) * [SAMPLE()](#sample) * [TOP()](#top) * [Transformations](#transformations) * [ABS()](#abs) * [ACOS()](#acos) * [ASIN()](#asin) * [ATAN()](#atan) * [ATAN2()](#atan2) * [CEIL()](#ceil) * [COS()](#cos) * [CUMULATIVE\_SUM()](#cumulative_sum) * [DERIVATIVE()](#derivative) * [DIFFERENCE()](#difference) * [ELAPSED()](#elapsed) * [EXP()](#exp) * [FLOOR()](#floor) * [HISTOGRAM()](#histogram) * [LN()](#ln) * [LOG()](#log) * [LOG2()](#log2) * [LOG10()](#log10) * [MOVING\_AVERAGE()](#moving_average) * [NON\_NEGATIVE\_DERIVATIVE()](#non_negative_derivative) * [NON\_NEGATIVE\_DIFFERENCE()](#non_negative_difference) * [POW()](#pow) * [ROUND()](#round) * [SIN()](#sin) * [SQRT()](#sqrt) * [TAN()](#tan) * [Predictors](#predictors) * [HOLT\_WINTERS()](#holt_winters) * [Technical Analysis](#technical-analysis) * [CHANDE\_MOMENTUM\_OSCILLATOR()](#chande_momentum_oscillator) * [EXPONENTIAL\_MOVING\_AVERAGE()](#exponential_moving_average) * [DOUBLE\_EXPONENTIAL\_MOVING\_AVERAGE()](#double_exponential_moving_average) * [KAUFMANS\_EFFICIENCY\_RATIO()](#kaufmans_efficiency_ratio) * [KAUFMANS\_ADAPTIVE\_MOVING\_AVERAGE()](#kaufmans_adaptive_moving_average) * [TRIPLE\_EXPONENTIAL\_MOVING\_AVERAGE()](#triple_exponential_moving_average) * [TRIPLE\_EXPONENTIAL\_DERIVATIVE()](#triple_exponential_derivative) * [RELATIVE\_STRENGTH\_INDEX()](#relative_strength_index) * [Other](#other) * [Sample Data](#sample_data) * [General Syntax for Functions](#general_syntax_for_functions) * [Specify Multiple Functions in the SELECT clause](#specify_multiple_functions_in_the_select_clause) * [Rename the Output Field Key](#rename_the_output_field_key) * [Change the Values Reported for Intervals with no Data](#change_the_values_reported_for_intervals_with_no_data) * [Common Issues with Functions](#common_issues_with_functions) ## Aggregations ### COUNT() Returns the number of non-null [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT COUNT( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` ##### Nested Syntax ``` SELECT COUNT(DISTINCT( [ * | | // ] )) [...] ``` `COUNT(field_key)` Returns the number of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `COUNT(/regular_expression/)` Returns the number of field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `COUNT(*)` Returns the number of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `COUNT()` supports all field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). InfluxQL supports nesting [`DISTINCT()`](#distinct) with `COUNT()`. #### Examples ##### Count the field values associated with a field key ```sql > SELECT COUNT("water_level") FROM "h2o_feet" name: h2o_feet time count ---- ----- 1970-01-01T00:00:00Z 15258 ``` The query returns the number of non-null field values in the `water_level` field key in the `h2o_feet` measurement. ##### Count the field values associated with each field key in a measurement ```sql > SELECT COUNT(*) FROM "h2o_feet" name: h2o_feet time count_level description count_water_level ---- ----------------------- ----------------- 1970-01-01T00:00:00Z 15258 15258 ``` The query returns the number of non-null field values for each field key associated with the `h2o_feet` measurement. The `h2o_feet` measurement has two field keys: `level description` and `water_level`. ##### Count the field values associated with each field key that matches a regular expression ```sql > SELECT COUNT(/water/) FROM "h2o_feet" name: h2o_feet time count_water_level ---- ----------------- 1970-01-01T00:00:00Z 15258 ``` The query returns the number of non-null field values for every field key that contains the word `water` in the `h2o_feet` measurement. ##### Count the field values associated with a field key and include several clauses ```sql > SELECT COUNT("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(200) LIMIT 7 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time count ---- ----- 2015-08-17T23:48:00Z 200 2015-08-18T00:00:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:24:00Z 2 2015-08-18T00:36:00Z 2 2015-08-18T00:48:00Z 2 ``` The query returns the number of non-null field values in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `200` and [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to seven and one. ##### Count the distinct field values associated with a field key ```sql > SELECT COUNT(DISTINCT("level description")) FROM "h2o_feet" name: h2o_feet time count ---- ----- 1970-01-01T00:00:00Z 4 ``` The query returns the number of unique field values for the `level description` field key and the `h2o_feet` measurement. ### Common Issues with COUNT() #### COUNT() and fill() Most InfluxQL functions report `null` values for time intervals with no data, and[`fill()`](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill)replaces that `null` value with the `fill_option`.`COUNT()` reports `0` for time intervals with no data, and `fill()` replaces any `0` values with the `fill_option`. ##### Example The first query in the codeblock below does not include `fill()`. The last time interval has no data so the reported value for that time interval is zero. The second query includes `fill(800000)`; it replaces the zero in the last interval with `800000`. ```sql > SELECT COUNT("water_level") FROM "h2o_feet" WHERE time >= '2015-09-18T21:24:00Z' AND time <= '2015-09-18T21:54:00Z' GROUP BY time(12m) name: h2o_feet time count ---- ----- 2015-09-18T21:24:00Z 2 2015-09-18T21:36:00Z 2 2015-09-18T21:48:00Z 0 > SELECT COUNT("water_level") FROM "h2o_feet" WHERE time >= '2015-09-18T21:24:00Z' AND time <= '2015-09-18T21:54:00Z' GROUP BY time(12m) fill(800000) name: h2o_feet time count ---- ----- 2015-09-18T21:24:00Z 2 2015-09-18T21:36:00Z 2 2015-09-18T21:48:00Z 800000 ``` ### DISTINCT() Returns the list of unique [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT DISTINCT( [ | // ] ) FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` ##### Nested Syntax ``` SELECT COUNT(DISTINCT( [ | // ] )) [...] ``` `DISTINCT(field_key)` Returns the unique field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `DISTINCT()` supports all field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). InfluxQL supports nesting `DISTINCT()` with [`COUNT()`](#count). #### Examples ##### List the distinct field values associated with a field key ```sql > SELECT DISTINCT("level description") FROM "h2o_feet" name: h2o_feet time distinct ---- -------- 1970-01-01T00:00:00Z between 6 and 9 feet 1970-01-01T00:00:00Z below 3 feet 1970-01-01T00:00:00Z between 3 and 6 feet 1970-01-01T00:00:00Z at or greater than 9 feet ``` The query returns a tabular list of the unique field values in the `level description` field key in the `h2o_feet` measurement. ##### List the distinct field values associated with each field key in a measurement ```sql > SELECT DISTINCT(*) FROM "h2o_feet" name: h2o_feet time distinct_level description distinct_water_level ---- -------------------------- -------------------- 1970-01-01T00:00:00Z between 6 and 9 feet 8.12 1970-01-01T00:00:00Z between 3 and 6 feet 8.005 1970-01-01T00:00:00Z at or greater than 9 feet 7.887 1970-01-01T00:00:00Z below 3 feet 7.762 [...] ``` The query returns a tabular list of the unique field values for each field key in the `h2o_feet` measurement. The `h2o_feet` measurement has two field keys: `level description` and `water_level`. #### List the distinct field values associated with a field key and include several clauses ```sql > SELECT DISTINCT("level description") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* SLIMIT 1 name: h2o_feet tags: location=coyote_creek time distinct ---- -------- 2015-08-18T00:00:00Z between 6 and 9 feet 2015-08-18T00:12:00Z between 6 and 9 feet 2015-08-18T00:24:00Z between 6 and 9 feet 2015-08-18T00:36:00Z between 6 and 9 feet 2015-08-18T00:48:00Z between 6 and 9 feet ``` The query returns a tabular list of the unique field values in the `level description` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of series returned to one. ##### Count the distinct field values associated with a field key ```sql > SELECT COUNT(DISTINCT("level description")) FROM "h2o_feet" name: h2o_feet time count ---- ----- 1970-01-01T00:00:00Z 4 ``` The query returns the number of unique field values in the `level description` field key and the `h2o_feet` measurement. ### Common Issues with DISTINCT() #### DISTINCT() and the INTO clause Using `DISTINCT()` with the [`INTO` clause](/influxdb/v1/query_language/explore-data/#the-into-clause) can cause InfluxDB to overwrite points in the destination measurement.`DISTINCT()` often returns several results with the same timestamp; InfluxDB assumes [points](/influxdb/v1/concepts/glossary/#point) with the same [series](/influxdb/v1/concepts/glossary/#series) and timestamp are duplicate points and simply overwrites any duplicate point with the most recent point in the destination measurement. ##### Example The first query in the codeblock below uses the `DISTINCT()` function and returns four results. Notice that each result has the same timestamp. The second query adds an `INTO` clause to the initial query and writes the query results to the `distincts` measurement. The last query in the code block selects all the data in the `distincts` measurement. The last query returns one point because the four initial results are duplicate points; they belong to the same series and have the same timestamp. When the system encounters duplicate points, it simply overwrites the previous point with the most recent point. ```sql > SELECT DISTINCT("level description") FROM "h2o_feet" name: h2o_feet time distinct ---- -------- 1970-01-01T00:00:00Z below 3 feet 1970-01-01T00:00:00Z between 6 and 9 feet 1970-01-01T00:00:00Z between 3 and 6 feet 1970-01-01T00:00:00Z at or greater than 9 feet > SELECT DISTINCT("level description") INTO "distincts" FROM "h2o_feet" name: result time written ---- ------- 1970-01-01T00:00:00Z 4 > SELECT * FROM "distincts" name: distincts time distinct ---- -------- 1970-01-01T00:00:00Z at or greater than 9 feet ``` ### INTEGRAL() Returns the area under the curve for subsequent [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT INTEGRAL( [ * | | // ] [ , ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` InfluxDB calculates the area under the curve for subsequent field values and converts those results into the summed area per `unit`. The `unit` argument is an integer followed by a [duration literal](/influxdb/v1/query_language/spec/#literals) and it is optional. If the query does not specify the `unit`, the unit defaults to one second (`1s`). `INTEGRAL(field_key)` Returns the area under the curve for subsequent field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `INTEGRAL(/regular_expression/)` Returns the area under the curve for subsequent field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `INTEGRAL(*)` Returns the average field value associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `INTEGRAL()` does not support [`fill()`](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill). `INTEGRAL()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples Examples 1-5 use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ##### Calculate the integral for the field values associated with a field key ```sql > SELECT INTEGRAL("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time integral ---- -------- 1970-01-01T00:00:00Z 3732.66 ``` The query returns the area under the curve (in seconds) for the field values associated with the `water_level` field key and in the `h2o_feet` measurement. ##### Calculate the integral for the field values associated with a field key and specify the unit option ```sql > SELECT INTEGRAL("water_level",1m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time integral ---- -------- 1970-01-01T00:00:00Z 62.211 ``` The query returns the area under the curve (in minutes) for the field values associated with the `water_level` field key and in the `h2o_feet` measurement. ##### Calculate the integral for the field values associated with each field key in a measurement and specify the unit option ```sql > SELECT INTEGRAL(*,1m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time integral_water_level ---- -------------------- 1970-01-01T00:00:00Z 62.211 ``` The query returns the area under the curve (in minutes) for the field values associated with each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has on numerical field: `water_level`. #### Calculate the integral for the field values associated with each field key that matches a regular expression and specify the unit option ```sql > SELECT INTEGRAL(/water/,1m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time integral_water_level ---- -------------------- 1970-01-01T00:00:00Z 62.211 ``` The query returns the area under the curve (in minutes) for the field values associated with each field key that stores numerical values includes the word `water` in the `h2o_feet` measurement. #### Calculate the integral for the field values associated with a field key and include several clauses ```sql > SELECT INTEGRAL("water_level",1m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) LIMIT 1 name: h2o_feet time integral ---- -------- 2015-08-18T00:00:00Z 24.972 ``` The query returns the area under the curve (in minutes) for the field values associated with the `water_level` field key and in the `h2o_feet` measurement. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z`, [groups](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) results into 12-minute intervals, and [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of results returned to one. ### MEAN() Returns the arithmetic mean (average) of [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT MEAN( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `MEAN(field_key)` Returns the average field value associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `MEAN(/regular_expression/)` Returns the average field value associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `MEAN(*)` Returns the average field value associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `MEAN()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Calculate the mean field value associated with a field key ```sql > SELECT MEAN("water_level") FROM "h2o_feet" name: h2o_feet time mean ---- ---- 1970-01-01T00:00:00Z 4.442107025822522 ``` The query returns the average field value in the `water_level` field key in the `h2o_feet` measurement. ##### Calculate the mean field value associated with each field key in a measurement ```sql > SELECT MEAN(*) FROM "h2o_feet" name: h2o_feet time mean_water_level ---- ---------------- 1970-01-01T00:00:00Z 4.442107025822522 ``` The query returns the average field value for every field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ##### Calculate the mean field value associated with each field key that matches a regular expression ```sql > SELECT MEAN(/water/) FROM "h2o_feet" name: h2o_feet time mean_water_level ---- ---------------- 1970-01-01T00:00:00Z 4.442107025822523 ``` The query returns the average field value for each field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. #### Calculate the mean field value associated with a field key and include several clauses ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(9.01) LIMIT 7 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time mean ---- ---- 2015-08-17T23:48:00Z 9.01 2015-08-18T00:00:00Z 8.0625 2015-08-18T00:12:00Z 7.8245 2015-08-18T00:24:00Z 7.5675 2015-08-18T00:36:00Z 7.303 2015-08-18T00:48:00Z 7.046 ``` The query returns the average of the values in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `9.01` and [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to seven and one. ### MEDIAN() Returns the middle value from a sorted list of [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT MEDIAN( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `MEDIAN(field_key)` Returns the middle field value associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `MEDIAN(/regular_expression/)` Returns the middle field value associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `MEDIAN(*)` Returns the middle field value associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `MEDIAN()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). > > > **Note:** `MEDIAN()` is nearly equivalent to [`PERCENTILE(field_key, 50)`](#percentile), except `MEDIAN()` returns the average of the two middle field values if the field contains an even number of values. > > #### Examples ##### Calculate the median field value associated with a field key ```sql > SELECT MEDIAN("water_level") FROM "h2o_feet" name: h2o_feet time median ---- ------ 1970-01-01T00:00:00Z 4.124 ``` The query returns the middle field value in the `water_level` field key and in the `h2o_feet` measurement. ##### Calculate the median field value associated with each field key in a measurement ```sql > SELECT MEDIAN(*) FROM "h2o_feet" name: h2o_feet time median_water_level ---- ------------------ 1970-01-01T00:00:00Z 4.124 ``` The query returns the middle field value for every field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ##### Calculate the median field value associated with each field key that matches a regular expression ```sql > SELECT MEDIAN(/water/) FROM "h2o_feet" name: h2o_feet time median_water_level ---- ------------------ 1970-01-01T00:00:00Z 4.124 ``` The query returns the middle field value for every field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. #### Calculate the median field value associated with a field key and include several clauses ```sql > SELECT MEDIAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(700) LIMIT 7 SLIMIT 1 SOFFSET 1 name: h2o_feet tags: location=santa_monica time median ---- ------ 2015-08-17T23:48:00Z 700 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 2015-08-18T00:36:00Z 2.0620000000000003 2015-08-18T00:48:00Z 700 ``` The query returns the middle field value in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `700 `, [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to seven and one, and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) the series returned by one. ### MODE() Returns the most frequent value in a list of [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT MODE( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `MODE(field_key)` Returns the most frequent field value associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `MODE(/regular_expression/)` Returns the most frequent field value associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `MODE(*)` Returns the most frequent field value associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `MODE()` supports all field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). > > > **Note:** `MODE()` returns the field value with the earliest [timestamp](/influxdb/v1/concepts/glossary/#timestamp) if there’s a tie between two or more values for the maximum number of occurrences. > > #### Examples ##### Calculate the mode field value associated with a field key ```sql > SELECT MODE("level description") FROM "h2o_feet" name: h2o_feet time mode ---- ---- 1970-01-01T00:00:00Z between 3 and 6 feet ``` The query returns the most frequent field value in the `level description` field key and in the `h2o_feet` measurement. ##### Calculate the mode field value associated with each field key in a measurement ```sql > SELECT MODE(*) FROM "h2o_feet" name: h2o_feet time mode_level description mode_water_level ---- ---------------------- ---------------- 1970-01-01T00:00:00Z between 3 and 6 feet 2.69 ``` The query returns the most frequent field value for every field key in the `h2o_feet` measurement. The `h2o_feet` measurement has two field keys: `level description` and `water_level`. ##### Calculate the mode field value associated with each field key that matches a regular expression ```sql > SELECT MODE(/water/) FROM "h2o_feet" name: h2o_feet time mode_water_level ---- ---------------- 1970-01-01T00:00:00Z 2.69 ``` The query returns the most frequent field value for every field key that includes the word `/water/` in the `h2o_feet` measurement. #### Calculate the mode field value associated with a field key and include several clauses ```sql > SELECT MODE("level description") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* LIMIT 3 SLIMIT 1 SOFFSET 1 name: h2o_feet tags: location=santa_monica time mode ---- ---- 2015-08-17T23:48:00Z 2015-08-18T00:00:00Z below 3 feet 2015-08-18T00:12:00Z below 3 feet ``` The query returns the mode of the values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to three and one, and it [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) the series returned by one. ### SPREAD() Returns the difference between the minimum and maximum [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT SPREAD( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `SPREAD(field_key)` Returns the difference between the minimum and maximum field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `SPREAD(/regular_expression/)` Returns the difference between the minimum and maximum field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `SPREAD(*)` Returns the difference between the minimum and maximum field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `SPREAD()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Calculate the spread for the field values associated with a field key ```sql > SELECT SPREAD("water_level") FROM "h2o_feet" name: h2o_feet time spread ---- ------ 1970-01-01T00:00:00Z 10.574 ``` The query returns the difference between the minimum and maximum field values in the `water_level` field key and in the `h2o_feet` measurement. ##### Calculate the spread for the field values associated with each field key in a measurement ```sql > SELECT SPREAD(*) FROM "h2o_feet" name: h2o_feet time spread_water_level ---- ------------------ 1970-01-01T00:00:00Z 10.574 ``` The query returns the difference between the minimum and maximum field values for every field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ##### Calculate the spread for the field values associated with each field key that matches a regular expression ```sql > SELECT SPREAD(/water/) FROM "h2o_feet" name: h2o_feet time spread_water_level ---- ------------------ 1970-01-01T00:00:00Z 10.574 ``` The query returns the difference between the minimum and maximum field values for every field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. #### Calculate the spread for the field values associated with a field key and include several clauses ```sql > SELECT SPREAD("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(18) LIMIT 3 SLIMIT 1 SOFFSET 1 name: h2o_feet tags: location=santa_monica time spread ---- ------ 2015-08-17T23:48:00Z 18 2015-08-18T00:00:00Z 0.052000000000000046 2015-08-18T00:12:00Z 0.09799999999999986 ``` The query returns the difference between the minimum and maximum field values in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z `and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `18`, [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to three and one, and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) the series returned by one. ### STDDEV() Returns the standard deviation of [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT STDDEV( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `STDDEV(field_key)` Returns the standard deviation of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `STDDEV(/regular_expression/)` Returns the standard deviation of field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `STDDEV(*)` Returns the standard deviation of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `STDDEV()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Calculate the standard deviation for the field values associated with a field key ```sql > SELECT STDDEV("water_level") FROM "h2o_feet" name: h2o_feet time stddev ---- ------ 1970-01-01T00:00:00Z 2.279144584196141 ``` The query returns the standard deviation of the field values in the `water_level` field key and in the `h2o_feet` measurement. ##### Calculate the standard deviation for the field values associated with each field key in a measurement ```sql > SELECT STDDEV(*) FROM "h2o_feet" name: h2o_feet time stddev_water_level ---- ------------------ 1970-01-01T00:00:00Z 2.279144584196141 ``` The query returns the standard deviation of the field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ##### Calculate the standard deviation for the field values associated with each field key that matches a regular expression ```sql > SELECT STDDEV(/water/) FROM "h2o_feet" name: h2o_feet time stddev_water_level ---- ------------------ 1970-01-01T00:00:00Z 2.279144584196141 ``` The query returns the standard deviation of the field values for each field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. #### Calculate the standard deviation for the field values associated with a field key and include several clauses ```sql > SELECT STDDEV("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(18000) LIMIT 2 SLIMIT 1 SOFFSET 1 name: h2o_feet tags: location=santa_monica time stddev ---- ------ 2015-08-17T23:48:00Z 18000 2015-08-18T00:00:00Z 0.03676955262170051 ``` The query returns the standard deviation of the field values in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `18000`, [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to two and one, and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) the series returned by one. ### SUM() Returns the sum of [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT SUM( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `SUM(field_key)` Returns the sum of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `SUM(/regular_expression/)` Returns the sum of field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `SUM(*)` Returns the sums of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `SUM()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples #### Calculate the sum of the field values associated with a field key ```sql > SELECT SUM("water_level") FROM "h2o_feet" name: h2o_feet time sum ---- --- 1970-01-01T00:00:00Z 67777.66900000004 ``` The query returns the summed total of the field values in the `water_level` field key and in the `h2o_feet` measurement. #### Calculate the sum of the field values associated with each field key in a measurement ```sql > SELECT SUM(*) FROM "h2o_feet" name: h2o_feet time sum_water_level ---- --------------- 1970-01-01T00:00:00Z 67777.66900000004 ``` The query returns the summed total of the field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. #### Calculate the sum of the field values associated with each field key that matches a regular expression ```sql > SELECT SUM(/water/) FROM "h2o_feet" name: h2o_feet time sum_water_level ---- --------------- 1970-01-01T00:00:00Z 67777.66900000004 ``` The query returns the summed total of the field values for each field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. #### Calculate the sum of the field values associated with a field key and include several clauses ```sql > SELECT SUM("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(18000) LIMIT 4 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time sum ---- --- 2015-08-17T23:48:00Z 18000 2015-08-18T00:00:00Z 16.125 2015-08-18T00:12:00Z 15.649 2015-08-18T00:24:00Z 15.135 ``` The query returns the summed total of the field values in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with 18000, and it [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to four and one. ## Selectors ### BOTTOM() Returns the smallest `N` [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT BOTTOM([,], )[,|] [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `BOTTOM(field_key,N)` Returns the smallest N field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `BOTTOM(field_key,tag_key(s),N)` Returns the smallest field value for N tag values of the [tag key](/influxdb/v1/concepts/glossary/#tag-key). `BOTTOM(field_key,N),tag_key(s),field_key(s)` Returns the smallest N field values associated with the field key in the parentheses and the relevant [tag](/influxdb/v1/concepts/glossary/#tag) and/or [field](/influxdb/v1/concepts/glossary/#field). `BOTTOM()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). > > > **Notes:** > > * `BOTTOM()` returns the field value with the earliest timestamp if there’s a tie between two or more values for the smallest value. * `BOTTOM()` differs from other InfluxQL functions when combined with an [`INTO` clause](/influxdb/v1/query_language/explore-data/#the-into-clause). See the [Common Issues](#common-issues-with-bottom) section for more information. #### Examples ##### Select the bottom three field values associated with a field key ```sql > SELECT BOTTOM("water_level",3) FROM "h2o_feet" name: h2o_feet time bottom ---- ------ 2015-08-29T14:30:00Z -0.61 2015-08-29T14:36:00Z -0.591 2015-08-30T15:18:00Z -0.594 ``` The query returns the smallest three field values in the `water_level` field key and in the `h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement). ##### Select the bottom field value associated with a field key for two tags ```sql > SELECT BOTTOM("water_level","location",2) FROM "h2o_feet" name: h2o_feet time bottom location ---- ------ -------- 2015-08-29T10:36:00Z -0.243 santa_monica 2015-08-29T14:30:00Z -0.61 coyote_creek ``` The query returns the smallest field values in the `water_level` field key for two tag values associated with the `location` tag key. ##### Select the bottom four field values associated with a field key and the relevant tags and fields ```sql > SELECT BOTTOM("water_level",4),"location","level description" FROM "h2o_feet" name: h2o_feet time bottom location level description ---- ------ -------- ----------------- 2015-08-29T14:24:00Z -0.587 coyote_creek below 3 feet 2015-08-29T14:30:00Z -0.61 coyote_creek below 3 feet 2015-08-29T14:36:00Z -0.591 coyote_creek below 3 feet 2015-08-30T15:18:00Z -0.594 coyote_creek below 3 feet ``` The query returns the smallest four field values in the `water_level` field key and the relevant values of the `location` tag key and the `level description` field key. ##### Select the bottom three field values associated with a field key and include several clauses ```sql > SELECT BOTTOM("water_level",3),"location" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(24m) ORDER BY time DESC name: h2o_feet time bottom location ---- ------ -------- 2015-08-18T00:48:00Z 1.991 santa_monica 2015-08-18T00:54:00Z 2.054 santa_monica 2015-08-18T00:54:00Z 6.982 coyote_creek 2015-08-18T00:24:00Z 2.041 santa_monica 2015-08-18T00:30:00Z 2.051 santa_monica 2015-08-18T00:42:00Z 2.057 santa_monica 2015-08-18T00:00:00Z 2.064 santa_monica 2015-08-18T00:06:00Z 2.116 santa_monica 2015-08-18T00:12:00Z 2.028 santa_monica ``` The query returns the smallest three values in the `water_level` field key for each 24-minute [interval](/influxdb/v1/query_language/explore-data/#basic-group-by-time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:54:00Z`. It also returns results in [descending timestamp](/influxdb/v1/query_language/explore-data/#order-by-time-desc) order. Notice that the [GROUP BY time() clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) does not override the points’ original timestamps. See [Issue 1](#bottom-with-a-group-by-time-clause) in the section below for a more detailed explanation of that behavior. #### Common Issues with `BOTTOM()` ##### `BOTTOM()` with a `GROUP BY time()` clause Queries with `BOTTOM()` and a `GROUP BY time()` clause return the specified number of points per `GROUP BY time()` interval. For[most `GROUP BY time()` queries](/influxdb/v1/query_language/explore-data/#group-by-time-intervals), the returned timestamps mark the start of the `GROUP BY time()` interval.`GROUP BY time()` queries with the `BOTTOM()` function behave differently; they maintain the timestamp of the original data point. ###### Example The query below returns two points per 18-minute`GROUP BY time()` interval. Notice that the returned timestamps are the points’ original timestamps; they are not forced to match the start of the `GROUP BY time()` intervals. ```sql > SELECT BOTTOM("water_level",2) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(18m) name: h2o_feet time bottom ---- ------ __ 2015-08-18T00:00:00Z 2.064 | 2015-08-18T00:12:00Z 2.028 | <------- Smallest points for the first time interval -- __ 2015-08-18T00:24:00Z 2.041 | 2015-08-18T00:30:00Z 2.051 | <------- Smallest points for the second time interval -- ``` ##### BOTTOM() and a tag key with fewer than N tag values Queries with the syntax `SELECT BOTTOM(,,)` can return fewer points than expected. If the tag key has `X` tag values, the query specifies `N` values, and `X` is smaller than `N`, then the query returns `X` points. ###### Example The query below asks for the smallest field values of `water_level` for three tag values of the `location` tag key. Because the `location` tag key has two tag values (`santa_monica` and `coyote_creek`), the query returns two points instead of three. ```sql > SELECT BOTTOM("water_level","location",3) FROM "h2o_feet" name: h2o_feet time bottom location ---- ------ -------- 2015-08-29T10:36:00Z -0.243 santa_monica 2015-08-29T14:30:00Z -0.61 coyote_creek ``` ##### BOTTOM(), tags, and the INTO clause When combined with an [`INTO` clause](/influxdb/v1/query_language/explore-data/#the-into-clause) and no [`GROUP BY tag` clause](/influxdb/v1/query_language/explore-data/#group-by-tags), most InfluxQL functions [convert](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-are-my-into-queries-missing-data) any tags in the initial data to fields in the newly written data. This behavior also applies to the `BOTTOM()` function unless `BOTTOM()` includes a tag key as an argument: `BOTTOM(field_key,tag_key(s),N)`. In those cases, the system preserves the specified tag as a tag in the newly written data. ###### Example The first query in the codeblock below returns the smallest field values in the `water_level` field key for two tag values associated with the `location` tag key. It also writes those results to the `bottom_water_levels` measurement. The second query [shows](/influxdb/v1/query_language/explore-schema/#show-tag-keys) that InfluxDB preserved the `location` tag as a tag in the `bottom_water_levels` measurement. ``` > SELECT BOTTOM("water_level","location",2) INTO "bottom_water_levels" FROM "h2o_feet" name: result time written ---- ------- 1970-01-01T00:00:00Z 2 > SHOW TAG KEYS FROM "bottom_water_levels" name: bottom_water_levels ## tagKey location ``` ### FIRST() Returns the [field value ](/influxdb/v1/concepts/glossary/#field-value)with the oldest timestamp. #### Syntax ``` SELECT FIRST()[,|] [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `FIRST(field_key)` Returns the oldest field value (determined by timestamp) associated with the field key. `FIRST(/regular_expression/)` Returns the oldest field value (determined by timestamp) associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `FIRST(*)` Returns the oldest field value (determined by timestamp) associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `FIRST(field_key),tag_key(s),field_key(s)` Returns the oldest field value (determined by timestamp) associated with the field key in the parentheses and the relevant [tag](/influxdb/v1/concepts/glossary/#tag) and/or [field](/influxdb/v1/concepts/glossary/#field). `FIRST()` supports all field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Select the first field value associated with a field key ```sql > SELECT FIRST("level description") FROM "h2o_feet" name: h2o_feet time first ---- ----- 2015-08-18T00:00:00Z between 6 and 9 feet ``` The query returns the oldest field value (determined by timestamp) associated with the `level description` field key and in the `h2o_feet` measurement. ##### Select the first field value associated with each field key in a measurement ```sql > SELECT FIRST(*) FROM "h2o_feet" name: h2o_feet time first_level description first_water_level ---- ----------------------- ----------------- 1970-01-01T00:00:00Z between 6 and 9 feet 8.12 ``` The query returns the oldest field value (determined by timestamp) for each field key in the `h2o_feet` measurement. The `h2o_feet` measurement has two field keys: `level description` and `water_level`. ##### Select the first field value associated with each field key that matches a regular expression ```sql > SELECT FIRST(/level/) FROM "h2o_feet" name: h2o_feet time first_level description first_water_level ---- ----------------------- ----------------- 1970-01-01T00:00:00Z between 6 and 9 feet 8.12 ``` The query returns the oldest field value for each field key that includes the word `level` in the `h2o_feet` measurement. ##### Select the first value associated with a field key and the relevant tags and fields ```sql > SELECT FIRST("level description"),"location","water_level" FROM "h2o_feet" name: h2o_feet time first location water_level ---- ----- -------- ----------- 2015-08-18T00:00:00Z between 6 and 9 feet coyote_creek 8.12 ``` The query returns the oldest field value (determined by timestamp) in the `level description` field key and the relevant values of the `location` tag key and the `water_level` field key. ##### Select the first field value associated with a field key and include several clauses ```sql > SELECT FIRST("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(9.01) LIMIT 4 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time first ---- ----- 2015-08-17T23:48:00Z 9.01 2015-08-18T00:00:00Z 8.12 2015-08-18T00:12:00Z 7.887 2015-08-18T00:24:00Z 7.635 ``` The query returns the oldest field value (determined by timestamp) in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `9.01`, and it [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to four and one. Notice that the [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) overrides the points’ original timestamps. The timestamps in the results indicate the the start of each 12-minute time interval; the first point in the results covers the time interval between `2015-08-17T23:48:00Z` and just before `2015-08-18T00:00:00Z` and the last point in the results covers the time interval between `2015-08-18T00:24:00Z` and just before `2015-08-18T00:36:00Z`. ### LAST() Returns the [field value](/influxdb/v1/concepts/glossary/#field-value) with the most recent timestamp. #### Syntax ```sql SELECT LAST()[,|] [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `LAST(field_key)` Returns the newest field value (determined by timestamp) associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `LAST(/regular_expression/)` Returns the newest field value (determined by timestamp) associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `LAST(*)` Returns the newest field value (determined by timestamp) associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `LAST(field_key),tag_key(s),field_key(s)` Returns the newest field value (determined by timestamp) associated with the field key in the parentheses and the relevant [tag](/influxdb/v1/concepts/glossary/#tag) and/or [field](/influxdb/v1/concepts/glossary/#field). `LAST()` supports all field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Select the last field values associated with a field key ```sql > SELECT LAST("level description") FROM "h2o_feet" name: h2o_feet time last ---- ---- 2015-09-18T21:42:00Z between 3 and 6 feet ``` The query returns the newest field value (determined by timestamp) associated with the `level description` field key and in the `h2o_feet` measurement. ##### Select the last field values associated with each field key in a measurement ```sql > SELECT LAST(*) FROM "h2o_feet" name: h2o_feet time last_level description last_water_level ---- ----------------------- ----------------- 1970-01-01T00:00:00Z between 3 and 6 feet 4.938 ``` The query returns the newest field value (determined by timestamp) for each field key in the `h2o_feet` measurement. The `h2o_feet` measurement has two field keys: `level description` and `water_level`. ##### Select the last field value associated with each field key that matches a regular expression ```sql > SELECT LAST(/level/) FROM "h2o_feet" name: h2o_feet time last_level description last_water_level ---- ----------------------- ----------------- 1970-01-01T00:00:00Z between 3 and 6 feet 4.938 ``` The query returns the newest field value for each field key that includes the word `level` in the `h2o_feet` measurement. ##### Select the last field value associated with a field key and the relevant tags and fields ```sql > SELECT LAST("level description"),"location","water_level" FROM "h2o_feet" name: h2o_feet time last location water_level ---- ---- -------- ----------- 2015-09-18T21:42:00Z between 3 and 6 feet santa_monica 4.938 ``` The query returns the newest field value (determined by timestamp) in the `level description` field key and the relevant values of the `location` tag key and the `water_level` field key. ##### Select the last field value associated with a field key and include several clauses ```sql > SELECT LAST("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(9.01) LIMIT 4 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time last ---- ---- 2015-08-17T23:48:00Z 9.01 2015-08-18T00:00:00Z 8.005 2015-08-18T00:12:00Z 7.762 2015-08-18T00:24:00Z 7.5 ``` The query returns the newest field value (determined by timestamp) in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results into 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `9.01`, and it [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to four and one. Notice that the [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) overrides the points’ original timestamps. The timestamps in the results indicate the the start of each 12-minute time interval; the first point in the results covers the time interval between `2015-08-17T23:48:00Z` and just before `2015-08-18T00:00:00Z` and the last point in the results covers the time interval between `2015-08-18T00:24:00Z` and just before `2015-08-18T00:36:00Z`. ### MAX() Returns the greatest [field value](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT MAX()[,|] [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `MAX(field_key)` Returns the greatest field value associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `MAX(/regular_expression/)` Returns the greatest field value associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `MAX(*)` Returns the greatest field value associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `MAX(field_key),tag_key(s),field_key(s)` Returns the greatest field value associated with the field key in the parentheses and the relevant [tag](/influxdb/v1/concepts/glossary/#tag) and/or [field](/influxdb/v1/concepts/glossary/#field). `MAX()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Select the maximum field value associated with a field key ```sql > SELECT MAX("water_level") FROM "h2o_feet" name: h2o_feet time max ---- --- 2015-08-29T07:24:00Z 9.964 ``` The query returns the greatest field value in the `water_level` field key and in the `h2o_feet` measurement. ##### Select the maximum field value associated with each field key in a measurement ```sql > SELECT MAX(*) FROM "h2o_feet" name: h2o_feet time max_water_level ---- --------------- 2015-08-29T07:24:00Z 9.964 ``` The query returns the greatest field value for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ##### Select the maximum field value associated with each field key that matches a regular expression ```sql > SELECT MAX(/level/) FROM "h2o_feet" name: h2o_feet time max_water_level ---- --------------- 2015-08-29T07:24:00Z 9.964 ``` The query returns the greatest field value for each field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. ##### Select the maximum field value associated with a field key and the relevant tags and fields ```sql > SELECT MAX("water_level"),"location","level description" FROM "h2o_feet" name: h2o_feet time max location level description ---- --- -------- ----------------- 2015-08-29T07:24:00Z 9.964 coyote_creek at or greater than 9 feet ``` The query returns the greatest field value in the `water_level` field key and the relevant values of the `location` tag key and the `level description` field key. ##### Select the maximum field value associated with a field key and include several clauses ```sql > SELECT MAX("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(9.01) LIMIT 4 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time max ---- --- 2015-08-17T23:48:00Z 9.01 2015-08-18T00:00:00Z 8.12 2015-08-18T00:12:00Z 7.887 2015-08-18T00:24:00Z 7.635 ``` The query returns the greatest field value in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results in to 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `9.01`, and it [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to four and one. Notice that the [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) overrides the points’ original timestamps. The timestamps in the results indicate the the start of each 12-minute time interval; the first point in the results covers the time interval between `2015-08-17T23:48:00Z` and just before `2015-08-18T00:00:00Z` and the last point in the results covers the time interval between `2015-08-18T00:24:00Z` and just before `2015-08-18T00:36:00Z`. ### MIN() Returns the lowest [field value](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT MIN()[,|] [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `MIN(field_key)` Returns the lowest field value associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `MIN(/regular_expression/)` Returns the lowest field value associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `MIN(*)` Returns the lowest field value associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `MIN(field_key),tag_key(s),field_key(s)` Returns the lowest field value associated with the field key in the parentheses and the relevant [tag](/influxdb/v1/concepts/glossary/#tag) and/or [field](/influxdb/v1/concepts/glossary/#field). `MIN()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Select the minimum field value associated with a field key ```sql > SELECT MIN("water_level") FROM "h2o_feet" name: h2o_feet time min ---- --- 2015-08-29T14:30:00Z -0.61 ``` The query returns the lowest field value in the `water_level` field key and in the `h2o_feet` measurement. ##### Select the minimum field value associated with each field key in a measurement ```sql > SELECT MIN(*) FROM "h2o_feet" name: h2o_feet time min_water_level ---- --------------- 2015-08-29T14:30:00Z -0.61 ``` The query returns the lowest field value for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ##### Select the minimum field value associated with each field key that matches a regular expression ```sql > SELECT MIN(/level/) FROM "h2o_feet" name: h2o_feet time min_water_level ---- --------------- 2015-08-29T14:30:00Z -0.61 ``` The query returns the lowest field value for each field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. ##### Select the minimum field value associated with a field key and the relevant tags and fields ```sql > SELECT MIN("water_level"),"location","level description" FROM "h2o_feet" name: h2o_feet time min location level description ---- --- -------- ----------------- 2015-08-29T14:30:00Z -0.61 coyote_creek below 3 feet ``` The query returns the lowest field value in the `water_level` field key and the relevant values of the `location` tag key and the `level description` field key. ##### Select the minimum field value associated with a field key and include several clauses ```sql > SELECT MIN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m),* fill(9.01) LIMIT 4 SLIMIT 1 name: h2o_feet tags: location=coyote_creek time min ---- --- 2015-08-17T23:48:00Z 9.01 2015-08-18T00:00:00Z 8.005 2015-08-18T00:12:00Z 7.762 2015-08-18T00:24:00Z 7.5 ``` The query returns the lowest field value in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#the-group-by-clause) results in to 12-minute time intervals and per tag. The query [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `9.01`, and it [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points and series returned to four and one. Notice that the [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) overrides the points’ original timestamps. The timestamps in the results indicate the the start of each 12-minute time interval; the first point in the results covers the time interval between `2015-08-17T23:48:00Z` and just before `2015-08-18T00:00:00Z` and the last point in the results covers the time interval between `2015-08-18T00:24:00Z` and just before `2015-08-18T00:36:00Z`. ### PERCENTILE() Returns the `N`th percentile [field value](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT PERCENTILE(, )[,|] [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `PERCENTILE(field_key,N)` Returns the Nth percentile field value associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `PERCENTILE(/regular_expression/,N)` Returns the Nth percentile field value associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `PERCENTILE(*,N)` Returns the Nth percentile field value associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `PERCENTILE(field_key,N),tag_key(s),field_key(s)` Returns the Nth percentile field value associated with the field key in the parentheses and the relevant [tag](/influxdb/v1/concepts/glossary/#tag) and/or [field](/influxdb/v1/concepts/glossary/#field). `N` must be an integer or floating point number between `0` and `100`, inclusive.`PERCENTILE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Select the fifth percentile field value associated with a field key ```sql > SELECT PERCENTILE("water_level",5) FROM "h2o_feet" name: h2o_feet time percentile ---- ---------- 2015-08-31T03:42:00Z 1.122 ``` The query returns the field value that is larger than five percent of the field values in the `water_level` field key and in the `h2o_feet` measurement. ##### Select the fifth percentile field value associated with each field key in a measurement ```sql > SELECT PERCENTILE(*,5) FROM "h2o_feet" name: h2o_feet time percentile_water_level ---- ---------------------- 2015-08-31T03:42:00Z 1.122 ``` The query returns the field value that is larger than five percent of the field values in each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ##### Select fifth percentile field value associated with each field key that matches a regular expression ```sql > SELECT PERCENTILE(/level/,5) FROM "h2o_feet" name: h2o_feet time percentile_water_level ---- ---------------------- 2015-08-31T03:42:00Z 1.122 ``` The query returns the field value that is larger than five percent of the field values in each field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. ##### Select the fifth percentile field values associated with a field key and the relevant tags and fields ```sql > SELECT PERCENTILE("water_level",5),"location","level description" FROM "h2o_feet" name: h2o_feet time percentile location level description ---- ---------- -------- ----------------- 2015-08-31T03:42:00Z 1.122 coyote_creek below 3 feet ``` The query returns the field value that is larger than five percent of the field values in the `water_level` field key and the relevant values of the `location` tag key and the `level description` field key. ##### Select the twentieth percentile field value associated with a field key and include several clauses ```sql > SELECT PERCENTILE("water_level",20) FROM "h2o_feet" WHERE time >= '2015-08-17T23:48:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(24m) fill(15) LIMIT 2 name: h2o_feet time percentile ---- ---------- 2015-08-17T23:36:00Z 15 2015-08-18T00:00:00Z 2.064 ``` The query returns the field value that is larger than 20 percent of the values in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-17T23:48:00Z` and `2015-08-18T00:54:00Z` and [groups](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) results into 24-minute intervals. It [fills](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) empty time intervals with `15` and it [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to two. Notice that the [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) overrides the points’ original timestamps. The timestamps in the results indicate the the start of each 24-minute time interval; the first point in the results covers the time interval between `2015-08-17T23:36:00Z` and just before `2015-08-18T00:00:00Z` and the last point in the results covers the time interval between `2015-08-18T00:00:00Z` and just before `2015-08-18T00:24:00Z`. #### Common Issues with PERCENTILE() ##### PERCENTILE() compared to other InfluxQL functions * `PERCENTILE(,100)` is equivalent to [`MAX()`](#max). * `PERCENTILE(, 50)` is nearly equivalent to [`MEDIAN()`](#median), except the `MEDIAN()` function returns the average of the two middle values if the field key contains an even number of field values. * `PERCENTILE(,0)` is not equivalent to [`MIN()`](#min). This is a known [issue](https://github.com/influxdata/influxdb/issues/4418). ### SAMPLE() Returns a random sample of `N` [field values](/influxdb/v1/concepts/glossary/#field-value).`SAMPLE()` uses [reservoir sampling](https://en.wikipedia.org/wiki/Reservoir_sampling) to generate the random points. #### Syntax ``` SELECT SAMPLE(, )[,|] [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `SAMPLE(field_key,N)` Returns N randomly selected field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `SAMPLE(/regular_expression/,N)` Returns N randomly selected field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `SAMPLE(*,N)` Returns N randomly selected field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `SAMPLE(field_key,N),tag_key(s),field_key(s)` Returns N randomly selected field values associated with the field key in the parentheses and the relevant [tag](/influxdb/v1/concepts/glossary/#tag) and/or [field](/influxdb/v1/concepts/glossary/#field). `N` must be an integer.`SAMPLE()` supports all field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Select a sample of the field values associated with a field key ```sql > SELECT SAMPLE("water_level",2) FROM "h2o_feet" name: h2o_feet time sample ---- ------ 2015-09-09T21:48:00Z 5.659 2015-09-18T10:00:00Z 6.939 ``` The query returns two randomly selected points from the `water_level` field key and in the `h2o_feet` measurement. #### Select a sample of the field values associated with each field key in a measurement ```sql > SELECT SAMPLE(*,2) FROM "h2o_feet" name: h2o_feet time sample_level description sample_water_level ---- ------------------------ ------------------ 2015-08-25T17:06:00Z 3.284 2015-09-03T04:30:00Z below 3 feet 2015-09-03T20:06:00Z between 3 and 6 feet 2015-09-08T21:54:00Z 3.412 ``` The query returns two randomly selected points for each field key in the `h2o_feet` measurement. The `h2o_feet` measurement has two field keys: `level description` and `water_level`. ##### Select a sample of the field values associated with each field key that matches a regular expression ```sql > SELECT SAMPLE(/level/,2) FROM "h2o_feet" name: h2o_feet time sample_level description sample_water_level ---- ------------------------ ------------------ 2015-08-30T05:54:00Z between 6 and 9 feet 2015-09-07T01:18:00Z 7.854 2015-09-09T20:30:00Z 7.32 2015-09-13T19:18:00Z between 3 and 6 feet ``` The query returns two randomly selected points for each field key that includes the word `level` in the `h2o_feet` measurement. ##### Select a sample of the field values associated with a field key and the relevant tags and fields ```sql > SELECT SAMPLE("water_level",2),"location","level description" FROM "h2o_feet" name: h2o_feet time sample location level description ---- ------ -------- ----------------- 2015-08-29T10:54:00Z 5.689 coyote_creek between 3 and 6 feet 2015-09-08T15:48:00Z 6.391 coyote_creek between 6 and 9 feet ``` The query returns two randomly selected points from the `water_level` field key and the relevant values of the `location` tag and the `level description` field. ##### Select a sample of the field values associated with a field key and include several clauses ```sql > SELECT SAMPLE("water_level",1) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(18m) name: h2o_feet time sample ---- ------ 2015-08-18T00:12:00Z 2.028 2015-08-18T00:30:00Z 2.051 ``` The query returns one randomly selected point from the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and [groups](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) results into 18-minute intervals. Notice that the [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) does not override the points’ original timestamps. See [Issue 1](#sample-with-a-group-by-time-clause) in the section below for a more detailed explanation of that behavior. #### Common Issues with `SAMPLE()` ##### `SAMPLE()` with a `GROUP BY time()` clause Queries with `SAMPLE()` and a `GROUP BY time()` clause return the specified number of points (`N`) per `GROUP BY time()` interval. For[most `GROUP BY time()` queries](/influxdb/v1/query_language/explore-data/#group-by-time-intervals), the returned timestamps mark the start of the `GROUP BY time()` interval.`GROUP BY time()` queries with the `SAMPLE()` function behave differently; they maintain the timestamp of the original data point. ###### Example The query below returns two randomly selected points per 18-minute`GROUP BY time()` interval. Notice that the returned timestamps are the points’ original timestamps; they are not forced to match the start of the `GROUP BY time()` intervals. ```sql > SELECT SAMPLE("water_level",2) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(18m) name: h2o_feet time sample ---- ------ __ 2015-08-18T00:06:00Z 2.116 | 2015-08-18T00:12:00Z 2.028 | <------- Randomly-selected points for the first time interval -- __ 2015-08-18T00:18:00Z 2.126 | 2015-08-18T00:30:00Z 2.051 | <------- Randomly-selected points for the second time interval -- ``` ### TOP() Returns the greatest `N` [field values](/influxdb/v1/concepts/glossary/#field-value). #### Syntax ``` SELECT TOP( [,], )[,|] [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `TOP(field_key,N)` Returns the greatest N field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `TOP(field_key,tag_key(s),N)` Returns the greatest field value for N tag values of the [tag key](/influxdb/v1/concepts/glossary/#tag-key). `TOP(field_key,N),tag_key(s),field_key(s)` Returns the greatest N field values associated with the field key in the parentheses and the relevant [tag](/influxdb/v1/concepts/glossary/#tag) and/or [field](/influxdb/v1/concepts/glossary/#field). `TOP()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). > > > **Notes:** > > * `TOP()` returns the field value with the earliest timestamp if there’s a tie between two or more values for the greatest value. * `TOP()` differs from other InfluxQL functions when combined with an [`INTO` clause](/influxdb/v1/query_language/explore-data/#the-into-clause). See the [Common Issues](#common-issues-with-top) section for more information. #### Examples ##### Select the top three field values associated with a field key ```sql > SELECT TOP("water_level",3) FROM "h2o_feet" name: h2o_feet time top ---- --- 2015-08-29T07:18:00Z 9.957 2015-08-29T07:24:00Z 9.964 2015-08-29T07:30:00Z 9.954 ``` The query returns the greatest three field values in the `water_level` field key and in the `h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement). ##### Select the top field value associated with a field key for two tags ```sql > SELECT TOP("water_level","location",2) FROM "h2o_feet" name: h2o_feet time top location ---- --- -------- 2015-08-29T03:54:00Z 7.205 santa_monica 2015-08-29T07:24:00Z 9.964 coyote_creek ``` The query returns the greatest field values in the `water_level` field key for two tag values associated with the `location` tag key. ##### Select the top four field values associated with a field key and the relevant tags and fields ```sql > SELECT TOP("water_level",4),"location","level description" FROM "h2o_feet" name: h2o_feet time top location level description ---- --- -------- ----------------- 2015-08-29T07:18:00Z 9.957 coyote_creek at or greater than 9 feet 2015-08-29T07:24:00Z 9.964 coyote_creek at or greater than 9 feet 2015-08-29T07:30:00Z 9.954 coyote_creek at or greater than 9 feet 2015-08-29T07:36:00Z 9.941 coyote_creek at or greater than 9 feet ``` The query returns the greatest four field values in the `water_level` field key and the relevant values of the `location` tag key and the `level description` field key. ##### Select the top three field values associated with a field key and include several clauses ```sql > SELECT TOP("water_level",3),"location" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(24m) ORDER BY time DESC name: h2o_feet time top location ---- --- -------- 2015-08-18T00:48:00Z 7.11 coyote_creek 2015-08-18T00:54:00Z 6.982 coyote_creek 2015-08-18T00:54:00Z 2.054 santa_monica 2015-08-18T00:24:00Z 7.635 coyote_creek 2015-08-18T00:30:00Z 7.5 coyote_creek 2015-08-18T00:36:00Z 7.372 coyote_creek 2015-08-18T00:00:00Z 8.12 coyote_creek 2015-08-18T00:06:00Z 8.005 coyote_creek 2015-08-18T00:12:00Z 7.887 coyote_creek ``` The query returns the greatest three values in the `water_level` field key for each 24-minute [interval](/influxdb/v1/query_language/explore-data/#basic-group-by-time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:54:00Z`. It also returns results in [descending timestamp](/influxdb/v1/query_language/explore-data/#order-by-time-desc) order. Notice that the [GROUP BY time() clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) does not override the points’ original timestamps. See [Issue 1](#top-with-a-group-by-time-clause) in the section below for a more detailed explanation of that behavior. #### Common Issues with `TOP()` ##### `TOP()` with a `GROUP BY time()` clause Queries with `TOP()` and a `GROUP BY time()` clause return the specified number of points per `GROUP BY time()` interval. For[most `GROUP BY time()` queries](/influxdb/v1/query_language/explore-data/#group-by-time-intervals), the returned timestamps mark the start of the `GROUP BY time()` interval.`GROUP BY time()` queries with the `TOP()` function behave differently; they maintain the timestamp of the original data point. ###### Example The query below returns two points per 18-minute`GROUP BY time()` interval. Notice that the returned timestamps are the points’ original timestamps; they are not forced to match the start of the `GROUP BY time()` intervals. ```sql > SELECT TOP("water_level",2) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(18m) name: h2o_feet time top ---- ------ __ 2015-08-18T00:00:00Z 2.064 | 2015-08-18T00:06:00Z 2.116 | <------- Greatest points for the first time interval -- __ 2015-08-18T00:18:00Z 2.126 | 2015-08-18T00:30:00Z 2.051 | <------- Greatest points for the second time interval -- ``` ##### TOP() and a tag key with fewer than N tag values Queries with the syntax `SELECT TOP(,,)` can return fewer points than expected. If the tag key has `X` tag values, the query specifies `N` values, and `X` is smaller than `N`, then the query returns `X` points. ###### Example The query below asks for the greatest field values of `water_level` for three tag values of the `location` tag key. Because the `location` tag key has two tag values (`santa_monica` and `coyote_creek`), the query returns two points instead of three. ```sql > SELECT TOP("water_level","location",3) FROM "h2o_feet" name: h2o_feet time top location ---- --- -------- 2015-08-29T03:54:00Z 7.205 santa_monica 2015-08-29T07:24:00Z 9.964 coyote_creek ``` ##### TOP(), tags, and the INTO clause When combined with an [`INTO` clause](/influxdb/v1/query_language/explore-data/#the-into-clause) and no [`GROUP BY tag` clause](/influxdb/v1/query_language/explore-data/#group-by-tags), most InfluxQL functions [convert](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-are-my-into-queries-missing-data) any tags in the initial data to fields in the newly written data. This behavior also applies to the `TOP()` function unless `TOP()` includes a tag key as an argument: `TOP(field_key,tag_key(s),N)`. In those cases, the system preserves the specified tag as a tag in the newly written data. ###### Example The first query in the codeblock below returns the greatest field values in the `water_level` field key for two tag values associated with the `location` tag key. It also writes those results to the `top_water_levels` measurement. The second query [shows](/influxdb/v1/query_language/explore-schema/#show-tag-keys) that InfluxDB preserved the `location` tag as a tag in the `top_water_levels` measurement. ``` > SELECT TOP("water_level","location",2) INTO "top_water_levels" FROM "h2o_feet" name: result time written ---- ------- 1970-01-01T00:00:00Z 2 > SHOW TAG KEYS FROM "top_water_levels" name: top_water_levels ## tagKey location ``` ## Transformations ### ABS() Returns the absolute value of the field value. #### Basic syntax ``` SELECT ABS( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `ABS(field_key)` Returns the absolute values of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `ABS(*)` Returns the absolute values of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `ABS()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `ABS()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of this [sample data](https://gist.github.com/sanderson/8f8aec94a60b2c31a61f44a37737bfea): ```sql > SELECT * FROM "data" WHERE time >= '2018-06-24T12:00:00Z' AND time <= '2018-06-24T12:05:00Z' name: data time a b ---- - - 1529841600000000000 1.33909108671076 -0.163643058925645 1529841660000000000 -0.774984088561186 0.137034364053949 1529841720000000000 -0.921037167720451 -0.482943221384294 1529841780000000000 -1.73880754843378 -0.0729732928756677 1529841840000000000 -0.905980032168252 1.77857552719844 1529841900000000000 -0.891164752631417 0.741147445214238 ``` ###### Calculate the absolute values of field values associated with a field key ```sql > SELECT ABS("a") FROM "data" WHERE time >= '2018-06-24T12:00:00Z' AND time <= '2018-06-24T12:05:00Z' name: data time abs ---- --- 1529841600000000000 1.33909108671076 1529841660000000000 0.774984088561186 1529841720000000000 0.921037167720451 1529841780000000000 1.73880754843378 1529841840000000000 0.905980032168252 1529841900000000000 0.891164752631417 ``` The query returns the absolute values of field values in the `a` field key in the `data` measurement. ###### Calculate the absolute Values of field values associated with each field key in a measurement ```sql > SELECT ABS(*) FROM "data" WHERE time >= '2018-06-24T12:00:00Z' AND time <= '2018-06-24T12:05:00Z' name: data time abs_a abs_b ---- ----- ----- 1529841600000000000 1.33909108671076 0.163643058925645 1529841660000000000 0.774984088561186 0.137034364053949 1529841720000000000 0.921037167720451 0.482943221384294 1529841780000000000 1.73880754843378 0.0729732928756677 1529841840000000000 0.905980032168252 1.77857552719844 1529841900000000000 0.891164752631417 0.741147445214238 ``` The query returns the absolute values of field values for each field key that stores numerical values in the `data` measurement. The `data` measurement has two numerical fields: `a` and `b`. ###### Calculate the absolute values of field values associated with a field key and include several clauses ```sql > SELECT ABS("a") FROM "data" WHERE time >= '2018-06-24T12:00:00Z' AND time <= '2018-06-24T12:05:00Z' ORDER BY time DESC LIMIT 4 OFFSET 2 name: data time abs ---- --- 1529841780000000000 1.73880754843378 1529841720000000000 0.921037167720451 1529841660000000000 0.774984088561186 1529841600000000000 1.33909108671076 ``` The query returns the absolute values of field values associated with the `a` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2018-06-24T12:00:00Z` and `2018-06-24T12:05:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT ABS(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `ABS()` function to those results. `ABS()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the absolute values of mean values ```sql > SELECT ABS(MEAN("a")) FROM "data" WHERE time >= '2018-06-24T12:00:00Z' AND time <= '2018-06-24T13:00:00Z' GROUP BY time(12m) name: data time abs ---- --- 1529841600000000000 0.3960977256302787 1529842320000000000 0.0010541018316373302 1529843040000000000 0.04494733240283668 1529843760000000000 0.2553594777104415 1529844480000000000 0.20382988543108413 1529845200000000000 0.790836070736962 ``` The query returns the absolute values of [average](#mean) `a`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `a`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `ABS()`: ```sql > SELECT MEAN("a") FROM "data" WHERE time >= '2018-06-24T12:00:00Z' AND time <= '2018-06-24T13:00:00Z' GROUP BY time(12m) name: data time mean ---- ---- 1529841600000000000 -0.3960977256302787 1529842320000000000 0.0010541018316373302 1529843040000000000 0.04494733240283668 1529843760000000000 0.2553594777104415 1529844480000000000 0.20382988543108413 1529845200000000000 -0.790836070736962 ``` InfluxDB then calculates absolute values of those averages. ### ACOS() Returns the arccosine (in radians) of the field value. Field values must be between -1 and 1. #### Basic syntax ``` SELECT ACOS( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `ACOS(field_key)` Returns the arccosine of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `ACOS(*)` Returns the arccosine of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `ACOS()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types) with values between -1 and 1. The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `ACOS()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following data sample of simulated park occupancy relative to total capacity. The important thing to note is that all field values fall within the calculable range (-1 to 1) of the `ACOS()` function: ```sql > SELECT "of_capacity" FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' name: park_occupancy time capacity ---- -------- 2017-05-01T00:00:00Z 0.83 2017-05-02T00:00:00Z 0.3 2017-05-03T00:00:00Z 0.84 2017-05-04T00:00:00Z 0.22 2017-05-05T00:00:00Z 0.17 2017-05-06T00:00:00Z 0.77 2017-05-07T00:00:00Z 0.64 2017-05-08T00:00:00Z 0.72 2017-05-09T00:00:00Z 0.16 ``` ###### Calculate the arccosine of field values associated with a field key ```sql > SELECT ACOS("of_capacity") FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' name: park_occupancy time acos ---- ---- 2017-05-01T00:00:00Z 0.591688642426544 2017-05-02T00:00:00Z 1.266103672779499 2017-05-03T00:00:00Z 0.5735131044230969 2017-05-04T00:00:00Z 1.3489818562981022 2017-05-05T00:00:00Z 1.399966657665792 2017-05-06T00:00:00Z 0.6919551751263169 2017-05-07T00:00:00Z 0.8762980611683406 2017-05-08T00:00:00Z 0.7669940078618667 2017-05-09T00:00:00Z 1.410105673842986 ``` The query returns arccosine of field values in the `of_capacity` field key in the `park_occupancy` measurement. ###### Calculate the arccosine of field values associated with each field key in a measurement ```sql > SELECT ACOS(*) FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' name: park_occupancy time acos_of_capacity ---- ------------- 2017-05-01T00:00:00Z 0.591688642426544 2017-05-02T00:00:00Z 1.266103672779499 2017-05-03T00:00:00Z 0.5735131044230969 2017-05-04T00:00:00Z 1.3489818562981022 2017-05-05T00:00:00Z 1.399966657665792 2017-05-06T00:00:00Z 0.6919551751263169 2017-05-07T00:00:00Z 0.8762980611683406 2017-05-08T00:00:00Z 0.7669940078618667 2017-05-09T00:00:00Z 1.410105673842986 ``` The query returns arccosine of field values for each field key that stores numerical values in the `park_occupancy` measurement. The `park_occupancy` measurement has one numerical field: `of_capacity`. ###### Calculate the arccosine of field values associated with a field key and include several clauses ```sql > SELECT ACOS("of_capacity") FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' ORDER BY time DESC LIMIT 4 OFFSET 2 name: park_occupancy time acos ---- ---- 2017-05-07T00:00:00Z 0.8762980611683406 2017-05-06T00:00:00Z 0.6919551751263169 2017-05-05T00:00:00Z 1.399966657665792 2017-05-04T00:00:00Z 1.3489818562981022 ``` The query returns arccosine of field values associated with the `of_capacity` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2017-05-01T00:00:00Z` and `2017-05-09T00:00:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT ACOS(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `ACOS()` function to those results. `ACOS()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the arccosine of mean values ```sql > SELECT ACOS(MEAN("of_capacity")) FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' GROUP BY time(3d) name: park_occupancy time acos ---- ---- 2017-04-30T00:00:00Z 0.9703630732143733 2017-05-03T00:00:00Z 1.1483422646081407 2017-05-06T00:00:00Z 0.7812981174487247 2017-05-09T00:00:00Z 1.410105673842986 ``` The query returns arccosine of [average](#mean) `of_capacity`s that are calculated at 3-day intervals. To get those results, InfluxDB first calculates the average `of_capacity`s at 3-day intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `ACOS()`: ```sql > SELECT MEAN("of_capacity") FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' GROUP BY time(3d) name: park_occupancy time mean ---- ---- 2017-04-30T00:00:00Z 0.565 2017-05-03T00:00:00Z 0.41 2017-05-06T00:00:00Z 0.71 2017-05-09T00:00:00Z 0.16 ``` InfluxDB then calculates arccosine of those averages. ### ASIN() Returns the arcsine (in radians) of the field value. Field values must be between -1 and 1. #### Basic syntax ``` SELECT ASIN( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `ASIN(field_key)` Returns the arcsine of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `ASIN(*)` Returns the arcsine of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `ASIN()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types) with values between -1 and 1. The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `ASIN()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following data sample of simulated park occupancy relative to total capacity. The important thing to note is that all field values fall within the calculable range (-1 to 1) of the `ASIN()` function: ```sql > SELECT "of_capacity" FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' name: park_occupancy time capacity ---- -------- 2017-05-01T00:00:00Z 0.83 2017-05-02T00:00:00Z 0.3 2017-05-03T00:00:00Z 0.84 2017-05-04T00:00:00Z 0.22 2017-05-05T00:00:00Z 0.17 2017-05-06T00:00:00Z 0.77 2017-05-07T00:00:00Z 0.64 2017-05-08T00:00:00Z 0.72 2017-05-09T00:00:00Z 0.16 ``` ###### Calculate the arcsine of field values associated with a field key ```sql > SELECT ASIN("of_capacity") FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' name: park_occupancy time asin ---- ---- 2017-05-01T00:00:00Z 0.9791076843683526 2017-05-02T00:00:00Z 0.3046926540153975 2017-05-03T00:00:00Z 0.9972832223717997 2017-05-04T00:00:00Z 0.22181447049679442 2017-05-05T00:00:00Z 0.1708296691291045 2017-05-06T00:00:00Z 0.8788411516685797 2017-05-07T00:00:00Z 0.6944982656265559 2017-05-08T00:00:00Z 0.8038023189330299 2017-05-09T00:00:00Z 0.1606906529519106 ``` The query returns arcsine of field values in the `of_capacity` field key in the `park_capacity` measurement. ###### Calculate the arcsine of field values associated with each field key in a measurement ```sql > SELECT ASIN(*) FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' name: park_occupancy time asin_of_capacity ---- ------------- 2017-05-01T00:00:00Z 0.9791076843683526 2017-05-02T00:00:00Z 0.3046926540153975 2017-05-03T00:00:00Z 0.9972832223717997 2017-05-04T00:00:00Z 0.22181447049679442 2017-05-05T00:00:00Z 0.1708296691291045 2017-05-06T00:00:00Z 0.8788411516685797 2017-05-07T00:00:00Z 0.6944982656265559 2017-05-08T00:00:00Z 0.8038023189330299 2017-05-09T00:00:00Z 0.1606906529519106 ``` The query returns arcsine of field values for each field key that stores numerical values in the `park_capacity` measurement. The `h2o_feet` measurement has one numerical field: `of_capacity`. ###### Calculate the arcsine of field values associated with a field key and include several clauses ```sql > SELECT ASIN("of_capacity") FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' ORDER BY time DESC LIMIT 4 OFFSET 2 name: park_occupancy time asin ---- ---- 2017-05-07T00:00:00Z 0.6944982656265559 2017-05-06T00:00:00Z 0.8788411516685797 2017-05-05T00:00:00Z 0.1708296691291045 2017-05-04T00:00:00Z 0.22181447049679442 ``` The query returns arcsine of field values associated with the `of_capacity` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2017-05-01T00:00:00Z` and `2017-05-09T00:00:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT ASIN(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `ASIN()` function to those results. `ASIN()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the arcsine of mean values. ```sql > SELECT ASIN(MEAN("of_capacity")) FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' GROUP BY time(3d) name: park_occupancy time asin ---- ---- 2017-04-30T00:00:00Z 0.6004332535805232 2017-05-03T00:00:00Z 0.42245406218675574 2017-05-06T00:00:00Z 0.7894982093461719 2017-05-09T00:00:00Z 0.1606906529519106 ``` The query returns arcsine of [average](#mean) `of_capacity`s that are calculated at 3-day intervals. To get those results, InfluxDB first calculates the average `of_capacity`s at 3-day intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `ASIN()`: ```sql > SELECT MEAN("of_capacity") FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' GROUP BY time(3d) name: park_occupancy time mean ---- ---- 2017-04-30T00:00:00Z 0.565 2017-05-03T00:00:00Z 0.41 2017-05-06T00:00:00Z 0.71 2017-05-09T00:00:00Z 0.16 ``` InfluxDB then calculates arcsine of those averages. ### ATAN() Returns the arctangent (in radians) of the field value. Field values must be between -1 and 1. #### Basic syntax ```sql SELECT ATAN( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `ATAN(field_key)` Returns the arctangent of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `ATAN(*)` Returns the arctangent of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `ATAN()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types) with values between -1 and 1. The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `ATAN()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following data sample of simulated park occupancy relative to total capacity. The important thing to note is that all field values fall within the calculable range (-1 to 1) of the `ATAN()` function: ```sql > SELECT "of_capacity" FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' name: park_occupancy time capacity ---- -------- 2017-05-01T00:00:00Z 0.83 2017-05-02T00:00:00Z 0.3 2017-05-03T00:00:00Z 0.84 2017-05-04T00:00:00Z 0.22 2017-05-05T00:00:00Z 0.17 2017-05-06T00:00:00Z 0.77 2017-05-07T00:00:00Z 0.64 2017-05-08T00:00:00Z 0.72 2017-05-09T00:00:00Z 0.16 ``` ###### Calculate the arctangent of field values associated with a field key ```sql > SELECT ATAN("of_capacity") FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' name: park_occupancy time atan ---- ---- 2017-05-01T00:00:00Z 0.6927678353971222 2017-05-02T00:00:00Z 0.2914567944778671 2017-05-03T00:00:00Z 0.6986598247214632 2017-05-04T00:00:00Z 0.2165503049760893 2017-05-05T00:00:00Z 0.16839015714752992 2017-05-06T00:00:00Z 0.6561787179913948 2017-05-07T00:00:00Z 0.5693131911006619 2017-05-08T00:00:00Z 0.6240230529767568 2017-05-09T00:00:00Z 0.1586552621864014 ``` The query returns arctangent of field values in the `of_capacity` field key in the `park_occupancy` measurement. ###### Calculate the arctangent of field values associated with each field key in a measurement ```sql > SELECT ATAN(*) FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' name: park_occupancy time atan_of_capacity ---- ------------- 2017-05-01T00:00:00Z 0.6927678353971222 2017-05-02T00:00:00Z 0.2914567944778671 2017-05-03T00:00:00Z 0.6986598247214632 2017-05-04T00:00:00Z 0.2165503049760893 2017-05-05T00:00:00Z 0.16839015714752992 2017-05-06T00:00:00Z 0.6561787179913948 2017-05-07T00:00:00Z 0.5693131911006619 2017-05-08T00:00:00Z 0.6240230529767568 2017-05-09T00:00:00Z 0.1586552621864014 ``` The query returns arctangent of field values for each field key that stores numerical values in the `park_occupancy` measurement. The `park_occupancy` measurement has one numerical field: `of_capacity`. ###### Calculate the arctangent of field values associated with a field key and include several clauses ```sql > SELECT ATAN("of_capacity") FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' ORDER BY time DESC LIMIT 4 OFFSET 2 name: park_occupancy time atan ---- ---- 2017-05-07T00:00:00Z 0.5693131911006619 2017-05-06T00:00:00Z 0.6561787179913948 2017-05-05T00:00:00Z 0.16839015714752992 2017-05-04T00:00:00Z 0.2165503049760893 ``` The query returns arctangent of field values associated with the `of_capacity` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2017-05-01T00:00:00Z` and `2017-05-09T00:00:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT ATAN(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `ATAN()` function to those results. `ATAN()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples of advanced syntax ###### Calculate the arctangent of mean values. ```sql > SELECT ATAN(MEAN("of_capacity")) FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' GROUP BY time(3d) name: park_occupancy time atan ---- ---- 2017-04-30T00:00:00Z 0.5142865412694495 2017-05-03T00:00:00Z 0.3890972310552784 2017-05-06T00:00:00Z 0.6174058917515726 2017-05-09T00:00:00Z 0.1586552621864014 ``` The query returns arctangent of [average](#mean) `of_capacity`s that are calculated at 3-day intervals. To get those results, InfluxDB first calculates the average `of_capacity`s at 3-day intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `ATAN()`: ```sql > SELECT MEAN("of_capacity") FROM "park_occupancy" WHERE time >= '2017-05-01T00:00:00Z' AND time <= '2017-05-09T00:00:00Z' GROUP BY time(3d) name: park_occupancy time mean ---- ---- 2017-04-30T00:00:00Z 0.565 2017-05-03T00:00:00Z 0.41 2017-05-06T00:00:00Z 0.71 2017-05-09T00:00:00Z 0.16 ``` InfluxDB then calculates arctangent of those averages. ### ATAN2() Returns the the arctangent of `y/x` in radians. #### Basic syntax ``` SELECT ATAN2( [ * | | num ], [ | num ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `ATAN2(field_key_y, field_key_x)` Returns the arctangent of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key), `field_key_y`, divided by field values associated with `field_key_x`. `ATAN2(*, field_key_x)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement)divided by field values associated with `field_key_x`. `ATAN2()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `ATAN2()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following sample of simulated flight data: ```sql > SELECT "altitude_ft", "distance_ft" FROM "flight_data" WHERE time >= '2018-05-16T12:01:00Z' AND time <= '2018-05-16T12:10:00Z' name: flight_data time altitude_ft distance_ft ---- ----------- ----------- 2018-05-16T12:01:00Z 1026 50094 2018-05-16T12:02:00Z 2549 53576 2018-05-16T12:03:00Z 4033 55208 2018-05-16T12:04:00Z 5579 58579 2018-05-16T12:05:00Z 7065 61213 2018-05-16T12:06:00Z 8589 64807 2018-05-16T12:07:00Z 10180 67707 2018-05-16T12:08:00Z 11777 69819 2018-05-16T12:09:00Z 13321 72452 2018-05-16T12:10:00Z 14885 75881 ``` ###### Calculate the arctangent of field\_key\_y over field\_key\_x ```sql > SELECT ATAN2("altitude_ft", "distance_ft") FROM "flight_data" WHERE time >= '2018-05-16T12:01:00Z' AND time <= '2018-05-16T12:10:00Z' name: flight_data time atan2 ---- ----- 2018-05-16T12:01:00Z 0.020478631571881498 2018-05-16T12:02:00Z 0.04754142349303296 2018-05-16T12:03:00Z 0.07292147724575364 2018-05-16T12:04:00Z 0.09495251193874832 2018-05-16T12:05:00Z 0.11490822875441563 2018-05-16T12:06:00Z 0.13176409347584003 2018-05-16T12:07:00Z 0.14923587589682233 2018-05-16T12:08:00Z 0.1671059946640312 2018-05-16T12:09:00Z 0.18182893717409565 2018-05-16T12:10:00Z 0.1937028631495223 ``` The query returns the arctangents of field values in the `altitude_ft` field key divided by values in the `distance_ft` field key. Both are part of the `flight_data` measurement. ###### Calculate the arctangent of values associated with each field key in a measurement divided by field\_key\_x ```sql > SELECT ATAN2(*, "distance_ft") FROM "flight_data" WHERE time >= '2018-05-16T12:01:00Z' AND time <= '2018-05-16T12:10:00Z' name: flight_data time atan2_altitude_ft atan2_distance_ft ---- ----------------- ----------------- 2018-05-16T12:01:00Z 0.020478631571881498 0.7853981633974483 2018-05-16T12:02:00Z 0.04754142349303296 0.7853981633974483 2018-05-16T12:03:00Z 0.07292147724575364 0.7853981633974483 2018-05-16T12:04:00Z 0.09495251193874832 0.7853981633974483 2018-05-16T12:05:00Z 0.11490822875441563 0.7853981633974483 2018-05-16T12:06:00Z 0.13176409347584003 0.7853981633974483 2018-05-16T12:07:00Z 0.14923587589682233 0.7853981633974483 2018-05-16T12:08:00Z 0.1671059946640312 0.7853981633974483 2018-05-16T12:09:00Z 0.18182893717409565 0.7853981633974483 2018-05-16T12:10:00Z 0.19370286314952234 0.7853981633974483 ``` The query returns the arctangents of all numeric field values in the `flight_data` measurement divided by values in the `distance_ft` field key. The `flight_data` measurement has two numeric fields: `altitude_ft` and `distance_ft`. ###### Calculate the arctangents of field values and include several clauses ```sql > SELECT ATAN2("altitude_ft", "distance_ft") FROM "flight_data" WHERE time >= '2018-05-16T12:01:00Z' AND time <= '2018-05-16T12:10:00Z' ORDER BY time DESC LIMIT 4 OFFSET 2 name: flight_data time atan2 ---- ----- 2018-05-16T12:08:00Z 0.1671059946640312 2018-05-16T12:07:00Z 0.14923587589682233 2018-05-16T12:06:00Z 0.13176409347584003 2018-05-16T12:05:00Z 0.11490822875441563 ``` The query returns the arctangent of field values associated with the `altitude_ft` field key divided by the `distance_ft` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2018-05-16T12:10:00Z` and `2018-05-16T12:10:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT ATAN2(, ) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `ATAN2()` function to those results. `ATAN2()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate arctangents of mean values ```sql > SELECT ATAN2(MEAN("altitude_ft"), MEAN("distance_ft")) FROM "flight_data" WHERE time >= '2018-05-16T12:01:00Z' AND time <= '2018-05-16T13:01:00Z' GROUP BY time(12m) name: flight_data time atan2 ---- ----- 2018-05-16T12:00:00Z 0.133815587896842 2018-05-16T12:12:00Z 0.2662716308351908 2018-05-16T12:24:00Z 0.2958845306108965 2018-05-16T12:36:00Z 0.23783439588429497 2018-05-16T12:48:00Z 0.1906803720242831 2018-05-16T13:00:00Z 0.17291511946158172 ``` The query returns the argtangents of [average](#mean) `altitude_ft`s divided by average `distance_ft`s. Averages are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `altitude_ft`s and `distance_ft` at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `ATAN2()`: ^ ```sql > SELECT MEAN("altitude_ft"), MEAN("distance_ft") FROM "flight_data" WHERE time >= '2018-05-16T12:01:00Z' AND time <= '2018-05-16T13:01:00Z' GROUP BY time(12m) name: flight_data time mean mean_1 ---- ---- ------ 2018-05-16T12:00:00Z 8674 64433.181818181816 2018-05-16T12:12:00Z 26419.833333333332 96865.25 2018-05-16T12:24:00Z 40337.416666666664 132326.41666666666 2018-05-16T12:36:00Z 41149.583333333336 169743.16666666666 2018-05-16T12:48:00Z 41230.416666666664 213600.91666666666 2018-05-16T13:00:00Z 41184.5 235799 ``` InfluxDB then calculates the arctangents of those averages. ### CEIL() Returns the subsequent value rounded up to the nearest integer. #### Basic syntax ``` SELECT CEIL( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `CEIL(field_key)` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key) rounded up to the nearest integer. `CEIL(*)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement) rounded up to the nearest integer. `CEIL()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `CEIL()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the ceiling of field values associated with a field key ```sql > SELECT CEIL("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time ceil ---- ---- 2015-08-18T00:00:00Z 3 2015-08-18T00:06:00Z 3 2015-08-18T00:12:00Z 3 2015-08-18T00:18:00Z 3 2015-08-18T00:24:00Z 3 2015-08-18T00:30:00Z 3 ``` The query returns field values in the `water_level` field key in the `h2o_feet` measurement rounded up to the nearest integer. ###### Calculate the ceiling of field values associated with each field key in a measurement ```sql > SELECT CEIL(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time ceil_water_level ---- ---------------- 2015-08-18T00:00:00Z 3 2015-08-18T00:06:00Z 3 2015-08-18T00:12:00Z 3 2015-08-18T00:18:00Z 3 2015-08-18T00:24:00Z 3 2015-08-18T00:30:00Z 3 ``` The query returns field values for each field key that stores numerical values in the `h2o_feet` measurement rounded up to the nearest integer. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the ceiling of field values associated with a field key and include several clauses ```sql > SELECT CEIL("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time ceil ---- ---- 2015-08-18T00:18:00Z 3 2015-08-18T00:12:00Z 3 2015-08-18T00:06:00Z 3 2015-08-18T00:00:00Z 3 ``` The query returns field values associated with the `water_level` field key rounded up to the nearest integer. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT CEIL(( [ * | | // ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `CEIL()` function to those results. `CEIL()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate mean values rounded up to the nearest integer ```sql > SELECT CEIL(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time ceil ---- ---- 2015-08-18T00:00:00Z 3 2015-08-18T00:12:00Z 3 2015-08-18T00:24:00Z 3 ``` The query returns the [average](#mean) `water_level`s that are calculated at 12-minute intervals and rounds them up to the nearest integer. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `CEIL()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then rounds those averages up to the nearest integer. ### COS() Returns the cosine of the field value. #### Basic syntax ``` SELECT COS( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `COS(field_key)` Returns the cosine of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `COS(*)` Returns the cosine of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `COS()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `COS()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ^ ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the cosine of field values associated with a field key ```sql > SELECT COS("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time cos ---- --- 2015-08-18T00:00:00Z -0.47345017433543124 2015-08-18T00:06:00Z -0.5185922462666872 2015-08-18T00:12:00Z -0.4414407189100776 2015-08-18T00:18:00Z -0.5271163912192579 2015-08-18T00:24:00Z -0.45306786455514825 2015-08-18T00:30:00Z -0.4619598230611262 ``` The query returns cosine of field values in the `water_level` field key in the `h2o_feet` measurement. ###### Calculate the cosine of field values associated with each field key in a measurement ```sql > SELECT COS(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time cos_water_level ---- --------------- 2015-08-18T00:00:00Z -0.47345017433543124 2015-08-18T00:06:00Z -0.5185922462666872 2015-08-18T00:12:00Z -0.4414407189100776 2015-08-18T00:18:00Z -0.5271163912192579 2015-08-18T00:24:00Z -0.45306786455514825 2015-08-18T00:30:00Z -0.4619598230611262 ``` The query returns cosine of field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the cosine of field values associated with a field key and include several clauses ```sql > SELECT COS("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time cos ---- --- 2015-08-18T00:18:00Z -0.5271163912192579 2015-08-18T00:12:00Z -0.4414407189100776 2015-08-18T00:06:00Z -0.5185922462666872 2015-08-18T00:00:00Z -0.47345017433543124 ``` The query returns cosine of field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT COS(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `COS()` function to those results. `COS()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). #### Examples ###### Calculate the cosine of mean values ```sql > SELECT COS(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time cos ---- --- 2015-08-18T00:00:00Z -0.49618891270599885 2015-08-18T00:12:00Z -0.4848605136571181 2015-08-18T00:24:00Z -0.4575195627907578 ``` The query returns cosine of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `COS()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates cosine of those averages. ### CUMULATIVE\_SUM() Returns the running total of subsequent [field values](/influxdb/v1/concepts/glossary/#field-value). #### Basic syntax ``` SELECT CUMULATIVE_SUM( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `CUMULATIVE_SUM(field_key)` Returns the running total of subsequent field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `CUMULATIVE_SUM(/regular_expression/)` Returns the running total of subsequent field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `CUMULATIVE_SUM(*)` Returns the running total of subsequent field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `CUMULATIVE_SUM()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `CUMULATIVE_SUM()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the cumulative sum of the field values associated with a field key ```sql > SELECT CUMULATIVE_SUM("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time cumulative_sum ---- -------------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 4.18 2015-08-18T00:12:00Z 6.208 2015-08-18T00:18:00Z 8.334 2015-08-18T00:24:00Z 10.375 2015-08-18T00:30:00Z 12.426 ``` The query returns the running total of the field values in the `water_level` field key and in the `h2o_feet` measurement. ###### Calculate the cumulative sum of the field values associated with each field key in a measurement ```sql > SELECT CUMULATIVE_SUM(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time cumulative_sum_water_level ---- -------------------------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 4.18 2015-08-18T00:12:00Z 6.208 2015-08-18T00:18:00Z 8.334 2015-08-18T00:24:00Z 10.375 2015-08-18T00:30:00Z 12.426 ``` The query returns the running total of the field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the cumulative sum of the field values associated with each field key that matches a regular expression ```sql > SELECT CUMULATIVE_SUM(/water/) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time cumulative_sum_water_level ---- -------------------------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 4.18 2015-08-18T00:12:00Z 6.208 2015-08-18T00:18:00Z 8.334 2015-08-18T00:24:00Z 10.375 2015-08-18T00:30:00Z 12.426 ``` The query returns the running total of the field values for each field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. ###### Calculate the cumulative sum of the field values associated with a field key and include several clauses ```sql > SELECT CUMULATIVE_SUM("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time cumulative_sum ---- -------------- 2015-08-18T00:18:00Z 6.218 2015-08-18T00:12:00Z 8.246 2015-08-18T00:06:00Z 10.362 2015-08-18T00:00:00Z 12.426 ``` The query returns the running total of the field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT CUMULATIVE_SUM(( [ * | | // ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `CUMULATIVE_SUM()` function to those results. `CUMULATIVE_SUM()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the cumulative sum of mean values ```sql > SELECT CUMULATIVE_SUM(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time cumulative_sum ---- -------------- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 4.167 2015-08-18T00:24:00Z 6.213 ``` The query returns the running total of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `CUMULATIVE_SUM()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` Next, InfluxDB calculates the running total of those averages. The second point in the final results (`4.167`) is the sum of `2.09` and `2.077`and the third point (`6.213`) is the sum of `2.09`, `2.077`, and `2.0460000000000003`. ### DERIVATIVE() Returns the rate of change between subsequent [field values](/influxdb/v1/concepts/glossary/#field-value). #### Basic syntax ``` SELECT DERIVATIVE( [ * | | // ] [ , ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` InfluxDB calculates the difference between subsequent field values and converts those results into the rate of change per `unit`. The `unit` argument is an integer followed by a [duration literal](/influxdb/v1/query_language/spec/#literals) and it is optional. If the query does not specify the `unit` the unit defaults to one second (`1s`). `DERIVATIVE(field_key)` Returns the rate of change between subsequent field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `DERIVATIVE(/regular_expression/)` Returns the rate of change between subsequent field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `DERIVATIVE(*)` Returns the rate of change between subsequent field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `DERIVATIVE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax-1) section for how to use `DERIVATIVE()` with a `GROUP BY time()` clause. ##### Examples Examples 1-5 use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the derivative between the field values associated with a field key ```sql > SELECT DERIVATIVE("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time derivative ---- ---------- 2015-08-18T00:06:00Z 0.00014444444444444457 2015-08-18T00:12:00Z -0.00024444444444444465 2015-08-18T00:18:00Z 0.0002722222222222218 2015-08-18T00:24:00Z -0.000236111111111111 2015-08-18T00:30:00Z 2.777777777777842e-05 ``` The query returns the one-second rate of change between the field values associated with the `water_level` field key and in the `h2o_feet` measurement. The first result (`0.00014444444444444457`) is the one-second rate of change between the first two subsequent field values in the raw data. InfluxDB calculates the difference between the field values and normalizes that value to the one-second rate of change: ``` (2.116 - 2.064) / (360s / 1s) -------------- ---------- | | | the difference between the field values' timestamps / the default unit second field value - first field value ``` ###### Calculate the derivative between the field values associated with a field key and specify the unit option ```sql > SELECT DERIVATIVE("water_level",6m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time derivative ---- ---------- 2015-08-18T00:06:00Z 0.052000000000000046 2015-08-18T00:12:00Z -0.08800000000000008 2015-08-18T00:18:00Z 0.09799999999999986 2015-08-18T00:24:00Z -0.08499999999999996 2015-08-18T00:30:00Z 0.010000000000000231 ``` The query returns the six-minute rate of change between the field values associated with the `water_level` field key and in the `h2o_feet` measurement. The first result (`0.052000000000000046`) is the six-minute rate of change between the first two subsequent field values in the raw data. InfluxDB calculates the difference between the field values and normalizes that value to the six-minute rate of change: ``` (2.116 - 2.064) / (6m / 6m) -------------- ---------- | | | the difference between the field values' timestamps / the specified unit second field value - first field value ``` ###### Calculate the derivative between the field values associated with each field key in a measurement and specify the unit option ``` > SELECT DERIVATIVE(*,3m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time derivative_water_level ---- ---------------------- 2015-08-18T00:06:00Z 0.026000000000000023 2015-08-18T00:12:00Z -0.04400000000000004 2015-08-18T00:18:00Z 0.04899999999999993 2015-08-18T00:24:00Z -0.04249999999999998 2015-08-18T00:30:00Z 0.0050000000000001155 ``` The query returns the three-minute rate of change between the field values associated with each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. The first result (`0.026000000000000023`) is the three-minute rate of change between the first two subsequent field values in the raw data. InfluxDB calculates the difference between the field values and normalizes that value to the three-minute rate of change: ``` (2.116 - 2.064) / (6m / 3m) -------------- ---------- | | | the difference between the field values' timestamps / the specified unit second field value - first field value ``` ###### Calculate the derivative between the field values associated with each field key that matches a regular expression and specify the unit option ```sql > SELECT DERIVATIVE(/water/,2m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time derivative_water_level ---- ---------------------- 2015-08-18T00:06:00Z 0.01733333333333335 2015-08-18T00:12:00Z -0.02933333333333336 2015-08-18T00:18:00Z 0.03266666666666662 2015-08-18T00:24:00Z -0.02833333333333332 2015-08-18T00:30:00Z 0.0033333333333334103 ``` The query returns the two-minute rate of change between the field values associated with each field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. The first result (`0.01733333333333335`) is the two-minute rate of change between the first two subsequent field values in the raw data. InfluxDB calculates the difference between the field values and normalizes that value to the two-minute rate of change: ``` (2.116 - 2.064) / (6m / 2m) -------------- ---------- | | | the difference between the field values' timestamps / the specified unit second field value - first field value ``` ###### Calculate the derivative between the field values associated with a field key and include several clauses ```sql > SELECT DERIVATIVE("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' ORDER BY time DESC LIMIT 1 OFFSET 2 name: h2o_feet time derivative ---- ---------- 2015-08-18T00:12:00Z -0.0002722222222222218 ``` The query returns the one-second rate of change between the field values associated with the `water_level` field key and in the `h2o_feet` measurement. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to one and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. The only result (`-0.0002722222222222218`) is the one-second rate of change between the relevant subsequent field values in the raw data. InfluxDB calculates the difference between the field values and normalizes that value to the one-second rate of change: ``` (2.126 - 2.028) / (360s / 1s) -------------- ---------- | | | the difference between the field values' timestamps / the default unit second field value - first field value ``` #### Advanced syntax ``` SELECT DERIVATIVE( ([ * | | // ]) [ , ] ) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `DERIVATIVE()` function to those results. The `unit` argument is an integer followed by a [duration literal](/influxdb/v1/query_language/spec/#literals) and it is optional. If the query does not specify the `unit` the `unit` defaults to the `GROUP BY time()` interval. Note that this behavior is different from the [basic syntax’s](#basic-syntax-1) default behavior. `DERIVATIVE()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the derivative of mean values ```sql > SELECT DERIVATIVE(MEAN("water_level")) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) name: h2o_feet time derivative ---- ---------- 2015-08-18T00:12:00Z -0.0129999999999999 2015-08-18T00:24:00Z -0.030999999999999694 ``` The query returns the 12-minute rate of change between [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `DERIVATIVE()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` Next, InfluxDB calculates the 12-minute rate of change between those averages. The first result (`-0.0129999999999999`) is the 12-minute rate of change between the first two averages. InfluxDB calculates the difference between the field values and normalizes that value to the 12-minute rate of change. ``` (2.077 - 2.09) / (12m / 12m) ------------- ---------- | | | the difference between the field values' timestamps / the default unit second field value - first field value ``` ###### Calculate the derivative of mean values and specify the unit option ```sql > SELECT DERIVATIVE(MEAN("water_level"),6m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) name: h2o_feet time derivative ---- ---------- 2015-08-18T00:12:00Z -0.00649999999999995 2015-08-18T00:24:00Z -0.015499999999999847 ``` The query returns the six-minute rate of change between average `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `DERIVATIVE()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` Next, InfluxDB calculates the six-minute rate of change between those averages. The first result (`-0.00649999999999995`) is the six-minute rate of change between the first two averages. InfluxDB calculates the difference between the field values and normalizes that value to the six-minute rate of change. ``` (2.077 - 2.09) / (12m / 6m) ------------- ---------- | | | the difference between the field values' timestamps / the specified unit second field value - first field value ``` ### DIFFERENCE() Returns the result of subtraction between subsequent [field values](/influxdb/v1/concepts/glossary/#field-value). #### Basic syntax ``` SELECT DIFFERENCE( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `DIFFERENCE(field_key)` Returns the difference between subsequent field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `DIFFERENCE(/regular_expression/)` Returns the difference between subsequent field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `DIFFERENCE(*)` Returns the difference between subsequent field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `DIFFERENCE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax-2) section for how to use `DIFFERENCE()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the difference between the field values associated with a field key ```sql > SELECT DIFFERENCE("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time difference ---- ---------- 2015-08-18T00:06:00Z 0.052000000000000046 2015-08-18T00:12:00Z -0.08800000000000008 2015-08-18T00:18:00Z 0.09799999999999986 2015-08-18T00:24:00Z -0.08499999999999996 2015-08-18T00:30:00Z 0.010000000000000231 ``` The query returns the difference between the subsequent field values in the `water_level` field key and in the `h2o_feet` measurement. ###### Calculate the difference between the field values associated with each field key in a measurement ```sql > SELECT DIFFERENCE(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time difference_water_level ---- ---------------------- 2015-08-18T00:06:00Z 0.052000000000000046 2015-08-18T00:12:00Z -0.08800000000000008 2015-08-18T00:18:00Z 0.09799999999999986 2015-08-18T00:24:00Z -0.08499999999999996 2015-08-18T00:30:00Z 0.010000000000000231 ``` The query returns the difference between the subsequent field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the difference between the field values associated with each field key that matches a regular expression ```sql > SELECT DIFFERENCE(/water/) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time difference_water_level ---- ---------------------- 2015-08-18T00:06:00Z 0.052000000000000046 2015-08-18T00:12:00Z -0.08800000000000008 2015-08-18T00:18:00Z 0.09799999999999986 2015-08-18T00:24:00Z -0.08499999999999996 2015-08-18T00:30:00Z 0.010000000000000231 ``` The query returns the difference between the subsequent field values for each field key that stores numerical values and includes the word `water` in the `h2o_feet` measurement. ###### Calculate the difference between the field values associated with a field key and include several clauses ```sql > SELECT DIFFERENCE("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 2 OFFSET 2 name: h2o_feet time difference ---- ---------- 2015-08-18T00:12:00Z -0.09799999999999986 2015-08-18T00:06:00Z 0.08800000000000008 ``` The query returns the difference between the subsequent field values in the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). They query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to two and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT DIFFERENCE(( [ * | | // ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `DIFFERENCE()` function to those results. `DIFFERENCE()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the difference between maximum values ```sql > SELECT DIFFERENCE(MAX("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time difference ---- ---------- 2015-08-18T00:12:00Z 0.009999999999999787 2015-08-18T00:24:00Z -0.07499999999999973 ``` The query returns the difference between [maximum](#max) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the maximum `water_level`s at 12-minute intervals. This step is the same as using the `MAX()` function with the `GROUP BY time()` clause and without `DIFFERENCE()`: ```sql > SELECT MAX("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time max ---- --- 2015-08-18T00:00:00Z 2.116 2015-08-18T00:12:00Z 2.126 2015-08-18T00:24:00Z 2.051 ``` Next, InfluxDB calculates the difference between those maximum values. The first point in the final results (`0.009999999999999787`) is the difference between `2.126` and `2.116`, and the second point in the final results (`-0.07499999999999973`) is the difference between `2.051` and `2.126`. ### ELAPSED() Returns the difference between subsequent [field value’s](/influxdb/v1/concepts/glossary/#field-value) timestamps. #### Syntax ``` SELECT ELAPSED( [ * | | // ] [ , ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` InfluxDB calculates the difference between subsequent timestamps. The `unit` option is an integer followed by a [duration literal](/influxdb/v1/query_language/spec/#literals) and it determines the unit of the returned difference. If the query does not specify the `unit` option the query returns the difference between timestamps in nanoseconds. `ELAPSED(field_key)` Returns the difference between subsequent timestamps associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `ELAPSED(/regular_expression/)` Returns the difference between subsequent timestamps associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `ELAPSED(*)` Returns the difference between subsequent timestamps associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `ELAPSED()` supports all field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples The examples use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:12:00Z' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 ``` ##### Calculate the elapsed time between field values associated with a field key ```sql > SELECT ELAPSED("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:12:00Z' name: h2o_feet time elapsed ---- ------- 2015-08-18T00:06:00Z 360000000000 2015-08-18T00:12:00Z 360000000000 ``` The query returns the difference (in nanoseconds) between subsequent timestamps in the `water_level` field key and in the `h2o_feet` measurement. ##### Calculate the elapsed time between field values associated with a field key and specify the unit option ```sql > SELECT ELAPSED("water_level",1m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:12:00Z' name: h2o_feet time elapsed ---- ------- 2015-08-18T00:06:00Z 6 2015-08-18T00:12:00Z 6 ``` The query returns the difference (in minutes) between subsequent timestamps in the `water_level` field key and in the `h2o_feet` measurement. ##### Calculate the elapsed time between field values associated with each field key in a measurement and specify the unit option ```sql > SELECT ELAPSED(*,1m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:12:00Z' name: h2o_feet time elapsed_level description elapsed_water_level ---- ------------------------- ------------------- 2015-08-18T00:06:00Z 6 6 2015-08-18T00:12:00Z 6 6 ``` The query returns the difference (in minutes) between subsequent timestamps associated with each field key in the `h2o_feet`measurement. The `h2o_feet` measurement has two field keys: `level description` and `water_level`. ##### Calculate the elapsed time between field values associated with each field key that matches a regular expression and specify the unit option ```sql > SELECT ELAPSED(/level/,1s) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:12:00Z' name: h2o_feet time elapsed_level description elapsed_water_level ---- ------------------------- ------------------- 2015-08-18T00:06:00Z 360 360 2015-08-18T00:12:00Z 360 360 ``` The query returns the difference (in seconds) between subsequent timestamps associated with each field key that includes the word `level` in the `h2o_feet` measurement. ##### Calculate the elapsed time between field values associated with a field key and include several clauses ```sql > SELECT ELAPSED("water_level",1ms) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:12:00Z' ORDER BY time DESC LIMIT 1 OFFSET 1 name: h2o_feet time elapsed ---- ------- 2015-08-18T00:00:00Z -360000 ``` The query returns the difference (in milliseconds) between subsequent timestamps in the `water_level` field key and in the `h2o_feet` measurement. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:12:00Z` and sorts timestamps in [descending order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to one and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by one point. Notice that the result is negative; the [`ORDER BY time DESC` clause](/influxdb/v1/query_language/explore-data/#order-by-time-desc) sorts timestamps in descending order so `ELAPSED()` calculates the difference between timestamps in reverse order. ### Common Issues with ELAPSED() #### ELAPSED() and units greater than the elapsed time InfluxDB returns `0` if the `unit` option is greater than the difference between the timestamps. ##### Example The timestamps in the `h2o_feet` measurement occur at six-minute intervals. If the query sets the `unit` option to one hour, InfluxDB returns `0`: ```sql > SELECT ELAPSED("water_level",1h) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:12:00Z' name: h2o_feet time elapsed ---- ------- 2015-08-18T00:06:00Z 0 2015-08-18T00:12:00Z 0 ``` #### ELAPSED() with GROUP BY time() clauses The `ELAPSED()` function supports the [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) but the query results aren’t particularly useful. Currently, an `ELAPSED()` query with a nested function and a `GROUP BY time()` clause simply returns the interval specified in the `GROUP BY time()` clause. The `GROUP BY time()` clause determines the timestamps in the results; each timestamp marks the start of a time interval. That behavior also applies to nested selector functions (like [`FIRST()`](#first) or [`MAX()`](#max)) which would, in all other cases, return a specific timestamp from the raw data. Because the `GROUP BY time()` clause overrides the original timestamps, the `ELAPSED()` calculation always returns the same value as the `GROUP BY time()` interval. ##### Example In the codeblock below, the first query attempts to use the `ELAPSED()` function with a `GROUP BY time()` clause to find the time elapsed (in minutes) between [minimum](#min) `water_level`s. The query returns 12 minutes for both time intervals. To get those results, InfluxDB first calculates the minimum `water_level`s at 12-minute intervals. The second query in the codeblock shows the results of that step. The step is the same as using the `MIN()` function with the `GROUP BY time()` clause and without the `ELAPSED()` function. Notice that the timestamps returned by the second query are 12 minutes apart. In the raw data, the first result (`2.057`) occurs at `2015-08-18T00:42:00Z` but the `GROUP BY time()` clause overrides that original timestamp. Because the timestamps are determined by the `GROUP BY time()` interval and not by the original data, the `ELAPSED()` calculation always returns the same value as the `GROUP BY time()` interval. ```sql > SELECT ELAPSED(MIN("water_level"),1m) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:36:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m) name: h2o_feet time elapsed ---- ------- 2015-08-18T00:36:00Z 12 2015-08-18T00:48:00Z 12 > SELECT MIN("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:36:00Z' AND time <= '2015-08-18T00:54:00Z' GROUP BY time(12m) name: h2o_feet time min ---- --- 2015-08-18T00:36:00Z 2.057 <--- Actually occurs at 2015-08-18T00:42:00Z 2015-08-18T00:48:00Z 1.991 ``` ### EXP() Returns the exponential of the field value. #### Basic syntax ``` SELECT EXP( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `EXP(field_key)` Returns the exponential of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `EXP(*)` Returns the exponential of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `EXP()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `EXP()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the exponential of field values associated with a field key ```sql > SELECT EXP("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time exp ---- --- 2015-08-18T00:00:00Z 7.877416541092307 2015-08-18T00:06:00Z 8.297879498060171 2015-08-18T00:12:00Z 7.598873404088091 2015-08-18T00:18:00Z 8.381274573459967 2015-08-18T00:24:00Z 7.6983036546645645 2015-08-18T00:30:00Z 7.775672892658607 ``` The query returns the exponential of field values in the `water_level` field key in the `h2o_feet` measurement. ###### Calculate the exponential of field values associated with each field key in a measurement ```sql > SELECT EXP(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time exp_water_level ---- --------------- 2015-08-18T00:00:00Z 7.877416541092307 2015-08-18T00:06:00Z 8.297879498060171 2015-08-18T00:12:00Z 7.598873404088091 2015-08-18T00:18:00Z 8.381274573459967 2015-08-18T00:24:00Z 7.6983036546645645 2015-08-18T00:30:00Z 7.775672892658607 ``` The query returns the exponential of field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the exponential of field values associated with a field key and include several clauses ```sql > SELECT EXP("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time exp ---- --- 2015-08-18T00:18:00Z 8.381274573459967 2015-08-18T00:12:00Z 7.598873404088091 2015-08-18T00:06:00Z 8.297879498060171 2015-08-18T00:00:00Z 7.877416541092307 ``` The query returns the exponentials of field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT EXP(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `EXP()` function to those results. `EXP()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the exponential of mean values. ```sql > SELECT EXP(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time exp ---- --- 2015-08-18T00:00:00Z 8.084915164305059 2015-08-18T00:12:00Z 7.980491491670466 2015-08-18T00:24:00Z 7.736891562315577 ``` The query returns the exponential of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `EXP()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates the exponentials of those averages. ### FLOOR() Returns the subsequent value rounded down to the nearest integer. #### Basic syntax ``` SELECT FLOOR( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `FLOOR(field_key)` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key) rounded down to the nearest integer. `FLOOR(*)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement) rounded down to the nearest integer. `FLOOR()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `FLOOR()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the floor of field values associated with a field key ```sql > SELECT FLOOR("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time floor ---- ----- 2015-08-18T00:00:00Z 2 2015-08-18T00:06:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:18:00Z 2 2015-08-18T00:24:00Z 2 2015-08-18T00:30:00Z 2 ``` The query returns field values in the `water_level` field key in the `h2o_feet` measurement rounded down to the nearest integer. ###### Calculate the floor of field values associated with each field key in a measurement ```sql > SELECT FLOOR(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time floor_water_level ---- ----------------- 2015-08-18T00:00:00Z 2 2015-08-18T00:06:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:18:00Z 2 2015-08-18T00:24:00Z 2 2015-08-18T00:30:00Z 2 ``` The query returns field values for each field key that stores numerical values in the `h2o_feet` measurement rounded down to the nearest integer. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the floor of field values associated with a field key and include several clauses ```sql > SELECT FLOOR("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time floor ---- ----- 2015-08-18T00:18:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:06:00Z 2 2015-08-18T00:00:00Z 2 ``` The query returns field values associated with the `water_level` field key rounded down to the nearest integer. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT FLOOR(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `FLOOR()` function to those results. `FLOOR()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate mean values rounded down to the nearest integer. ```sql > SELECT FLOOR(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time floor ---- ----- 2015-08-18T00:00:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:24:00Z 2 ``` The query returns the [average](#mean) `water_level`s that are calculated at 12-minute intervals and rounds them up to the nearest integer. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `FLOOR()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then rounds those averages down to the nearest integer. ### HISTOGRAM() *InfluxQL does not currently support histogram generation. For information about creating histograms with data stored in InfluxDB, see[Flux’s `histogram()` function](/flux/v0/stdlib/universe/histogram).* ### LN() Returns the natural logarithm of the field value. #### Basic syntax ``` SELECT LN( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `LN(field_key)` Returns the natural logarithm of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `LN(*)` Returns the natural logarithm of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `LN()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `LN()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the natural logarithm of field values associated with a field key ```sql > SELECT LN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time ln ---- -- 2015-08-18T00:00:00Z 0.7246458476193163 2015-08-18T00:06:00Z 0.749527513996053 2015-08-18T00:12:00Z 0.7070500857289368 2015-08-18T00:18:00Z 0.7542422799197561 2015-08-18T00:24:00Z 0.7134398838277077 2015-08-18T00:30:00Z 0.7183274790902436 ``` The query returns the natural logarithm of field values in the `water_level` field key in the `h2o_feet` measurement. ###### Calculate the natural logarithm of field values associated with each field key in a measurement ```sql > SELECT LN(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time ln_water_level ---- -------------- 2015-08-18T00:00:00Z 0.7246458476193163 2015-08-18T00:06:00Z 0.749527513996053 2015-08-18T00:12:00Z 0.7070500857289368 2015-08-18T00:18:00Z 0.7542422799197561 2015-08-18T00:24:00Z 0.7134398838277077 2015-08-18T00:30:00Z 0.7183274790902436 ``` The query returns the natural logarithm of field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the natural logarithm of field values associated with a field key and include several clauses ```sql > SELECT LN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time ln ---- -- 2015-08-18T00:18:00Z 0.7542422799197561 2015-08-18T00:12:00Z 0.7070500857289368 2015-08-18T00:06:00Z 0.749527513996053 2015-08-18T00:00:00Z 0.7246458476193163 ``` The query returns the natural logarithms of field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT LN(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `LN()` function to those results. `LN()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the natural logarithm of mean values. ```sql > SELECT LN(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time ln ---- -- 2015-08-18T00:00:00Z 0.7371640659767196 2015-08-18T00:12:00Z 0.7309245448939752 2015-08-18T00:24:00Z 0.7158866675294349 ``` The query returns the natural logarithm of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `LN()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates the natural logarithms of those averages. ### LOG() Returns the logarithm of the field value with base `b`. #### Basic syntax ``` SELECT LOG( [ * | ], ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `LOG(field_key, b)` Returns the logarithm of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key) with base `b`. `LOG(*, b)` Returns the logarithm of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement) with base `b`. `LOG()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `LOG()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the logarithm base 4 of field values associated with a field key ```sql > SELECT LOG("water_level", 4) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time log ---- --- 2015-08-18T00:00:00Z 0.5227214853805835 2015-08-18T00:06:00Z 0.5406698137259695 2015-08-18T00:12:00Z 0.5100288261706268 2015-08-18T00:18:00Z 0.5440707984345088 2015-08-18T00:24:00Z 0.5146380911853161 2015-08-18T00:30:00Z 0.5181637459088826 ``` The query returns the logarithm base 4 of field values in the `water_level` field key in the `h2o_feet` measurement. ###### Calculate the logarithm base 4 of field values associated with each field key in a measurement ```sql > SELECT LOG(*, 4) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time log_water_level ---- --------------- 2015-08-18T00:00:00Z 0.5227214853805835 2015-08-18T00:06:00Z 0.5406698137259695 2015-08-18T00:12:00Z 0.5100288261706268 2015-08-18T00:18:00Z 0.5440707984345088 2015-08-18T00:24:00Z 0.5146380911853161 2015-08-18T00:30:00Z 0.5181637459088826 ``` The query returns the logarithm base 4 of field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the logarithm base 4 of field values associated with a field key and include several clauses ```sql > SELECT LOG("water_level", 4) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time log ---- --- 2015-08-18T00:18:00Z 0.5440707984345088 2015-08-18T00:12:00Z 0.5100288261706268 2015-08-18T00:06:00Z 0.5406698137259695 2015-08-18T00:00:00Z 0.5227214853805835 ``` The query returns the logarithm base 4 of field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT LOG(( [ * | ] ), ) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `LOG()` function to those results. `LOG()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the logarithm base 4 of mean values ```sql > SELECT LOG(MEAN("water_level"), 4) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time log ---- --- 2015-08-18T00:00:00Z 0.531751471153079 2015-08-18T00:12:00Z 0.5272506080912802 2015-08-18T00:24:00Z 0.5164030725416209 ``` The query returns the logarithm base 4 of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `LOG()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates the logarithm base 4 of those averages. ### LOG2() Returns the logarithm of the field value to the base 2. #### Basic syntax ``` SELECT LOG2( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `LOG2(field_key)` Returns the logarithm of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key) to the base 2. `LOG2(*)` Returns the logarithm of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement) to the base 2. `LOG2()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced syntax](#advanced-syntax) section for how to use `LOG2()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the logarithm base 2 of field values associated with a field key ```sql > SELECT LOG2("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time log2 ---- ---- 2015-08-18T00:00:00Z 1.045442970761167 2015-08-18T00:06:00Z 1.081339627451939 2015-08-18T00:12:00Z 1.0200576523412537 2015-08-18T00:18:00Z 1.0881415968690176 2015-08-18T00:24:00Z 1.0292761823706322 2015-08-18T00:30:00Z 1.0363274918177652 ``` The query returns the logarithm base 2 of field values in the `water_level` field key in the `h2o_feet` measurement. ###### Calculate the logarithm base 2 of field values associated with each field key in a measurement ```sql > SELECT LOG2(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time log2_water_level ---- ---------------- 2015-08-18T00:00:00Z 1.045442970761167 2015-08-18T00:06:00Z 1.081339627451939 2015-08-18T00:12:00Z 1.0200576523412537 2015-08-18T00:18:00Z 1.0881415968690176 2015-08-18T00:24:00Z 1.0292761823706322 2015-08-18T00:30:00Z 1.0363274918177652 ``` The query returns the logarithm base 2 of field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the logarithm base 2 of field values associated with a field key and include several clauses ```sql > SELECT LOG2("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time log2 ---- ---- 2015-08-18T00:18:00Z 1.0881415968690176 2015-08-18T00:12:00Z 1.0200576523412537 2015-08-18T00:06:00Z 1.081339627451939 2015-08-18T00:00:00Z 1.045442970761167 ``` The query returns the logarithm base 2 of field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ```sql SELECT LOG2(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `LOG2()` function to those results. `LOG2()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the logarithm base 2 of mean values ```sql > SELECT LOG2(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time log2 ---- ---- 2015-08-18T00:00:00Z 1.063502942306158 2015-08-18T00:12:00Z 1.0545012161825604 2015-08-18T00:24:00Z 1.0328061450832418 ``` The query returns the logarithm base 2 of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `LOG2()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates the logarithm base 2 of those averages. ### LOG10() Returns the logarithm of the field value to the base 10. #### Basic syntax ``` SELECT LOG10( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `LOG10(field_key)` Returns the logarithm of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key) to the base 10. `LOG10(*)` Returns the logarithm of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement) to the base 10. `LOG10()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `LOG10()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the logarithm base 10 of field values associated with a field key ```sql > SELECT LOG10("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time log10 ---- ----- 2015-08-18T00:00:00Z 0.3147096929551737 2015-08-18T00:06:00Z 0.32551566336314813 2015-08-18T00:12:00Z 0.3070679506612984 2015-08-18T00:18:00Z 0.32756326018727794 2015-08-18T00:24:00Z 0.3098430047160705 2015-08-18T00:30:00Z 0.3119656603683663 ``` The query returns the logarithm base 10 of field values in the `water_level` field key in the `h2o_feet` measurement. ###### Calculate the logarithm base 10 of field values associated with each field key in a measurement ```sql > SELECT LOG10(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time log10_water_level ---- ----------------- 2015-08-18T00:00:00Z 0.3147096929551737 2015-08-18T00:06:00Z 0.32551566336314813 2015-08-18T00:12:00Z 0.3070679506612984 2015-08-18T00:18:00Z 0.32756326018727794 2015-08-18T00:24:00Z 0.3098430047160705 2015-08-18T00:30:00Z 0.3119656603683663 ``` The query returns the logarithm base 10 of field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the logarithm base 10 of field values associated with a field key and include several clauses ```sql > SELECT LOG10("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time log10 ---- ----- 2015-08-18T00:18:00Z 0.32756326018727794 2015-08-18T00:12:00Z 0.3070679506612984 2015-08-18T00:06:00Z 0.32551566336314813 2015-08-18T00:00:00Z 0.3147096929551737 ``` The query returns the logarithm base 10 of field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT LOG10(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `LOG10()` function to those results. `LOG10()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the logarithm base 10 of mean values ```sql > SELECT LOG10(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time log10 ---- ----- 2015-08-18T00:00:00Z 0.32014628611105395 2015-08-18T00:12:00Z 0.3174364965350991 2015-08-18T00:24:00Z 0.3109056293761414 ``` The query returns the logarithm base 10 of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `LOG10()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates the logarithm base 10 of those averages. ### MOVING\_AVERAGE() Returns the rolling average across a window of subsequent [field values](/influxdb/v1/concepts/glossary/#field-value). #### Basic syntax ``` SELECT MOVING_AVERAGE( [ * | | // ] , ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `MOVING_AVERAGE()` calculates the rolling average across a window of `N` subsequent field values. The `N` argument is an integer and it is required. `MOVING_AVERAGE(field_key,N)` Returns the rolling average across `N` field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `MOVING_AVERAGE(/regular_expression/,N)` Returns the rolling average across `N` field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `MOVING_AVERAGE(*,N)` Returns the rolling average across `N` field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `MOVING_AVERAGE()` int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax-3) section for how to use `MOVING_AVERAGE()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the moving average of the field values associated with a field key ```sql > SELECT MOVING_AVERAGE("water_level",2) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time moving_average ---- -------------- 2015-08-18T00:06:00Z 2.09 2015-08-18T00:12:00Z 2.072 2015-08-18T00:18:00Z 2.077 2015-08-18T00:24:00Z 2.0835 2015-08-18T00:30:00Z 2.0460000000000003 ``` The query returns the rolling average across a two-field-value window for the `water_level` field key and the `h2o_feet` measurement. The first result (`2.09`) is the average of the first two points in the raw data: (`2.064 + 2.116) / 2`). The second result (`2.072`) is the average of the second two points in the raw data: (`2.116 + 2.028) / 2`). ###### Calculate the moving average of the field values associated with each field key in a measurement ```sql > SELECT MOVING_AVERAGE(*,3) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time moving_average_water_level ---- -------------------------- 2015-08-18T00:12:00Z 2.0693333333333332 2015-08-18T00:18:00Z 2.09 2015-08-18T00:24:00Z 2.065 2015-08-18T00:30:00Z 2.0726666666666667 ``` The query returns the rolling average across a three-field-value window for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the moving average of the field values associated with each field key that matches a regular expression ```sql > SELECT MOVING_AVERAGE(/level/,4) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' name: h2o_feet time moving_average_water_level ---- -------------------------- 2015-08-18T00:18:00Z 2.0835 2015-08-18T00:24:00Z 2.07775 2015-08-18T00:30:00Z 2.0615 ``` The query returns the rolling average across a four-field-value window for each field key that stores numerical values and includes the word `level` in the `h2o_feet` measurement. ###### Calculate the moving average of the field values associated with a field key and include several clauses ```sql > SELECT MOVING_AVERAGE("water_level",2) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' ORDER BY time DESC LIMIT 2 OFFSET 3 name: h2o_feet time moving_average ---- -------------- 2015-08-18T00:06:00Z 2.072 2015-08-18T00:00:00Z 2.09 ``` The query returns the rolling average across a two-field-value window for the `water_level` field key in the `h2o_feet` measurement. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to two and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by three points. #### Advanced syntax ``` SELECT MOVING_AVERAGE( ([ * | | // ]) , N ) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `MOVING_AVERAGE()` function to those results. `MOVING_AVERAGE()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the moving average of maximum values ```sql > SELECT MOVING_AVERAGE(MAX("water_level"),2) FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) name: h2o_feet time moving_average ---- -------------- 2015-08-18T00:12:00Z 2.121 2015-08-18T00:24:00Z 2.0885 ``` The query returns the rolling average across a two-value window of [maximum](#max) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the maximum `water_level`s at 12-minute intervals. This step is the same as using the `MAX()` function with the `GROUP BY time()` clause and without `MOVING_AVERAGE()`: ```sql > SELECT MAX("water_level") FROM "h2o_feet" WHERE "location" = 'santa_monica' AND time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' GROUP BY time(12m) name: h2o_feet time max ---- --- 2015-08-18T00:00:00Z 2.116 2015-08-18T00:12:00Z 2.126 2015-08-18T00:24:00Z 2.051 ``` Next, InfluxDB calculates the rolling average across a two-value window using those maximum values. The first final result (`2.121`) is the average of the first two maximum values (`(2.116 + 2.126) / 2`). ### NON\_NEGATIVE\_DERIVATIVE() Returns the non-negative rate of change between subsequent [field values](/influxdb/v1/concepts/glossary/#field-value). Non-negative rates of change include positive rates of change and rates of change that equal zero. #### Basic syntax ``` SELECT NON_NEGATIVE_DERIVATIVE( [ * | | // ] [ , ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` InfluxDB calculates the difference between subsequent field values and converts those results into the rate of change per `unit`. The `unit` argument is an integer followed by a [duration literal](/influxdb/v1/query_language/spec/#literals) and it is optional. If the query does not specify the `unit`, the unit defaults to one second (`1s`).`NON_NEGATIVE_DERIVATIVE()` returns only positive rates of change or rates of change that equal zero. `NON_NEGATIVE_DERIVATIVE(field_key)` Returns the non-negative rate of change between subsequent field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `NON_NEGATIVE_DERIVATIVE(/regular_expression/)` Returns the non-negative rate of change between subsequent field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `NON_NEGATIVE_DERIVATIVE(*)` Returns the non-negative rate of change between subsequent field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `NON_NEGATIVE_DERIVATIVE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax-4) section for how to use `NON_NEGATIVE_DERIVATIVE()` with a `GROUP BY time()` clause. ##### Examples See the examples in the [`DERIVATIVE()` documentation](#basic-syntax-8).`NON_NEGATIVE_DERIVATIVE()` behaves the same as the `DERIVATIVE()` function but `NON_NEGATIVE_DERIVATIVE()` returns only positive rates of change or rates of change that equal zero. #### Advanced syntax ``` SELECT NON_NEGATIVE_DERIVATIVE( ([ * | | // ]) [ , ] ) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `NON_NEGATIVE_DERIVATIVE()` function to those results. The `unit` argument is an integer followed by a [duration literal](/influxdb/v1/query_language/spec/#literals) and it is optional. If the query does not specify the `unit`, the `unit` defaults to the `GROUP BY time()` interval. Note that this behavior is different from the [basic syntax’s](#basic-syntax-4) default behavior.`NON_NEGATIVE_DERIVATIVE()` returns only positive rates of change or rates of change that equal zero. `NON_NEGATIVE_DERIVATIVE()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples See the examples in the [`DERIVATIVE()` documentation](#advanced-syntax-8).`NON_NEGATIVE_DERIVATIVE()` behaves the same as the `DERIVATIVE()` function but `NON_NEGATIVE_DERIVATIVE()` returns only positive rates of change or rates of change that equal zero. ### NON\_NEGATIVE\_DIFFERENCE() Returns the non-negative result of subtraction between subsequent [field values](/influxdb/v1/concepts/glossary/#field-value). Non-negative results of subtraction include positive differences and differences that equal zero. #### Basic syntax ``` SELECT NON_NEGATIVE_DIFFERENCE( [ * | | // ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `NON_NEGATIVE_DIFFERENCE(field_key)` Returns the non-negative difference between subsequent field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `NON_NEGATIVE_DIFFERENCE(/regular_expression/)` Returns the non-negative difference between subsequent field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions). `NON_NEGATIVE_DIFFERENCE(*)` Returns the non-negative difference between subsequent field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `NON_NEGATIVE_DIFFERENCE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax-5) section for how to use `NON_NEGATIVE_DIFFERENCE()` with a `GROUP BY time()` clause. ##### Examples See the examples in the [`DIFFERENCE()` documentation](#basic-syntax-9).`NON_NEGATIVE_DIFFERENCE()` behaves the same as the `DIFFERENCE()` function but `NON_NEGATIVE_DIFFERENCE()` returns only positive differences or differences that equal zero. #### Advanced syntax ``` SELECT NON_NEGATIVE_DIFFERENCE(( [ * | | // ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `NON_NEGATIVE_DIFFERENCE()` function to those results. `NON_NEGATIVE_DIFFERENCE()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples See the examples in the [`DIFFERENCE()` documentation](#advanced-syntax-9).`NON_NEGATIVE_DIFFERENCE()` behaves the same as the `DIFFERENCE()` function but `NON_NEGATIVE_DIFFERENCE()` returns only positive differences or differences that equal zero. ### POW() Returns the field value to the power of `x`. #### Basic syntax ``` SELECT POW( [ * | ], ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `POW(field_key, x)` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key) to the power of `x`. `POW(*, x)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement) to the power of `x`. `POW()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `POW()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate field values associated with a field key to the power of 4 ```sql > SELECT POW("water_level", 4) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time pow ---- --- 2015-08-18T00:00:00Z 18.148417929216 2015-08-18T00:06:00Z 20.047612231936 2015-08-18T00:12:00Z 16.914992230656004 2015-08-18T00:18:00Z 20.429279055375993 2015-08-18T00:24:00Z 17.352898193760993 2015-08-18T00:30:00Z 17.69549197320101 ``` The query returns field values in the `water_level` field key in the `h2o_feet` measurement multiplied to a power of 4. ###### Calculate field values associated with each field key in a measurement to the power of 4 ```sql > SELECT POW(*, 4) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time pow_water_level ---- --------------- 2015-08-18T00:00:00Z 18.148417929216 2015-08-18T00:06:00Z 20.047612231936 2015-08-18T00:12:00Z 16.914992230656004 2015-08-18T00:18:00Z 20.429279055375993 2015-08-18T00:24:00Z 17.352898193760993 2015-08-18T00:30:00Z 17.69549197320101 ``` The query returns field values for each field key that stores numerical values in the `h2o_feet` measurement multiplied to the power of 4. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate field values associated with a field key to the power of 4 and include several clauses ```sql > SELECT POW("water_level", 4) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time pow ---- --- 2015-08-18T00:18:00Z 20.429279055375993 2015-08-18T00:12:00Z 16.914992230656004 2015-08-18T00:06:00Z 20.047612231936 2015-08-18T00:00:00Z 18.148417929216 ``` The query returns field values associated with the `water_level` field key multiplied to the power of 4. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT POW(( [ * | ] ), ) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `POW()` function to those results. `POW()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate mean values to the power of 4 ```sql > SELECT POW(MEAN("water_level"), 4) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time pow ---- --- 2015-08-18T00:00:00Z 19.08029760999999 2015-08-18T00:12:00Z 18.609983417041 2015-08-18T00:24:00Z 17.523567165456008 ``` The query returns [average](#mean) `water_level`s that are calculated at 12-minute intervals multiplied to the power of 4. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `POW()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates those averages multiplied to the power of 4. ### ROUND() Returns the subsequent value rounded to the nearest integer. #### Basic syntax ``` SELECT ROUND( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `ROUND(field_key)` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key) rounded to the nearest integer. `ROUND(*)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement) rounded to the nearest integer. `ROUND()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `ROUND()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Round field values associated with a field key ```sql > SELECT ROUND("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time round ---- ----- 2015-08-18T00:00:00Z 2 2015-08-18T00:06:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:18:00Z 2 2015-08-18T00:24:00Z 2 2015-08-18T00:30:00Z 2 ``` The query returns field values in the `water_level` field key in the `h2o_feet` measurement rounded to the nearest integer. ###### Round field values associated with each field key in a measurement ```sql > SELECT ROUND(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time round_water_level ---- ----------------- 2015-08-18T00:00:00Z 2 2015-08-18T00:06:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:18:00Z 2 2015-08-18T00:24:00Z 2 2015-08-18T00:30:00Z 2 ``` The query returns field values for each field key that stores numerical values in the `h2o_feet` measurement rounded to the nearest integer. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Round field values associated with a field key and include several clauses ```sql > SELECT ROUND("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time round ---- ----- 2015-08-18T00:18:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:06:00Z 2 2015-08-18T00:00:00Z 2 ``` The query returns field values associated with the `water_level` field key rounded to the nearest integer. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT ROUND(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `ROUND()` function to those results. `ROUND()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate mean values rounded to the nearest integer ```sql > SELECT ROUND(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time round ---- ----- 2015-08-18T00:00:00Z 2 2015-08-18T00:12:00Z 2 2015-08-18T00:24:00Z 2 ``` The query returns the [average](#mean) `water_level`s that are calculated at 12-minute intervals and rounds to the nearest integer. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `ROUND()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then rounds those averages to the nearest integer. ### SIN() Returns the sine of the field value. #### Basic syntax ``` SELECT SIN( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `SIN(field_key)` Returns the sine of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `SIN(*)` Returns the sine of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `SIN()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `SIN()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the sine of field values associated with a field key ```sql > SELECT SIN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time sin ---- --- 2015-08-18T00:00:00Z 0.8808206017241819 2015-08-18T00:06:00Z 0.8550216851706579 2015-08-18T00:12:00Z 0.8972904165810275 2015-08-18T00:18:00Z 0.8497930984115993 2015-08-18T00:24:00Z 0.8914760289023131 2015-08-18T00:30:00Z 0.8869008523376968 ``` The query returns sine of field values in the `water_level` field key in the `h2o_feet` measurement. ###### Calculate the sine of field values associated with each field key in a measurement ```sql > SELECT SIN(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time sin_water_level ---- --------------- 2015-08-18T00:00:00Z 0.8808206017241819 2015-08-18T00:06:00Z 0.8550216851706579 2015-08-18T00:12:00Z 0.8972904165810275 2015-08-18T00:18:00Z 0.8497930984115993 2015-08-18T00:24:00Z 0.8914760289023131 2015-08-18T00:30:00Z 0.8869008523376968 ``` The query returns sine of field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the sine of field values associated with a field key and include several clauses ```sql > SELECT SIN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time sin ---- --- 2015-08-18T00:18:00Z 0.8497930984115993 2015-08-18T00:12:00Z 0.8972904165810275 2015-08-18T00:06:00Z 0.8550216851706579 2015-08-18T00:00:00Z 0.8808206017241819 ``` The query returns sine of field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT SIN(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `SIN()` function to those results. `SIN()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the sine of mean values ```sql > SELECT SIN(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time sin ---- --- 2015-08-18T00:00:00Z 0.8682145834456126 2015-08-18T00:12:00Z 0.8745914945253902 2015-08-18T00:24:00Z 0.8891995555912935 ``` The query returns the sine of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `SIN()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates sine of those averages. ### SQRT() Returns the square root of field value. #### Basic syntax ``` SELECT SQRT( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `SQRT(field_key)` Returns the square root of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `SQRT(*)` Returns the square root field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `SQRT()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `SQRT()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the square root of field values associated with a field key ```sql > SELECT SQRT("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time sqrt ---- ---- 2015-08-18T00:00:00Z 1.4366627996854378 2015-08-18T00:06:00Z 1.4546477236774544 2015-08-18T00:12:00Z 1.4240786495134319 2015-08-18T00:18:00Z 1.4580809305384939 2015-08-18T00:24:00Z 1.4286357128393508 2015-08-18T00:30:00Z 1.4321312788986909 ``` The query returns the square roots of field values in the `water_level` field key in the `h2o_feet` measurement. ###### Calculate the square root of field values associated with each field key in a measurement ```sql > SELECT SQRT(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time sqrt_water_level ---- ---------------- 2015-08-18T00:00:00Z 1.4366627996854378 2015-08-18T00:06:00Z 1.4546477236774544 2015-08-18T00:12:00Z 1.4240786495134319 2015-08-18T00:18:00Z 1.4580809305384939 2015-08-18T00:24:00Z 1.4286357128393508 2015-08-18T00:30:00Z 1.4321312788986909 ``` The query returns the square roots of field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the square root of field values associated with a field key and include several clauses ```sql > SELECT SQRT("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time sqrt ---- ---- 2015-08-18T00:18:00Z 1.4580809305384939 2015-08-18T00:12:00Z 1.4240786495134319 2015-08-18T00:06:00Z 1.4546477236774544 2015-08-18T00:00:00Z 1.4366627996854378 ``` The query returns the square roots of field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT SQRT(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `SQRT()` function to those results. `SQRT()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the square root of mean values ```sql > SELECT SQRT(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time sqrt ---- ---- 2015-08-18T00:00:00Z 1.445683229480096 2015-08-18T00:12:00Z 1.4411800720243115 2015-08-18T00:24:00Z 1.430384563675098 ``` The query returns the square roots of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `SQRT()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates the square roots of those averages. ### TAN() Returns the tangent of the field value. #### Basic syntax ``` SELECT TAN( [ * | ] ) [INTO_clause] FROM_clause [WHERE_clause] [GROUP_BY_clause] [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `TAN(field_key)` Returns the tangent of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key). `TAN(*)` Returns the tangent of field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement). `TAN()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `TAN()` with a `GROUP BY time()` clause. ##### Examples The examples below use the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql > SELECT "water_level" FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time water_level ---- ----------- 2015-08-18T00:00:00Z 2.064 2015-08-18T00:06:00Z 2.116 2015-08-18T00:12:00Z 2.028 2015-08-18T00:18:00Z 2.126 2015-08-18T00:24:00Z 2.041 2015-08-18T00:30:00Z 2.051 ``` ###### Calculate the tangent of field values associated with a field key ```sql > SELECT TAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time tan ---- --- 2015-08-18T00:00:00Z -1.8604293534384375 2015-08-18T00:06:00Z -1.6487359603347427 2015-08-18T00:12:00Z -2.0326408012302273 2015-08-18T00:18:00Z -1.6121545688343464 2015-08-18T00:24:00Z -1.9676434782626282 2015-08-18T00:30:00Z -1.9198657720074992 ``` The query returns tangent of field values in the `water_level` field key in the `h2o_feet` measurement. ###### Calculate the tangent of field values associated with each field key in a measurement ```sql > SELECT TAN(*) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' name: h2o_feet time tan_water_level ---- --------------- 2015-08-18T00:00:00Z -1.8604293534384375 2015-08-18T00:06:00Z -1.6487359603347427 2015-08-18T00:12:00Z -2.0326408012302273 2015-08-18T00:18:00Z -1.6121545688343464 2015-08-18T00:24:00Z -1.9676434782626282 2015-08-18T00:30:00Z -1.9198657720074992 ``` The query returns tangent of field values for each field key that stores numerical values in the `h2o_feet` measurement. The `h2o_feet` measurement has one numerical field: `water_level`. ###### Calculate the tangent of field values associated with a field key and include several clauses ```sql > SELECT TAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' ORDER BY time DESC LIMIT 4 OFFSET 2 name: h2o_feet time tan ---- --- 2015-08-18T00:18:00Z -1.6121545688343464 2015-08-18T00:12:00Z -2.0326408012302273 2015-08-18T00:06:00Z -1.6487359603347427 2015-08-18T00:00:00Z -1.8604293534384375 ``` The query returns tangent of field values associated with the `water_level` field key. It covers the [time range](/influxdb/v1/query_language/explore-data/#time-syntax) between `2015-08-18T00:00:00Z` and `2015-08-18T00:30:00Z` and returns results in [descending timestamp order](/influxdb/v1/query_language/explore-data/#order-by-time-desc). The query also [limits](/influxdb/v1/query_language/explore-data/#the-limit-and-slimit-clauses) the number of points returned to four and [offsets](/influxdb/v1/query_language/explore-data/#the-offset-and-soffset-clauses) results by two points. #### Advanced syntax ``` SELECT TAN(( [ * | ] )) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` The advanced syntax requires a [`GROUP BY time() `clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) and a nested InfluxQL function. The query first calculates the results for the nested function at the specified `GROUP BY time()` interval and then applies the `TAN()` function to those results. `TAN()` supports the following nested functions:[`COUNT()`](#count),[`MEAN()`](#mean),[`MEDIAN()`](#median),[`MODE()`](#mode),[`SUM()`](#sum),[`FIRST()`](#first),[`LAST()`](#last),[`MIN()`](#min),[`MAX()`](#max), and[`PERCENTILE()`](#percentile). ##### Examples ###### Calculate the tangent of mean values ```sql > SELECT TAN(MEAN("water_level")) FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time tan ---- --- 2015-08-18T00:00:00Z -1.7497661902817365 2015-08-18T00:12:00Z -1.8038002062256624 2015-08-18T00:24:00Z -1.9435224805850773 ``` The query returns tangent of [average](#mean) `water_level`s that are calculated at 12-minute intervals. To get those results, InfluxDB first calculates the average `water_level`s at 12-minute intervals. This step is the same as using the `MEAN()` function with the `GROUP BY time()` clause and without `TAN()`: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:30:00Z' AND "location" = 'santa_monica' GROUP BY time(12m) name: h2o_feet time mean ---- ---- 2015-08-18T00:00:00Z 2.09 2015-08-18T00:12:00Z 2.077 2015-08-18T00:24:00Z 2.0460000000000003 ``` InfluxDB then calculates tangent of those averages. ## Predictors ### HOLT\_WINTERS() Returns N number of predicted [field values](/influxdb/v1/concepts/glossary/#field-value) using the[Holt-Winters](https://www.otexts.org/fpp/7/5) seasonal method. Use `HOLT_WINTERS()` to: * Predict when data values will cross a given threshold * Compare predicted values with actual values to detect anomalies in your data #### Syntax ``` SELECT HOLT_WINTERS[_WITH-FIT]((),,) [INTO_clause] FROM_clause [WHERE_clause] GROUP_BY_clause [ORDER_BY_clause] [LIMIT_clause] [OFFSET_clause] [SLIMIT_clause] [SOFFSET_clause] ``` `HOLT_WINTERS(function(field_key),N,S)` returns `N` seasonally adjusted predicted field values for the specified [field key](/influxdb/v1/concepts/glossary/#field-key). The `N` predicted values occur at the same interval as the [`GROUP BY time()` interval](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). If your `GROUP BY time()` interval is `6m` and `N` is `3` you’ll receive three predicted values that are each six minutes apart. `S` is the seasonal pattern parameter and delimits the length of a seasonal pattern according to the `GROUP BY time()` interval. If your `GROUP BY time()` interval is `2m` and `S` is `3`, then the seasonal pattern occurs every six minutes, that is, every three data points. If you do not want to seasonally adjust your predicted values, set `S` to `0`or `1.` `HOLT_WINTERS_WITH_FIT(function(field_key),N,S)` returns the fitted values in addition to `N` seasonally adjusted predicted field values for the specified field key. `HOLT_WINTERS()` and `HOLT_WINTERS_WITH_FIT()` work with data that occur at consistent time intervals; the nested InfluxQL function and the`GROUP BY time()` clause ensure that the Holt-Winters functions operate on regular data. `HOLT_WINTERS()` and `HOLT_WINTERS_WITH_FIT()` support int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). #### Examples ##### Predict field values associated with a field key ###### Raw Data The example uses [Chronograf](https://github.com/influxdata/chronograf) to visualize the data. The example focuses on the following subsample of the [`NOAA_water_database` data](/influxdb/v1/query_language/data_download/): ```sql SELECT "water_level" FROM "NOAA_water_database"."autogen"."h2o_feet" WHERE "location"='santa_monica' AND time >= '2015-08-22 22:12:00' AND time <= '2015-08-28 03:00:00' ``` ![Raw Data](/img/influxdb/1-3-hw-raw-data-1-2.png) ###### Step 1: Match the Trends of the Raw Data Write a `GROUP BY time()` query that matches the general trends of the raw `water_level` data. Here, we use the [`FIRST()`](#first) function: ```sql SELECT FIRST("water_level") FROM "NOAA_water_database"."autogen"."h2o_feet" WHERE "location"='santa_monica' and time >= '2015-08-22 22:12:00' and time <= '2015-08-28 03:00:00' GROUP BY time(379m,348m) ``` In the `GROUP BY time()` clause, the first argument (`379m`) matches the length of time that occurs between each peak and trough in the `water_level` data. The second argument (`348m`) is the[offset interval](/influxdb/v1/query_language/explore-data/#advanced-group-by-time-syntax). The offset interval alters the default `GROUP BY time()` boundaries to match the time range of the raw data. The blue line shows the results of the query: ![First step](/img/influxdb/1-3-hw-first-step-1-2.png) ###### Step 2: Determine the Seasonal Pattern Identify the seasonal pattern in the data using the information from the query in step 1. Focusing on the blue line in the graph below, the pattern in the `water_level` data repeats about every 25 hours and 15 minutes. There are four data points per season, so `4` is the seasonal pattern argument. ![Second step](/img/influxdb/1-3-hw-second-step-1-2.png) ###### Step 3: Apply the HOLT\_WINTERS() function Add the Holt-Winters function to the query. Here, we use `HOLT_WINTERS_WITH_FIT()` to view both the fitted values and the predicted values: ```sql SELECT HOLT_WINTERS_WITH_FIT(FIRST("water_level"),10,4) FROM "NOAA_water_database"."autogen"."h2o_feet" WHERE "location"='santa_monica' AND time >= '2015-08-22 22:12:00' AND time <= '2015-08-28 03:00:00' GROUP BY time(379m,348m) ``` In the `HOLT_WINTERS_WITH_FIT()` function, the first argument (`10`) requests 10 predicted field values. Each predicted point is `379m` apart, the same interval as the first argument in the `GROUP BY time()` clause. The second argument in the `HOLT_WINTERS_WITH_FIT()` function (`4`) is the seasonal pattern that we determined in the previous step. The blue line shows the results of the query: ![Third step](/img/influxdb/1-3-hw-third-step-1-2.png) #### Common Issues with `HOLT_WINTERS()` ##### `HOLT_WINTERS()` and receiving fewer than `N` points In some cases, users may receive fewer predicted points than requested by the `N` parameter. That behavior occurs when the math becomes unstable and cannot forecast more points. It implies that either `HOLT_WINTERS()` is not suited for the dataset or that the seasonal adjustment parameter is invalid and is confusing the algorithm. ## Technical Analysis The following technical analysis functions apply widely used algorithms to your data. While they are primarily used in the world of finance and investing, they have application in other industries and use cases as well. [CHANDE\_MOMENTUM\_OSCILLATOR()](#chande_momentum_oscillator) [EXPONENTIAL\_MOVING\_AVERAGE()](#exponential_moving_average) [DOUBLE\_EXPONENTIAL\_MOVING\_AVERAGE()](#double_exponential_moving_average) [KAUFMANS\_EFFICIENCY\_RATIO()](#kaufmans_efficiency_ratio) [KAUFMANS\_ADAPTIVE\_MOVING\_AVERAGE()](#kaufmans_adaptive_moving_average) [TRIPLE\_EXPONENTIAL\_MOVING\_AVERAGE()](#triple_exponential_moving_average) [TRIPLE\_EXPONENTIAL\_DERIVATIVE()](#triple_exponential_derivative) [RELATIVE\_STRENGTH\_INDEX()](#relative_strength_index) ### Arguments Along with a [field key](/influxdb/v1/concepts/glossary/#field-key), technical analysis function accept the following arguments: #### `PERIOD` **Required, integer, min=1** The sample size of the algorithm. This is essentially the number of historical samples which have any significant effect on the output of the algorithm. E.G. `2` means the current point and the point before it. The algorithm uses an exponential decay rate to determine the weight of a historical point, generally known as the alpha (α). The `PERIOD` controls the decay rate. > > > NOTE: Older points can still have an impact. > > #### `HOLD_PERIOD` **integer, min=-1** How many samples the algorithm needs before it will start emitting results. The default of `-1` means the value is based on the algorithm, the `PERIOD`, and the `WARMUP_TYPE`, but is a value in which the algorithm can emit meaningful results. ***Default Hold Periods:*** For most of the available technical analysis, the default `HOLD_PERIOD` is determined by which technical analysis algorithm you’re using and the [`WARMUP_TYPE`](#warmup_type) | Algorithm \\ Warmup Type | simple |exponential| none | |--------------------------------------------------------------------------|-----------------------|-----------|----------| | [EXPONENTIAL\_MOVING\_AVERAGE](#exponential_moving_average) | PERIOD - 1 |PERIOD - 1 | n/a | |[DOUBLE\_EXPONENTIAL\_MOVING\_AVERAGE](#double_exponential_moving_average)| ( PERIOD - 1 ) \* 2 |PERIOD - 1 | n/a | |[TRIPLE\_EXPONENTIAL\_MOVING\_AVERAGE](#triple_exponential_moving_average)| ( PERIOD - 1 ) \* 3 |PERIOD - 1 | n/a | | [TRIPLE\_EXPONENTIAL\_DERIVATIVE](#triple_exponential_derivative) |( PERIOD - 1 ) \* 3 + 1| PERIOD | n/a | | [RELATIVE\_STRENGTH\_INDEX](#relative_strength_index) | PERIOD | PERIOD | n/a | | [CHANDE\_MOMENTUM\_OSCILLATOR](#chande_momentum_oscillator) | PERIOD | PERIOD |PERIOD - 1| ***Kaufman Algorithm Default Hold Periods:*** | Algorithm |Default Hold Period| |--------------------------------------------------------------------------|-------------------| | [KAUFMANS\_EFFICIENCY\_RATIO()](#kaufmans_efficiency_ratio) | PERIOD | |[KAUFMANS\_ADAPTIVE\_MOVING\_AVERAGE()](#kaufmans_adaptive_moving_average)| PERIOD | #### `WARMUP_TYPE` **default=‘exponential’** This controls how the algorithm initializes itself for the first `PERIOD` samples. It is essentially the duration for which it has an incomplete sample set. `simple` Simple moving average (SMA) of the first `PERIOD` samples. This is the method used by [ta-lib](https://www.ta-lib.org/). `exponential` Exponential moving average (EMA) with scaling alpha (α). This basically uses an EMA with `PERIOD=1` for the first point, `PERIOD=2`for the second point, etc., until algorithm has consumed `PERIOD` number of points. As the algorithm immediately starts using an EMA, when this method is used and`HOLD_PERIOD` is unspecified or `-1`, the algorithm may start emitting points after a much smaller sample size than with `simple`. `none` The algorithm does not perform any smoothing at all. This is the method used by [ta-lib](https://www.ta-lib.org/). When this method is used and `HOLD_PERIOD` is unspecified, `HOLD_PERIOD`defaults to `PERIOD - 1`. > > > The `none` warmup type is only available with the[`CHANDE_MOMENTUM_OSCILLATOR()`](#chande_momentum_oscillator) function. > > ### CHANDE\_MOMENTUM\_OSCILLATOR() The Chande Momentum Oscillator (CMO) is a technical momentum indicator developed by Tushar Chande. The CMO indicator is created by calculating the difference between the sum of all recent higher data points and the sum of all recent lower data points, then dividing the result by the sum of all data movement over a given time period. The result is multiplied by 100 to give the -100 to +100 range.Source #### Basic syntax ``` CHANDE_MOMENTUM_OSCILLATOR([ * | | /regular_expression/ ], [, , [warmup_type]]) ``` **Available Arguments:** [period](#period) [hold\_period](#hold_period) (Optional) [warmup\_type](#warmup_type) (Optional) `CHANDE_MOMENTUM_OSCILLATOR(field_key, 2)` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Chande Momentum Oscillator algorithm with a 2-value period and the default hold period and warmup type. `CHANDE_MOMENTUM_OSCILLATOR(field_key, 10, 9, 'none')` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Chande Momentum Oscillator algorithm with a 10-value period a 9-value hold period, and the `none` warmup type. `CHANDE_MOMENTUM_OSCILLATOR(MEAN(), 2) ... GROUP BY time(1d)` Returns the mean of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Chande Momentum Oscillator algorithm with a 2-value period and the default hold period and warmup type. > > > **Note:** When aggregating data with a `GROUP BY` clause, you must include an[aggregate function](#aggregations) in your call to the `CHANDE_MOMENTUM_OSCILLATOR()` function. > > `CHANDE_MOMENTUM_OSCILLATOR(/regular_expression/, 2)` Returns the field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions)processed using the Chande Momentum Oscillator algorithm with a 2-value period and the default hold period and warmup type. `CHANDE_MOMENTUM_OSCILLATOR(*, 2)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement)processed using the Chande Momentum Oscillator algorithm with a 2-value period and the default hold period and warmup type. `CHANDE_MOMENTUM_OSCILLATOR()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `CHANDE_MOMENTUM_OSCILLATOR()` with a `GROUP BY time()` clause. ### EXPONENTIAL\_MOVING\_AVERAGE() An exponential moving average (EMA) is a type of moving average that is similar to a [simple moving average](#moving_average), except that more weight is given to the latest data. It’s also known as the “exponentially weighted moving average.” This type of moving average reacts faster to recent data changes than a simple moving average. Source #### Basic syntax ``` EXPONENTIAL_MOVING_AVERAGE([ * | | /regular_expression/ ], [, ), 2) ... GROUP BY time(1d)` Returns the mean of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Exponential Moving Average algorithm with a 2-value period and the default hold period and warmup type. > > > **Note:** When aggregating data with a `GROUP BY` clause, you must include an[aggregate function](#aggregations) in your call to the `EXPONENTIAL_MOVING_AVERAGE()` function. > > `EXPONENTIAL_MOVING_AVERAGE(/regular_expression/, 2)` Returns the field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions)processed using the Exponential Moving Average algorithm with a 2-value period and the default hold period and warmup type. `EXPONENTIAL_MOVING_AVERAGE(*, 2)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement)processed using the Exponential Moving Average algorithm with a 2-value period and the default hold period and warmup type. `EXPONENTIAL_MOVING_AVERAGE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `EXPONENTIAL_MOVING_AVERAGE()` with a `GROUP BY time()` clause. ### DOUBLE\_EXPONENTIAL\_MOVING\_AVERAGE() The Double Exponential Moving Average (DEMA) attempts to remove the inherent lag associated to Moving Averages by placing more weight on recent values. The name suggests this is achieved by applying a double exponential smoothing which is not the case. The name double comes from the fact that the value of an [EMA](#exponential_moving_average) is doubled. To keep it in line with the actual data and to remove the lag, the value “EMA of EMA” is subtracted from the previously doubled EMA. Source #### Basic syntax ``` DOUBLE_EXPONENTIAL_MOVING_AVERAGE([ * | | /regular_expression/ ], [, ), 2) ... GROUP BY time(1d)` Returns the mean of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Double Exponential Moving Average algorithm with a 2-value period and the default hold period and warmup type. > > > **Note:** When aggregating data with a `GROUP BY` clause, you must include an[aggregate function](#aggregations) in your call to the `DOUBLE_EXPONENTIAL_MOVING_AVERAGE()` function. > > `DOUBLE_EXPONENTIAL_MOVING_AVERAGE(/regular_expression/, 2)` Returns the field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions)processed using the Double Exponential Moving Average algorithm with a 2-value period and the default hold period and warmup type. `DOUBLE_EXPONENTIAL_MOVING_AVERAGE(*, 2)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement)processed using the Double Exponential Moving Average algorithm with a 2-value period and the default hold period and warmup type. `DOUBLE_EXPONENTIAL_MOVING_AVERAGE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `DOUBLE_EXPONENTIAL_MOVING_AVERAGE()` with a `GROUP BY time()` clause. ### KAUFMANS\_EFFICIENCY\_RATIO() Kaufman’s Efficiency Ration, or simply “Efficiency Ratio” (ER), is calculated by dividing the data change over a period by the absolute sum of the data movements that occurred to achieve that change. The resulting ratio ranges between 0 and 1 with higher values representing a more efficient or trending market. The ER is very similar to the [Chande Momentum Oscillator](#chande-momentum-oscillator) (CMO). The difference is that the CMO takes market direction into account, but if you take the absolute CMO and divide by 100, you you get the Efficiency Ratio. Source #### Basic syntax ``` KAUFMANS_EFFICIENCY_RATIO([ * | | /regular_expression/ ], [, ]) ``` **Available Arguments:** [period](#period) [hold\_period](#hold_period) (Optional) `KAUFMANS_EFFICIENCY_RATIO(field_key, 2)` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Efficiency Index algorithm with a 2-value period and the default hold period and warmup type. `KAUFMANS_EFFICIENCY_RATIO(field_key, 10, 10)` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Efficiency Index algorithm with a 10-value period and a 10-value hold period. `KAUFMANS_EFFICIENCY_RATIO(MEAN(), 2) ... GROUP BY time(1d)` Returns the mean of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Efficiency Index algorithm with a 2-value period and the default hold period. > > > **Note:** When aggregating data with a `GROUP BY` clause, you must include an[aggregate function](#aggregations) in your call to the `KAUFMANS_EFFICIENCY_RATIO()` function. > > `KAUFMANS_EFFICIENCY_RATIO(/regular_expression/, 2)` Returns the field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions)processed using the Efficiency Index algorithm with a 2-value period and the default hold period and warmup type. `KAUFMANS_EFFICIENCY_RATIO(*, 2)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement)processed using the Efficiency Index algorithm with a 2-value period and the default hold period and warmup type. `KAUFMANS_EFFICIENCY_RATIO()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `KAUFMANS_EFFICIENCY_RATIO()` with a `GROUP BY time()` clause. ### KAUFMANS\_ADAPTIVE\_MOVING\_AVERAGE() Kaufman’s Adaptive Moving Average (KAMA) is a moving average designed to account for sample noise or volatility. KAMA will closely follow data points when the data swings are relatively small and noise is low. KAMA will adjust when the data swings widen and follow data from a greater distance. This trend-following indicator can be used to identify the overall trend, time turning points and filter data movements. Source #### Basic syntax ``` KAUFMANS_ADAPTIVE_MOVING_AVERAGE([ * | | /regular_expression/ ], [, ]) ``` **Available Arguments:** [period](#period) [hold\_period](#hold_period) (Optional) `KAUFMANS_ADAPTIVE_MOVING_AVERAGE(field_key, 2)` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Kaufman Adaptive Moving Average algorithm with a 2-value period and the default hold period and warmup type. `KAUFMANS_ADAPTIVE_MOVING_AVERAGE(field_key, 10, 10)` Returns the field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Kaufman Adaptive Moving Average algorithm with a 10-value period and a 10-value hold period. `KAUFMANS_ADAPTIVE_MOVING_AVERAGE(MEAN(), 2) ... GROUP BY time(1d)` Returns the mean of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Kaufman Adaptive Moving Average algorithm with a 2-value period and the default hold period. > > > **Note:** When aggregating data with a `GROUP BY` clause, you must include an[aggregate function](#aggregations) in your call to the `KAUFMANS_ADAPTIVE_MOVING_AVERAGE()` function. > > `KAUFMANS_ADAPTIVE_MOVING_AVERAGE(/regular_expression/, 2)` Returns the field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions)processed using the Kaufman Adaptive Moving Average algorithm with a 2-value period and the default hold period and warmup type. `KAUFMANS_ADAPTIVE_MOVING_AVERAGE(*, 2)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement)processed using the Kaufman Adaptive Moving Average algorithm with a 2-value period and the default hold period and warmup type. `KAUFMANS_ADAPTIVE_MOVING_AVERAGE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `KAUFMANS_ADAPTIVE_MOVING_AVERAGE()` with a `GROUP BY time()` clause. ### TRIPLE\_EXPONENTIAL\_MOVING\_AVERAGE() The triple exponential moving average (TEMA) was developed to filter out volatility from conventional moving averages. While the name implies that it’s a triple exponential smoothing, it’s actually a composite of a [single exponential moving average](#exponential_moving_average), a [double exponential moving average](#double_exponential_moving_average), and a triple exponential moving average. Source #### Basic syntax ``` TRIPLE_EXPONENTIAL_MOVING_AVERAGE([ * | | /regular_expression/ ], [, ), 2) ... GROUP BY time(1d)` Returns the mean of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Triple Exponential Moving Average algorithm with a 2-value period and the default hold period and warmup type. > > > **Note:** When aggregating data with a `GROUP BY` clause, you must include an[aggregate function](#aggregations) in your call to the `TRIPLE_EXPONENTIAL_MOVING_AVERAGE()` function. > > `TRIPLE_EXPONENTIAL_MOVING_AVERAGE(/regular_expression/, 2)` Returns the field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions)processed using the Triple Exponential Moving Average algorithm with a 2-value period and the default hold period and warmup type. `TRIPLE_EXPONENTIAL_MOVING_AVERAGE(*, 2)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement)processed using the Triple Exponential Moving Average algorithm with a 2-value period and the default hold period and warmup type. `TRIPLE_EXPONENTIAL_MOVING_AVERAGE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `TRIPLE_EXPONENTIAL_MOVING_AVERAGE()` with a `GROUP BY time()` clause. ### TRIPLE\_EXPONENTIAL\_DERIVATIVE() The triple exponential derivative indicator, commonly referred to as “TRIX,” is an oscillator used to identify oversold and overbought markets, and can also be used as a momentum indicator. TRIX calculates a [triple exponential moving average](#triple-exponential-moving-average)of the [log](#log) of the data input over the period of time. The previous value is subtracted from the previous value. This prevents cycles that are shorter than the defined period from being considered by the indicator. Like many oscillators, TRIX oscillates around a zero line. When used as an oscillator, a positive value indicates an overbought market while a negative value indicates an oversold market. When used as a momentum indicator, a positive value suggests momentum is increasing while a negative value suggests momentum is decreasing. Many analysts believe that when the TRIX crosses above the zero line it gives a buy signal, and when it closes below the zero line, it gives a sell signal. Source #### Basic syntax ``` TRIPLE_EXPONENTIAL_DERIVATIVE([ * | | /regular_expression/ ], [, ), 2) ... GROUP BY time(1d)` Returns the mean of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Triple Exponential Derivative algorithm with a 2-value period and the default hold period and warmup type. > > > **Note:** When aggregating data with a `GROUP BY` clause, you must include an[aggregate function](#aggregations) in your call to the `TRIPLE_EXPONENTIAL_DERIVATIVE()` function. > > `TRIPLE_EXPONENTIAL_DERIVATIVE(/regular_expression/, 2)` Returns the field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions)processed using the Triple Exponential Derivative algorithm with a 2-value period and the default hold period and warmup type. `TRIPLE_EXPONENTIAL_DERIVATIVE(*, 2)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement)processed using the Triple Exponential Derivative algorithm with a 2-value period and the default hold period and warmup type. `TRIPLE_EXPONENTIAL_DERIVATIVE()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `TRIPLE_EXPONENTIAL_DERIVATIVE()` with a `GROUP BY time()` clause. ### RELATIVE\_STRENGTH\_INDEX() The relative strength index (RSI) is a momentum indicator that compares the magnitude of recent increases and decreases over a specified time period to measure speed and change of data movements. Source #### Basic syntax ``` RELATIVE_STRENGTH_INDEX([ * | | /regular_expression/ ], [, ), 2) ... GROUP BY time(1d)` Returns the mean of field values associated with the [field key](/influxdb/v1/concepts/glossary/#field-key)processed using the Relative Strength Index algorithm with a 2-value period and the default hold period and warmup type. > > > **Note:** When aggregating data with a `GROUP BY` clause, you must include an[aggregate function](#aggregations) in your call to the `RELATIVE_STRENGTH_INDEX()` function. > > `RELATIVE_STRENGTH_INDEX(/regular_expression/, 2)` Returns the field values associated with each field key that matches the [regular expression](/influxdb/v1/query_language/explore-data/#regular-expressions)processed using the Relative Strength Index algorithm with a 2-value period and the default hold period and warmup type. `RELATIVE_STRENGTH_INDEX(*, 2)` Returns the field values associated with each field key in the [measurement](/influxdb/v1/concepts/glossary/#measurement)processed using the Relative Strength Index algorithm with a 2-value period and the default hold period and warmup type. `RELATIVE_STRENGTH_INDEX()` supports int64 and float64 field value [data types](/influxdb/v1/write_protocols/line_protocol_reference/#data-types). The basic syntax supports `GROUP BY` clauses that [group by tags](/influxdb/v1/query_language/explore-data/#group-by-tags) but not `GROUP BY` clauses that [group by time](/influxdb/v1/query_language/explore-data/#group-by-time-intervals). See the [Advanced Syntax](#advanced-syntax) section for how to use `RELATIVE_STRENGTH_INDEX()` with a `GROUP BY time()` clause. ## Other ### Sample Data The data used in this document are available for download on the [Sample Data](/influxdb/v1/query_language/data_download/) page. ### General Syntax for Functions #### Specify Multiple Functions in the SELECT Clause ##### Syntax ``` SELECT (),() FROM_clause [...] ``` Separate multiple functions in one `SELECT` statement with a comma (`,`). The syntax applies to all InfluxQL functions except [`TOP()`](#top) and [`BOTTOM()`](#bottom). The `SELECT` clause does not support specifying `TOP()` or `BOTTOM()` with another function. ##### Examples ###### Calculate the mean and median field values in one query ```sql > SELECT MEAN("water_level"),MEDIAN("water_level") FROM "h2o_feet" name: h2o_feet time mean median ---- ---- ------ 1970-01-01T00:00:00Z 4.442107025822522 4.124 ``` The query returns the [average](#mean) and [median](#median) field values in the `water_level` field key. ###### Calculate the mode of two fields in one query ```sql > SELECT MODE("water_level"),MODE("level description") FROM "h2o_feet" name: h2o_feet time mode mode_1 ---- ---- ------ 1970-01-01T00:00:00Z 2.69 between 3 and 6 feet ``` The query returns the [mode](#mode) field values for the `water_level` field key and for the `level description` field key. The `water_level` mode is in the `mode` column and the `level description` mode is in the `mode_1` column. The system can’t return more than one column with the same name so it renames the second `mode` column to `mode_1`. See [Rename the Output Field Key](#rename_the_output_field_key) for how to configure the output column headers. ###### Calculate the minimum and maximum field values in one query ```sql > SELECT MIN("water_level"), MAX("water_level") [...] name: h2o_feet time min max ---- --- --- 1970-01-01T00:00:00Z -0.61 9.964 ``` The query returns the [minimum](#min) and [maximum](#max) field values in the `water_level` field key. Notice that the query returns `1970-01-01T00:00:00Z`, the InfluxDB equivalent to a null timestamp, as the timestamp value.`MIN()` and `MAX()` are [selector](#selectors) functions; when a selector function is the only function in the `SELECT` clause, it returns a specific timestamp. Because `MIN()` and `MAX()` return two different timestamps (see below), the system overrides those timestamps with the null timestamp equivalent. ```sql > SELECT MIN("water_level") FROM "h2o_feet" name: h2o_feet time min ---- --- 2015-08-29T14:30:00Z -0.61 <--- Timestamp 1 > SELECT MAX("water_level") FROM "h2o_feet" name: h2o_feet time max ---- --- 2015-08-29T07:24:00Z 9.964 <--- Timestamp 2 ``` #### Rename the Output Field Key ##### Syntax ``` SELECT () AS [...] ``` By default, functions return results under a field key that matches the function name. Include an `AS` clause to specify the name of the output field key. ##### Examples ###### Specify the output field key ```sql > SELECT MEAN("water_level") AS "dream_name" FROM "h2o_feet" name: h2o_feet time dream_name ---- ---------- 1970-01-01T00:00:00Z 4.442107025822522 ``` The query returns the [average](#mean) field value of the `water_level` field key and renames the output field key to `dream_name`. Without the `AS` clause, the query returns `mean` as the output field key: ```sql > SELECT MEAN("water_level") FROM "h2o_feet" name: h2o_feet time mean ---- ---- 1970-01-01T00:00:00Z 4.442107025822522 ``` ###### Specify the output field key for multiple functions ```sql > SELECT MEDIAN("water_level") AS "med_wat",MODE("water_level") AS "mode_wat" FROM "h2o_feet" name: h2o_feet time med_wat mode_wat ---- ------- -------- 1970-01-01T00:00:00Z 4.124 2.69 ``` The query returns the [median](#median) and [mode](#mode) field values for the `water_level` field key and renames the output field keys to `med_wat` and `mode_wat`. Without the `AS` clauses, the query returns `median` and `mode` as the output field keys: ```sql > SELECT MEDIAN("water_level"),MODE("water_level") FROM "h2o_feet" name: h2o_feet time median mode ---- ------ ---- 1970-01-01T00:00:00Z 4.124 2.69 ``` #### Change the Values Reported for Intervals with no Data By default, queries with an InfluxQL function and a [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) report null values for intervals with no data. Include `fill()` at the end of the `GROUP BY` clause to change that value. See [Data Exploration](/influxdb/v1/query_language/explore-data/#group-by-time-intervals-and-fill) for a complete discussion of `fill()`. ### Common Issues with Functions The following sections describe frequent sources of confusion with all functions, aggregation functions, and selector functions. See the function-specific documentation for common issues with individual functions: * [DISTINCT()](#common-issues-with-distinct) * [BOTTOM()](#common-issues-with-bottom) * [PERCENTILE()](#common-issues-with-percentile) * [SAMPLE()](#common-issues-with-sample) * [TOP()](#common-issues-with-top) * [ELAPSED()](#common-issues-with-elapsed) * [HOLT\_WINTERS()](#common-issues-with-holt-winters) #### All Functions ##### Nesting functions Some InfluxQL functions support nesting in the [`SELECT` clause](/influxdb/v1/query_language/explore-data/#select-clause): * [`COUNT()`](#count) with [`DISTINCT()`](#distinct) * [`CUMULATIVE_SUM()`](#cumulative_sum) * [`DERIVATIVE()`](#derivative) * [`DIFFERENCE()`](#difference) * [`ELAPSED()`](#elapsed) * [`MOVING_AVERAGE()`](#moving_average) * [`NON_NEGATIVE_DERIVATIVE()`](#non_negative_derivative) * [`HOLT_WINTERS()`](#holt_winters) and [`HOLT_WINTERS_WITH_FIT()`](#holt_winters) For other functions, use InfluxQL’s [subqueries](/influxdb/v1/query_language/explore-data/#subqueries) to nest functions in the [`FROM` clause](/influxdb/v1/query_language/explore-data/#from-clause). See the [Data Exploration](/influxdb/v1/query_language/explore-data/#subqueries) page more on using subqueries. ##### Querying time ranges after now() Most `SELECT` statements have a default time range between [`1677-09-21 00:12:43.145224194` and `2262-04-11T23:47:16.854775806Z` UTC](/influxdb/v1/troubleshooting/frequently-asked-questions/#what-are-the-minimum-and-maximum-timestamps-that-influxdb-can-store). For `SELECT` statements with an InfluxQL function and a [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals), the default time range is between `1677-09-21 00:12:43.145224194` UTC and [`now()`](/influxdb/v1/concepts/glossary/#now). To query data with timestamps that occur after `now()`, `SELECT` statements with an InfluxQL function and a `GROUP BY time()` clause must provide an alternative upper bound in the[`WHERE` clause](/influxdb/v1/query_language/explore-data/#the-where-clause). See the [Frequently Asked Questions](/influxdb/v1/troubleshooting/frequently-asked-questions/#why-don-t-my-group-by-time-queries-return-timestamps-that-occur-after-now) page for an example. #### Aggregation Functions ##### Understanding the returned timestamp A query with an [aggregation function](#aggregations) and no time range in the [`WHERE` clause](/influxdb/v1/query_language/explore-data/#the-where-clause) returns epoch 0 (`1970-01-01T00:00:00Z`) as the timestamp. InfluxDB uses epoch 0 as the null timestamp equivalent. A query with an aggregate function that includes a time range in the `WHERE` clause returns the lower time bound as the timestamp. ##### Examples ###### Use an aggregate function without a specified time range ```sql > SELECT SUM("water_level") FROM "h2o_feet" name: h2o_feet time sum ---- --- 1970-01-01T00:00:00Z 67777.66900000004 ``` The query returns the InfluxDB equivalent of a null timestamp (epoch 0: `1970-01-01T00:00:00Z`) as the timestamp.[`SUM()`](#sum) aggregates points across several timestamps and has no single timestamp to return. ###### Use an aggregate function with a specified time range ```sql > SELECT SUM("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' name: h2o_feet time sum ---- --- 2015-08-18T00:00:00Z 67777.66900000004 ``` The query returns the lower time bound (`WHERE time >= '2015-08-18T00:00:00Z'`) as the timestamp. ###### Use an aggregate function with a specified time range and a GROUP BY time() clause ```sql > SELECT SUM("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:18:00Z' GROUP BY time(12m) name: h2o_feet time sum ---- --- 2015-08-18T00:00:00Z 20.305 2015-08-18T00:12:00Z 19.802999999999997 ``` The query returns the lower time bound for each [`GROUP BY time()`](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) interval as the timestamps. ###### Mixing aggregation functions with non-aggregates Aggregation functions do not support specifying standalone [field keys](/influxdb/v1/concepts/glossary/#field-key) or [tag keys](/influxdb/v1/concepts/glossary/#tag-key) in the [`SELECT` clause](/influxdb/v1/query_language/explore-data/#select-clause). Aggregation functions return a single calculated value and there is no obvious single value to return for any unaggregated fields or tags. Including a standalone field key or tag key with an aggregation function in the `SELECT` clause returns an error: ```sql > SELECT SUM("water_level"),"location" FROM "h2o_feet" ERR: error parsing query: mixing aggregate and non-aggregate queries is not supported ``` ##### Getting slightly different results For some aggregation functions, executing the same function on the same set of [float64](/influxdb/v1/write_protocols/line_protocol_reference/#data-types) points may yield slightly different results. InfluxDB does not sort points before it applies the aggregation function; that behavior can cause small discrepancies in the query results. #### Selector Functions ##### Understanding the returned timestamp The timestamps returned by [selector functions](#selectors) depend on the number of functions in the query and on the other clauses in the query: A query with a single selector function, a single [field key](/influxdb/v1/concepts/glossary/#field-key) argument, and no [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) returns the timestamp for the point that appears in the raw data. A query with a single selector function, multiple field key arguments, and no [`GROUP BY time()` clause](/influxdb/v1/query_language/explore-data/#group-by-time-intervals) returns the timestamp for the point that appears in the raw data or the InfluxDB equivalent of a null timestamp (epoch 0: `1970-01-01T00:00:00Z`). A query with more than one function and no time range in the [`WHERE` clause](/influxdb/v1/query_language/explore-data/#the-where-clause) returns the InfluxDB equivalent of a null timestamp (epoch 0: `1970-01-01T00:00:00Z`). A query with more than one function and a time range in the `WHERE` clause returns the lower time bound as the timestamp. A query with a selector function and a `GROUP BY time()` clause returns the lower time bound for each `GROUP BY time()` interval. Note that the `SAMPLE()` function behaves differently from other selector functions when paired with the `GROUP BY time()` clause. See [Common Issues with `SAMPLE()`](#common-issues-with-sample) for more information. ##### Examples ###### Use a single selector function with a single field key and without a specified time range ```sql > SELECT MAX("water_level") FROM "h2o_feet" name: h2o_feet time max ---- --- 2015-08-29T07:24:00Z 9.964 > SELECT MAX("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' name: h2o_feet time max ---- --- 2015-08-29T07:24:00Z 9.964 ``` The queries return the timestamp for the [maximum](#max) point that appears in the raw data. ###### Use a single selector function with multiple field keys and without a specified time range ```sql > SELECT FIRST(*) FROM "h2o_feet" name: h2o_feet time first_level description first_water_level ---- ----------------------- ----------------- 1970-01-01T00:00:00Z between 6 and 9 feet 8.12 > SELECT MAX(*) FROM "h2o_feet" name: h2o_feet time max_water_level ---- --------------- 2015-08-29T07:24:00Z 9.964 ``` The first query returns the InfluxDB equivalent of a null timestamp (epoch 0: `1970-01-01T00:00:00Z`) as the timestamp.`FIRST(*)` returns two timestamps (one for each field key in the `h2o_feet` [measurement](/influxdb/v1/concepts/glossary/#measurement)) so the system overrides those timestamps with the null timestamp equivalent. The second query returns the timestamp for the maximum point that appears in the raw data.`MAX(*)` returns one timestamp (the `h2o-feet` measurement has only one numerical field) so the system does not overwrite the original timestamp. ###### Use a selector function with another function and without a specified time range ```sql > SELECT MAX("water_level"),MIN("water_level") FROM "h2o_feet" name: h2o_feet time max min ---- --- --- 1970-01-01T00:00:00Z 9.964 -0.61 ``` The query returns the InfluxDB equivalent of a null timestamp (epoch 0: `1970-01-01T00:00:00Z`) as the timestamp. The `MAX()` and [`MIN()`](#min) functions return different timestamps so the system has no single timestamp to return. ###### Use a selector function with another function and with a specified time range ```sql > SELECT MAX("water_level"),MIN("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' name: h2o_feet time max min ---- --- --- 2015-08-18T00:00:00Z 9.964 -0.61 ``` The query returns the lower time bound (`WHERE time >= '2015-08-18T00:00:00Z'`) as the timestamp. ###### Use a selector function with a GROUP BY time() clause ```sql > SELECT MAX("water_level") FROM "h2o_feet" WHERE time >= '2015-08-18T00:00:00Z' AND time <= '2015-08-18T00:18:00Z' GROUP BY time(12m) name: h2o_feet time max ---- --- 2015-08-18T00:00:00Z 8.12 2015-08-18T00:12:00Z 7.887 ``` The query returns the lower time bound for each `GROUP BY time()` interval as the timestamp. | Algorithm \ Warmup Type | simple | exponential | none | | --- | --- | --- | --- | | Algorithm \ Warmup Type | simple | exponential | none | | EXPONENTIAL_MOVING_AVERAGE | PERIOD - 1 | PERIOD - 1 | n/a | | DOUBLE_EXPONENTIAL_MOVING_AVERAGE | ( PERIOD - 1 ) * 2 | PERIOD - 1 | n/a | | TRIPLE_EXPONENTIAL_MOVING_AVERAGE | ( PERIOD - 1 ) * 3 | PERIOD - 1 | n/a | | TRIPLE_EXPONENTIAL_DERIVATIVE | ( PERIOD - 1 ) * 3 + 1 | PERIOD | n/a | | RELATIVE_STRENGTH_INDEX | PERIOD | PERIOD | n/a | | CHANDE_MOMENTUM_OSCILLATOR | PERIOD | PERIOD | PERIOD - 1 | | Algorithm | Default Hold Period | | --- | --- | | Algorithm | Default Hold Period | | KAUFMANS_EFFICIENCY_RATIO() | PERIOD | | KAUFMANS_ADAPTIVE_MOVING_AVERAGE() | PERIOD | --- # Manage your database using InfluxQL Source: https://docs.influxdata.com/influxdb/v1/query_language/manage-database/ This page documents an earlier version of InfluxDB OSS.[InfluxDB 3 Core](/influxdb3/core/) is the latest stable version. InfluxQL offers a full suite of administrative commands. | **Data Management:** | **Retention Policy Management:** | |--------------------------------------------------------------|----------------------------------------------------------------------------------| | [CREATE DATABASE](#create-database) |[CREATE RETENTION POLICY](#create-retention-policies-with-create-retention-policy)| | [DROP DATABASE](#delete-a-database-with-drop-database) | [ALTER RETENTION POLICY](#modify-retention-policies-with-alter-retention-policy) | | [DROP SERIES](#drop-series-from-the-index-with-drop-series) | [DROP RETENTION POLICY](#delete-retention-policies-with-drop-retention-policy) | | [DELETE](#delete-series-with-delete) | | |[DROP MEASUREMENT](#delete-measurements-with-drop-measurement)| | | [DROP SHARD](#delete-a-shard-with-drop-shard) | | If you’re looking for `SHOW` queries (for example, `SHOW DATABASES` or `SHOW RETENTION POLICIES`), see [Schema Exploration](/influxdb/v1/query_language/explore-schema). The examples in the sections below use the InfluxDB [Command Line Interface (CLI)](/influxdb/v1/introduction/getting-started/). You can also execute the commands using the InfluxDB API; simply send a `GET` request to the `/query` endpoint and include the command in the URL parameter `q`. For more on using the InfluxDB API, see [Querying data](/influxdb/v1/guides/querying_data/). > > > **Note:** When authentication is enabled, only admin users can execute most of the commands listed on this page. > See the documentation on [authentication and authorization](/influxdb/v1/administration/authentication_and_authorization/) for more information. > > ## Data management ### CREATE DATABASE Creates a new database. #### Syntax ```sql CREATE DATABASE [WITH [DURATION ] [REPLICATION ] [SHARD DURATION ] [FUTURE LIMIT ] [PAST LIMIT ] [NAME ]] ``` #### Description of syntax `CREATE DATABASE` requires a database [name](/influxdb/v1/troubleshooting/frequently-asked-questions/#what-words-and-characters-should-i-avoid-when-writing-data-to-influxdb). The `WITH`, `DURATION`, `REPLICATION`, `SHARD DURATION`, `FUTURE LIMIT`,`PAST LIMIT`, and `NAME` clauses are optional and create a single[retention policy](/influxdb/v1/concepts/glossary/#retention-policy-rp)associated with the created database. If you do not specify one of the clauses after `WITH`, the relevant behavior defaults to the `autogen` retention policy settings. The created retention policy automatically serves as the database’s default retention policy. For more information about those clauses, see[Retention Policy Management](/influxdb/v1/query_language/manage-database/#retention-policy-management). A successful `CREATE DATABASE` query returns an empty result. If you attempt to create a database that already exists, InfluxDB does nothing and does not return an error. #### Examples ##### Create a database ``` CREATE DATABASE "NOAA_water_database" ``` The query creates a database called `NOAA_water_database`.[By default](/influxdb/v1/administration/config/#retention-autocreate), InfluxDB also creates the `autogen` retention policy and associates it with the `NOAA_water_database`. ##### Create a database with a specific retention policy ``` CREATE DATABASE "NOAA_water_database" WITH DURATION 3d REPLICATION 1 SHARD DURATION 1h NAME "liquid" ``` The query creates a database called `NOAA_water_database`. It also creates a default retention policy for `NOAA_water_database` with a `DURATION` of three days, a [replication factor](/influxdb/v1/concepts/glossary/#replication-factor-rf) of one, a [shard group](/influxdb/v1/concepts/glossary/#shard-group) duration of one hour, and with the name `liquid`. ### Delete a database with DROP DATABASE The `DROP DATABASE` query deletes all of the data, measurements, series, continuous queries, and retention policies from the specified database. The query takes the following form: ```sql DROP DATABASE ``` Drop the database NOAA\_water\_database: ```bash DROP DATABASE "NOAA_water_database" ``` A successful `DROP DATABASE` query returns an empty result. If you attempt to drop a database that does not exist, InfluxDB does not return an error. ### Drop series from the index with DROP SERIES The `DROP SERIES` query deletes all points from a [series](/influxdb/v1/concepts/glossary/#series) in a database, and it drops the series from the index. The query takes the following form, where you must specify either the `FROM` clause or the `WHERE` clause: ```sql DROP SERIES FROM WHERE ='' ``` Drop all series from a single measurement: ```sql DROP SERIES FROM "h2o_feet" ``` Drop series with a specific tag pair from a single measurement: ```sql DROP SERIES FROM "h2o_feet" WHERE "location" = 'santa_monica' ``` Drop all points in the series that have a specific tag pair from all measurements in the database: ```sql DROP SERIES WHERE "location" = 'santa_monica' ``` A successful `DROP SERIES` query returns an empty result. ### Delete series with DELETE The `DELETE` query deletes all points from a[series](/influxdb/v1/concepts/glossary/#series) in a database. Unlike[`DROP SERIES`](/influxdb/v1/query_language/manage-database/#drop-series-from-the-index-with-drop-series), `DELETE` does not drop the series from the index. You must include either the `FROM` clause, the `WHERE` clause, or both: ```sql DELETE FROM WHERE [=''] | [