Enable trace logging
Learn how to enable trace logging to help you identify performance bottlenecks and troubleshoot problems in queries.
When you enable trace logging for a query, InfluxDB propagates your trace ID through system processes and collects additional log information. InfluxData Support can then use the trace ID that you provide to filter, collate, and analyze log information for the query run. The tracing system follows the OpenTelemetry traces model for providing observability into a request.
Avoid unnecessary tracing
Only enable tracing for a query when you need to request troubleshooting help from InfluxDB Support. To manage resources, InfluxDB has an upper limit for the number of trace requests. Too many traces can cause InfluxDB to evict log information.
To enable tracing for a query, include the influx-trace-id
header in your query request.
Syntax
Use the following syntax for the influx-trace-id
header:
influx-trace-id: TRACE_ID:1112223334445:0:1
In the header value, replace the following:
TRACE_ID
: a unique string, 8-16 bytes long, encoded as hexadecimal (32 maximum hex characters). The trace ID should uniquely identify the query run.:1112223334445:0:1
: InfluxDB constant values (required, but ignored)
Example
The following examples show how to create and pass a trace ID to enable query tracing in InfluxDB:
Use the InfluxDBClient3
InfluxDB Python client and pass the headers
argument in the
query()
method.
from influxdb_client_3 import InfluxDBClient3
import secrets
def use_flightcalloptions_trace_header():
print('# Use FlightCallOptions to enable tracing.')
client = InfluxDBClient3(token=f"DATABASE_TOKEN",
host=f"cluster-id.a.influxdb.io",
database=f"DATABASE_NAME")
# Generate a trace ID for the query:
# 1. Generate a random 8-byte value as bytes.
# 2. Encode the value as hexadecimal.
random_bytes = secrets.token_bytes(8)
trace_id = random_bytes.hex()
# Append required constants to the trace ID.
trace_value = f"{trace_id}:1112223334445:0:1"
# Encode the header key and value as bytes.
# Create a list of header tuples.
headers = [((b"influx-trace-id", trace_value.encode('utf-8')))]
sql = "SELECT * FROM home WHERE time >= now() - INTERVAL '30 days'"
influxql = "SELECT * FROM home WHERE time >= -90d"
# Use the query() headers argument to pass the list as FlightCallOptions.
client.query(sql, headers=headers)
client.close()
use_flightcalloptions_trace_header()
Use the InfluxDBClient3
InfluxDB Python client and flight.ClientMiddleware
to pass and inspect headers.
Tracing response header
With tracing enabled and a valid trace ID in the request, InfluxDB’s DoGet
action response contains a header with the trace ID that you sent.
Trace response header syntax
trace-id: TRACE_ID
Inspect Flight response headers
To inspect Flight response headers when using a client library, pass a FlightClientMiddleware
instance
that defines a middleware callback function for the onHeadersReceived
event (the particular function name you use depends on the client library language).
The following example uses Python client middleware that adds request headers and extracts the trace ID from the DoGet
response headers:
import pyarrow.flight as flight
class TracingClientMiddleWareFactory(flight.ClientMiddleware):
# Defines a custom middleware factory that returns a middleware instance.
def __init__(self):
self.request_headers = []
self.response_headers = []
self.traces = []
def addRequestHeader(self, header):
self.request_headers.append(header)
def addResponseHeader(self, header):
self.response_headers.append(header)
def addTrace(self, traceid):
self.traces.append(traceid)
def createTrace(self, traceid):
# Append InfluxDB constants to the trace ID.
trace = f"{traceid}:1112223334445:0:1"
# To the list of request headers,
# add a tuple with the header key and value as bytes.
self.addRequestHeader((b"influx-trace-id", trace.encode('utf-8')))
def start_call(self, info):
return TracingClientMiddleware(info.method, self)
class TracingClientMiddleware(flight.ClientMiddleware):
# Defines middleware with client event callback methods.
def __init__(self, method, callback_obj):
self._method = method
self.callback = callback_obj
def call_completed(self, exception):
print('callback: call_completed')
if(exception):
print(f" ...with exception: {exception}")
def sending_headers(self):
print('callback: sending_headers: ', self.callback.request_headers)
if len(self.callback.request_headers) > 0:
return dict(self.callback.request_headers)
def received_headers(self, headers):
self.callback.addResponseHeader(headers)
# For the DO_GET action, extract the trace ID from the response headers.
if str(self._method) == "FlightMethod.DO_GET" and "trace-id" in headers:
trace_id = headers["trace-id"][0]
self.callback.addTrace(trace_id)
from influxdb_client_3 import InfluxDBClient3
import secrets
def use_middleware_trace_header():
print('# Use Flight client middleware to enable tracing.')
# Instantiate the middleware.
res = TracingClientMiddleWareFactory()
# Instantiate the client, passing in the middleware instance that provides
# event callbacks for the request.
client = InfluxDBClient3(token=f"DATABASE_TOKEN",
host=f"cluster-id.a.influxdb.io",
database=f"DATABASE_NAME",
flight_client_options={"middleware": (res,)})
# Generate a trace ID for the query:
# 1. Generate a random 8-byte value as bytes.
# 2. Encode the value as hexadecimal.
random_bytes = secrets.token_bytes(8)
trace_id = random_bytes.hex()
res.createTrace(trace_id)
sql = "SELECT * FROM home WHERE time >= now() - INTERVAL '30 days'"
client.query(sql)
client.close()
assert trace_id in res.traces[0], "Expect trace ID in DoGet response."
use_middleware_trace_header()
Replace the following:
DATABASE_NAME
: your InfluxDB Cloud Dedicated databaseDATABASE_TOKEN
: a database token with sufficient permissions to the specified database
Store or log your query trace ID to ensure you can provide it to InfluxData Support for troubleshooting.
After you run your query with tracing enabled, do the following:
- Remove the tracing header from subsequent runs of the query (to avoid unnecessary tracing).
- Provide the trace ID in a request to InfluxData Support.
Retrieve system information for a query
If you enable trace logging for a query, the trace-id
appears in the system.queries
table.
Was this page helpful?
Thank you for your feedback!
Support and feedback
Thank you for being part of our community! We welcome and encourage your feedback and bug reports for InfluxDB and this documentation. To find support, use the following resources:
Customers with an annual or support contract can contact InfluxData Support.