Filepath Processor Plugin
This plugin allows transforming a path, using e.g. basename to extract the last path element, for tag and field values. Values can be modified in place or stored in another key.
Introduced in: Telegraf v1.15.0 Tags: transformation OS support: all
Global configuration options
Plugins support additional global and plugin configuration settings for tasks such as modifying metrics, tags, and fields, creating aliases, and configuring plugin ordering. See CONFIGURATION.md for more details.
Configuration
# Performs file path manipulations on tags and fields
[[processors.filepath]]
## Treat the tag value as a path and convert it to its last element, storing the result in a new tag
# [[processors.filepath.basename]]
# tag = "path"
# dest = "basepath"
## Treat the field value as a path and keep all but the last element of path, typically the path's directory
# [[processors.filepath.dirname]]
# field = "path"
## Treat the tag value as a path, converting it to its the last element without its suffix
# [[processors.filepath.stem]]
# tag = "path"
## Treat the tag value as a path, converting it to the shortest path name equivalent
## to path by purely lexical processing
# [[processors.filepath.clean]]
# tag = "path"
## Treat the tag value as a path, converting it to a relative path that is lexically
## equivalent to the source path when joined to 'base_path'
# [[processors.filepath.rel]]
# tag = "path"
# base_path = "/var/log"
## Treat the tag value as a path, replacing each separator character in path with a '/' character. Has only
## effect on Windows
# [[processors.filepath.toslash]]
# tag = "path"Considerations
Processing order
This plugin processes the specified functions in the order they appear in
the configuration. One exceptition is the stem section which is applied first.
If you plan to apply multiple transformations to the same tag/field, bear in
mind the processing order stated above.
Clean Automatic Invocation
Even though clean is provided a standalone function, it is also invoked when
using the rel and dirname functions, so there is no need to use it along
with them.
That is:
[[processors.filepath]]
[[processors.filepath.dir]]
tag = "path"
[[processors.filepath.clean]]
tag = "path"Is equivalent to:
[[processors.filepath]]
[[processors.filepath.dir]]
tag = "path"ToSlash Platform-specific Behavior
The effects of this function are only noticeable on Windows platforms, because of the underlying golang implementation.
Examples
Basename
[[processors.filepath]]
[[processors.filepath.basename]]
tag = "path"- my_metric,path="/var/log/batch/ajob.log" duration_seconds=134 1587920425000000000
+ my_metric,path="ajob.log" duration_seconds=134 1587920425000000000
Dirname
[[processors.filepath]]
[[processors.filepath.dirname]]
field = "path"
dest = "folder"- my_metric path="/var/log/batch/ajob.log",duration_seconds=134 1587920425000000000
+ my_metric path="/var/log/batch/ajob.log",folder="/var/log/batch",duration_seconds=134 1587920425000000000
Stem
[[processors.filepath]]
[[processors.filepath.stem]]
tag = "path"- my_metric,path="/var/log/batch/ajob.log" duration_seconds=134 1587920425000000000
+ my_metric,path="ajob" duration_seconds=134 1587920425000000000
Clean
[[processors.filepath]]
[[processors.filepath.clean]]
tag = "path"- my_metric,path="/var/log/dummy/../batch//ajob.log" duration_seconds=134 1587920425000000000
+ my_metric,path="/var/log/batch/ajob.log" duration_seconds=134 1587920425000000000
Rel
[[processors.filepath]]
[[processors.filepath.rel]]
tag = "path"
base_path = "/var/log"- my_metric,path="/var/log/batch/ajob.log" duration_seconds=134 1587920425000000000
+ my_metric,path="batch/ajob.log" duration_seconds=134 1587920425000000000
ToSlash
[[processors.filepath]]
[[processors.filepath.rel]]
tag = "path"- my_metric,path="\var\log\batch\ajob.log" duration_seconds=134 1587920425000000000
+ my_metric,path="/var/log/batch/ajob.log" duration_seconds=134 1587920425000000000
Processing paths from tail plugin
This plugin can be used together with the tail input
plugin to make modifications to the path tag
injected for every file.
Scenario:
- A log file
/var/log/myjobs/mysql_backup.log, containing logs for a job execution. Whenever the job ends, a line is written to the log file following this format:2020-04-05 11:45:21 total time execution: 70 seconds - We want to generate a measurement that captures the duration of the script as
a field and includes the
pathas a tag- We are interested in the filename without its extensions, since it might be enough information for plotting our execution times in a dashboard
- Just in case, we don’t want to override the original path (if for some reason we end up having duplicates we might want this information)
For this purpose, we will use the tail input plugin, the grok parser plugin
and the filepath processor.
# Performs file path manipulations on tags and fields
[[inputs.tail]]
files = ["/var/log/myjobs/**.log"]
data_format = "grok"
grok_patterns = ['%{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02 15:04:05"} total time execution: %{NUMBER:duration_seconds:int}']
name_override = "myjobs"
[[processors.filepath]]
[[processors.filepath.stem]]
tag = "path"
dest = "stempath"The resulting output for a job taking 70 seconds for the mentioned log file would look like:
myjobs_duration_seconds,host="my-host",path="/var/log/myjobs/mysql_backup.log",stempath="mysql_backup" 70 1587920425000000000Was this page helpful?
Thank you for your feedback!
Support and feedback
Thank you for being part of our community! We welcome and encourage your feedback and bug reports for Telegraf and this documentation. To find support, use the following resources:
Customers with an annual or support contract can contact InfluxData Support.