Microsoft Fabric Output Plugin

This plugin writes metrics to Fabric Eventhouse and Fabric Eventstream artifacts of Real-Time Intelligence in Microsoft Fabric.

Real-Time Intelligence is a SaaS service in Microsoft Fabric that allows you to extract insights and visualize data in motion. It offers an end-to-end solution for event-driven scenarios, streaming data, and data logs.

Introduced in: Telegraf v1.35.0 Tags: datastore OS support: all

Global configuration options

In addition to the plugin-specific configuration settings, plugins support additional global and plugin configuration settings. These settings are used to modify metrics, tags, and field or create aliases and configure ordering, etc. See the CONFIGURATION.md for more details.

Configuration

# Sends metrics to Microsoft Fabric
[[outputs.microsoft_fabric]]
  ## The URI property of the resource on Microsoft Fabric
  connection_string = "https://trd-abcd.xx.kusto.fabric.microsoft.com;Database=kusto_eh;Table Name=telegraf_dump;Key=value"

  ## Client timeout
  # timeout = "30s"

Connection String

The connection_string provide information necessary for the plugin to establish a connection to the Fabric service endpoint. It is a semicolon-delimited list of name-value parameter pairs, optionally prefixed by a single URI. The setting is specific to the type of endpoint you are using. The sections below will detail on the required and available name-value pairs for each type.

EventHouse

This plugin allows you to leverage Microsoft Fabric’s capabilities to store and analyze your Telegraf metrics. Eventhouse is a high-performance, scalable data-store designed for real-time analytics. It allows you to ingest, store and query large volumes of data with low latency. For more information, visit the Eventhouse documentation.

The following table lists all the possible properties that can be included in a connection string and provide alias names for each property.

Metrics Grouping

Metrics can be grouped in two ways to be sent to Eventhouse. To specify which metric grouping type the plugin should use, the respective value should be given to the Metrics Grouping Type in the connection string. If no value is given, by default, the metrics will be grouped using tablepermetric.

TablePerMetric

The plugin will group the metrics by the metric name and will send each group of metrics to an Eventhouse KQL DB table. If the table doesn’t exist the plugin will create the table, if the table exists then the plugin will try to merge the Telegraf metric schema to the existing table. For more information about the merge process check the .create-merge documentation.

The table name will match the metric name, i.e. the name of the metric must comply with the Eventhouse KQL DB table naming constraints in case you plan to add a prefix to the metric name.

SingleTable

The plugin will send all the metrics received to a single Eventhouse KQL DB table. The name of the table must be supplied via table_name parameter in the connection_string. If the table doesn’t exist the plugin will create the table, if the table exists then the plugin will try to merge the Telegraf metric schema to the existing table. For more information about the merge process check the .create-merge documentation.

Tables Schema

The schema of the Eventhouse table will match the structure of the metric. The corresponding command generated by the plugin would be like the following:

.create-merge table ['table-name']  (['fields']:dynamic, ['name']:string, ['tags']:dynamic, ['timestamp']:datetime)

The corresponding table mapping would be like the following:

.create-or-alter table ['table-name'] ingestion json mapping 'table-name_mapping' '[{"column":"fields", "Properties":{"Path":"$[\'fields\']"}},{"column":"name", "Properties":{"Path":"$[\'name\']"}},{"column":"tags", "Properties":{"Path":"$[\'tags\']"}},{"column":"timestamp", "Properties":{"Path":"$[\'timestamp\']"}}]'

Ingestion type

Refer to the following query below to check if streaming is enabled:

.show database <DB-Name> policy streamingingestion

To learn more about configuration, supported authentication methods and querying ingested data, check the documentation.

Eventstream

Eventstreams allow you to bring real-time events into Fabric, transform them, and then route them to various destinations without writing any code (no-code). For more information, visit the Eventstream documentation.

To communicate with an eventstream, you need to specify a connection string for the namespace or the event hub. The following properties can be added to the standard Eventstream connection string using key-value pairs: