Log Destinations & Telemetry Export
Relevant source files
The following files were used as context for generating this wiki page:
Fleet provides a robust system for streaming endpoint telemetry, audit logs, and status data to various external logging destinations. This capability allows organizations to centralize security data in Data Lakes, SIEMs (Security Information and Event Management), and analytics platforms. By leveraging cloud-native streaming services, Fleet ensures that high-volume data from thousands of osquery-enabled endpoints is delivered reliably for real-time analysis and long-term retention.
The telemetry export system supports three main categories of logs: osquery status logs (agent health), osquery results logs (query outputs), and Fleet audit logs (administrative actions). These logs can be routed through mechanisms such as AWS Kinesis Data Firehose or Kinesis Data Streams to reach final destinations like Splunk, Sumo Logic, or Amazon S3.
Sources: articles/how-to-configure-logging-destinations.md:7-16, tools/fleet-slackbot/system-prompt.js:203-210
Supported Data Types
Fleet exports three primary types of telemetry and administrative data to external destinations:
| Log Type | Description |
|---|---|
| Osquery Status Logs | Logs indicating the health and operational status of the osquery agent on the host. |
| Osquery Results Logs | The actual output and differential data generated by scheduled or ad-hoc osquery queries. |
| Fleet Audit Logs | Records of administrative actions taken within the Fleet server (e.g., policy changes, user additions). |
Sources: articles/how-to-configure-logging-destinations.md:10-16, tools/fleet-slackbot/system-prompt.js:203-210
Export Mechanisms & Architecture
The following diagram illustrates the high-level data flow from a Fleet instance to cloud-native storage and analytics platforms via AWS Kinesis.
Note: The flow shows how Fleet instances interface with Kinesis to route logs to various third-party and native AWS services.
Sources: articles/how-to-configure-logging-destinations.md:21-42
AWS Kinesis Data Firehose
Kinesis Data Firehose is a fully managed service used to load streaming data into data stores and analytics services. It is the preferred choice for simplifying the loading of data into Amazon S3, Redshift, or third-party providers like Splunk and Sumo Logic. It also supports capturing undelivered logs in an AWS S3 bucket as a fallback mechanism.
Sources: articles/how-to-configure-logging-destinations.md:28-32, articles/how-to-configure-logging-destinations.md:37-41
AWS Kinesis Data Streams
Kinesis Data Streams is optimized for real-time data processing with extremely low latency. It is designed to capture gigabytes of data per second from hundreds of thousands of sources, enabling continuous real-time analysis of website clickstreams, database event streams, and social media feeds.
Sources: articles/how-to-configure-logging-destinations.md:26-27
Configuration & Security
Exporting telemetry requires specific infrastructure setup, primarily focused on AWS Identity and Access Management (IAM) to ensure secure cross-account or cross-service communication.
Required Infrastructure Parameters
| Parameter | Description | Requirement |
|---|---|---|
| Fleet Service IAM Role ARN | The ARN of the IAM role assigned to the Fleet service that needs to write to the stream. | Required |
| Assumed IAM Role ARN | The role that the Fleet service will assume to grant permissions in the target environment. | Required |
| ExternalId | An optional unique identifier used to enhance security during role assumption. | Recommended |
| Stream Name | The specific name of the Kinesis Stream or Firehose delivery stream. | Required |
| Region | The AWS region where the logging destination is hosted. | Required |
Sources: articles/how-to-configure-logging-destinations.md:47-65
Integration with Analytics Dashboards
Telemetry exported via these destinations can be consumed by external tools to build specialized security and compliance dashboards.
Vulnerability Dashboard
Fleet supports an enterprise vulnerability dashboard that tracks progress on fixing CVEs detected across the fleet. This dashboard utilizes data annotated with NVD, CVSS, EPSS, and CISA KEV information. It can be deployed using Docker and connects to the Fleet instance via an API token.
Sources: ee/vulnerability-dashboard/README.md:1-25, ee/vulnerability-dashboard/README.md:35-50
Custom Dashboards (REST API)
For use cases where pre-built integrations are insufficient, users can leverage the Fleet REST API to collect telemetry data programmatically. Common tech stacks for this include:
- Flask: A Python web application framework used to host the dashboard.
- Plotly/Dash: Open-source libraries for creating interactive data visualizations.
- Requests/Pandas: Used for data retrieval and manipulation from Fleet API endpoints such as
/softwareor/hosts.
Sources: articles/building-an-effective-dashboard-with-fleet-rest-api-flask-and-plotly.md:15-32, articles/building-an-effective-dashboard-with-fleet-rest-api-flask-and-plotly.md:65-75
Summary
The Log Destinations and Telemetry Export system in Fleet is a critical component for enterprise security operations. By supporting native AWS streaming services, Fleet allows for the scalable export of osquery results, agent status, and audit logs. This architecture enables organizations to maintain long-term compliance records in S3 while simultaneously powering real-time security alerts and custom analytics dashboards in platforms like Splunk, Sumo Logic, or custom-built Flask applications.
Sources: articles/how-to-configure-logging-destinations.md:100-105, articles/building-an-effective-dashboard-with-fleet-rest-api-flask-and-plotly.md:102-108