TUF & Agent Updates
Relevant source files
The following files were used as context for generating this wiki page:
The Update Framework (TUF) serves as the secure foundation for distributing agent updates within the Fleet ecosystem. It is primarily used to manage and deliver updates for fleetd components, including Orbit and osquery. By utilizing a set of cryptographic keys and metadata, TUF ensures that updates are signed, verified, and protected against various repository compromise attacks.
In Fleet, agent updates are decoupled from server releases, allowing for independent lifecycle management of the endpoint agents. This system provides deployment flexibility, enabling updates to be pushed to specific channels (e.g., stable, edge) and verified before broad rollout.
Sources: handbook/engineering/tuf.md, handbook/engineering/releases.md:143-150
TUF Repository Architecture
The TUF repository in Fleet is structured to maintain high security by separating online and offline roles. Fleet hosts these repositories in Cloudflare R2 buckets (e.g., updates.fleetdm.com and updates-staging.fleetdm.com).
Key Roles and Responsibilities
TUF employs four standard roles, each associated with specific keys:
| Role | Key Storage | Responsibility |
|---|---|---|
| Root | Offline (USB) | The "root of trust"; signs for all other keys and metadata. |
| Targets | Offline (USB) | Signs the metadata for specific file targets (binaries). |
| Snapshot | Online/Automated | Signs the state of the entire repository to prevent mix-and-match attacks. |
| Timestamp | Online/Automated | Provides a fresh timestamp to prevent replay attacks. |
Sources: handbook/engineering/tuf.md:73-82
Security Logic: Air-Gapped Initialization
Initialization of a TUF repository requires an air-gapped Ubuntu environment to ensure keys are never exposed to the internet. The process involves generating multiple root keys (typically three) to provide redundancy.
Sources: handbook/engineering/tuf.md:16-56
Agent Release Process
The fleetd agent release involves updating components like Orbit, osqueryd, and Fleet Desktop. These updates are published to the TUF repository using specialized scripts.
Component Delivery
Updates for macOS, Windows, and Linux are managed via TUF targets. The fleetctl updates init and tuf CLI binaries are used to initialize and manage these targets.
Release Channels
Updates are typically released to an "edge" channel for initial verification before moving to "stable". The status of these channels can be queried using the tuf-status tool.
Key Management and Rotation
Maintaining the security of the TUF repository requires strict key rotation policies and access controls.
Access Control
The CTO determines who has access to push agent updates. While Timestamp and Snapshot keys can be held online for automation, Targets and Root keys must always be held offline. Targets keys are generated to provide approved team members (Releasers) the ability to push updates without possessing Root keys.
Annual Rotation
Root keys expire every year and must be manually rotated at least 30 days prior to expiration. This process requires the physical retrieval of offline keys and a multi-signature signing process in an air-gapped environment.
Testing TUF Systems
Fleet provides a suite of tools in tools/tuf/test/ to simulate the TUF update flow locally.
Local Simulation
The main.sh script creates a local TUF repository and generates installers for various platforms (macOS, Windows, Linux, including ARM64 variants). It uses host.docker.internal to resolve the local Fleet server for virtualized hosts.
Test Extensions and Expirations
The testing framework allows developers to:
- Publish test extensions (e.g.,
hello_world). - Include third-party tools like
Escrow Buddy,Nudge, orSwift Dialog. - Simulate expired signatures by setting short durations for metadata (e.g.,
5mforSNAPSHOT_EXPIRATION_DURATION).
Sources: tools/tuf/test/README.md:1-85
Tooling Reference
| Tool | Path | Purpose |
|---|---|---|
fleetctl updates | orbit/cmd/fleetctl | Primary command for initializing TUF repos. |
tuf | go-tuf | Underlying CLI for advanced signing and key generation. |
tuf-status | tools/tuf/status/tuf-status.go | Queries channel versions (e.g., stable/edge). |
push_target.sh | tools/tuf/test/push_target.sh | Utility to push new binaries to a test TUF repo. |
releaser.sh | tools/tuf/releaser.sh | Script for publishing official releases to TUF. |
Sources: tools/README.md:213-219, tools/tuf/test/README.md:92-95
Conclusion
TUF & Agent Updates provide a secure, air-gapped root of trust for Fleet's endpoint agents. By enforcing offline key storage for critical roles and providing robust local testing tools, the system ensures that endpoint security is never compromised during the software delivery lifecycle. Professional management of the TUF repository—specifically the annual rotation of root keys and the careful delegation of target-signing authority—is vital for maintaining the integrity of the entire Fleet deployment.