Log Streaming

Relevant source files

The following files were used as context for generating this wiki page:

• packages/cli/src/modules/log-streaming.ee/log-streaming-destination.service.ts
• packages/cli/src/modules/log-streaming.ee/destinations/message-event-bus-destination-syslog.ee.ts
• packages/cli/src/modules/log-streaming.ee/log-streaming.module.ee.ts
• packages/cli/src/modules/log-streaming.ee/log-streaming.controller.ee.ts
• packages/cli/src/modules/log-streaming.ee/log-streaming.service.ee.ts

Log Streaming is an Enterprise Edition (EE) feature in n8n designed to export internal application logs and event data to external monitoring systems. It provides a standardized way to relay messages from n8n's internal Event Bus to external destinations like Syslog servers, enabling centralized logging, audit trails, and real-time monitoring of workflow executions and system events.

The system is built around a modular architecture where a central service manages various "destinations." Each destination represents a connection to an external service (e.g., a Syslog server via TCP or UDP) and can be configured with specific filters to determine which types of log messages are streamed.

Architecture and Components

The Log Streaming module follows a controller-service-destination pattern, integrated with n8n's dependency injection system and the core Event Bus.

System Overview Diagram

The following diagram illustrates how internal events flow from the Event Bus through the Log Streaming service to external destinations.

Key Components

Component	Description
LogStreamingController	Handles REST API requests for managing log streaming configurations (CRUD operations).
LogStreamingService	Acts as the primary entry point, subscribing to the Event Bus and coordinating with the destination service.
LogStreamingDestinationService	Manages the lifecycle of log destinations, including creation, updates, and deletion of active streams.
MessageEventBusDestinationSyslog	A specialized implementation for streaming logs to Syslog servers using various protocols.

Sources: packages/cli/src/modules/log-streaming.ee/log-streaming.controller.ee.ts, packages/cli/src/modules/log-streaming.ee/log-streaming-destination.service.ts

Data Flow: Event Subscription and Relay

The system operates by subscribing to the internal MessageEventBus. When a new message is published to the bus, the Log Streaming service evaluates active destinations to determine if the message should be forwarded.

Subscription Sequence

This diagram shows the sequence of events when a system log is generated and relayed.

Destination Logic

The LogStreamingDestinationService maintains an internal map of active destination instances. When a configuration is added or updated via the API, the service instantiates the appropriate class (e.g., MessageEventBusDestinationSyslog) and initializes the connection.

Sources: packages/cli/src/modules/log-streaming.ee/log-streaming-destination.service.ts:45-65

Syslog Destination Implementation

The Syslog destination is the primary implementation for log streaming. It supports multiple transport protocols and adheres to standard logging formats.

Supported Protocols and Formats

The MessageEventBusDestinationSyslog class implements the MessageEventBusDestination interface. It provides robust error handling, including a "cool-off" period if the external server becomes unreachable to prevent system performance degradation.

• Transport Protocols: Supports TCP, UDP, and TLS (for encrypted log transmission).
• Message Format: Implements formatting consistent with RFC 5424.
• Customization: Allows setting Syslog Facility and App Name via configuration.

Configuration Structure

export interface SyslogDestinationOptions {
	host: string;
	port: number;
	protocol: 'tcp' | 'udp' | 'tls';
	facility: string;
	appName: string;
	ca?: string; // For TLS
	cert?: string; // For TLS
	key?: string; // For TLS
}

Sources: packages/cli/src/modules/log-streaming.ee/destinations/message-event-bus-destination-syslog.ee.ts:25-45

API Endpoints

The LogStreamingController exposes endpoints to manage destinations. These endpoints are restricted to users with appropriate Enterprise permissions.

Method	Path	Description
GET	/log-streaming/destinations	List all configured log streaming destinations.
POST	/log-streaming/destinations	Create a new log streaming destination.
PATCH	/log-streaming/destinations/:id	Update an existing destination configuration.
DELETE	/log-streaming/destinations/:id	Remove a destination and close active connections.

Sources: packages/cli/src/modules/log-streaming.ee/log-streaming.controller.ee.ts:20-80

Error Handling and Performance

The Log Streaming module is designed to be non-blocking. If a destination fails to receive a message:

1. Async Transmission: Messages are sent asynchronously to avoid delaying the core workflow execution.
2. Circuit Breaking: The MessageEventBusDestinationSyslog tracks connection failures. If failures exceed a threshold, the destination enters a temporary disabled state (cool-off) to avoid repeated connection attempts that could consume system resources.
3. Validation: Input configurations for destinations are validated before activation to ensure hostnames and ports are reachable.

Sources: packages/cli/src/modules/log-streaming.ee/destinations/message-event-bus-destination-syslog.ee.ts:120-150, packages/cli/src/modules/log-streaming.ee/log-streaming-destination.service.ts:80-95

Summary

Log Streaming provides an enterprise-grade solution for externalizing n8n logs. By leveraging n8n's internal Event Bus and providing a flexible destination framework, it allows administrators to maintain compliance and gain visibility into workflow performance through external monitoring tools like Syslog servers. The implementation ensures that external logging does not impact the stability of the primary automation engine through robust error handling and asynchronous data transfer.