Workflow Analytics & Goals

Relevant source files

The following files were used as context for generating this wiki page:

• backend/src/workflows/application/analytics_aggregation_service.py
• backend/src/workflows/application/goal_detection_service.py
• backend/src/workflows/domain/analytics_entities.py
• DDD_IMPLEMENTATION_REPORT_SPEC_WFL_009.md
• SPEC-WFL-007-EXECUTIVE-SUMMARY.md
• SPEC-WFL-007-IMPLEMENTATION-REPORT.md
• DDD_SUMMARY.md

Introduction

The Workflow Analytics & Goals system is a critical infrastructure component of the automation engine, designed to track, aggregate, and visualize the performance of marketing automation sequences. It provides real-time insights into contact enrollments, conversion rates, and step-level performance, enabling data-driven optimization of business processes. The system is built using Clean Architecture and Domain-Driven Design (DDD) principles, ensuring high maintainability and scalability.

The scope of this module encompasses the tracking of individual workflow executions, the detection of business goal achievements based on specific event criteria (e.g., purchases, form submissions), and the aggregation of these granular events into daily snapshots for high-performance dashboard rendering.

Sources: DDD_IMPLEMENTATION_REPORT_SPEC_WFL_009.md, SPEC-WFL-007-IMPLEMENTATION-REPORT.md

Analytics Architecture & Data Flow

The analytics system operates across four distinct layers, separating domain logic from infrastructure concerns. It utilizes a pre-aggregation strategy where real-time execution data is processed into daily snapshots to minimize query latency for dashboards.

Data Aggregation Process

The AnalyticsAggregationService manages the lifecycle of performance data, including caching strategies using Redis and real-time updates via Server-Sent Events (SSE).

Key Analytics Entities

The domain layer defines several aggregate roots and entities to represent performance metrics:

Entity	Purpose
WorkflowAnalytics	Aggregate root for daily performance summaries (enrollments, completions).
WorkflowStepMetrics	Tracks funnel data at the individual step level (entry, drop-off, success).
MetricsSnapshot	A time-series snapshot for historical trend analysis.

Sources: backend/src/workflows/domain/analytics_entities.py:1-25, DDD_SUMMARY.md

Goal Tracking & Detection

Goal tracking allows users to define specific outcomes (Goals) that signify the success of a workflow. When a goal is achieved, the system can trigger specific logic, such as exiting the contact from the automation sequence.

Goal Detection Logic

The GoalDetectionService evaluates incoming events against defined GoalConfig criteria. It supports five primary goal types:

1. Tag Added: Triggered when a specific CRM tag is applied to a contact.
2. Purchase Made: Triggered by payment events.
3. Appointment Booked: Triggered by calendar bookings.
4. Form Submitted: Triggered by lead capture forms.
5. Pipeline Stage Reached: Triggered when a contact moves to a specific sales stage.

Goal Achievement Schema

Achievements are persisted in the workflow_goal_achievements table, which includes an audit trail of the triggering event.

Field	Type	Description
goal_config_id	UUID	Reference to the goal definition.
contact_id	UUID	The contact who achieved the goal.
trigger_event	JSONB	The raw event data that caused the achievement.
achieved_at	TIMESTAMPTZ	Timestamp of achievement.

Sources: SPEC-WFL-007-IMPLEMENTATION-REPORT.md, DDD_SUMMARY.md

Performance Metrics & Funnel Analysis

The system performs automated funnel analysis to identify bottlenecks in automation workflows. This is handled by the FunnelAnalysisService, which calculates conversion rates between sequential steps.

Conversion Calculations

The system tracks three core rate types:

• Completion Rate: Percentage of enrolled contacts who reached the final step.
• Conversion Rate: Percentage of contacts who achieved a defined goal.
• Step Conversion Rate: Percentage of contacts who successfully transitioned from step $N$ to step $N+1$.

Sources: backend/src/workflows/domain/analytics_entities.py:100-150, DDD_SUMMARY.md

API Endpoints for Analytics

The presentation layer provides several endpoints for retrieving aggregated data:

# Analytics Routes (FastAPI)
GET  /api/v1/workflows/{id}/analytics           # Summary dashboard
GET  /api/v1/workflows/{id}/analytics/funnel    # Bottleneck analysis
GET  /api/v1/workflows/{id}/analytics/actions   # Action performance
POST /api/v1/workflows/{id}/analytics/export    # CSV/PDF/JSON export
GET  /api/v1/workflows/{id}/analytics/stream    # Real-time SSE updates

Sources: DDD_IMPLEMENTATION_REPORT_SPEC_WFL_009.md, DDD_SUMMARY.md

Data Retention & Maintenance

To maintain system performance, a DataRetentionService enforces cleanup policies on granular data while preserving aggregated snapshots.

• Daily Aggregated Metrics: Retained for 2 years.
• Step-level Funnel Data: Retained for 90 days.
• Individual Executions: Subject to cleanup based on account policy.

Sources: DDD_IMPLEMENTATION_REPORT_SPEC_WFL_009.md, DDD_SUMMARY.md

Summary

The Workflow Analytics & Goals module provides a robust framework for measuring automation success. By combining granular goal detection with high-level performance aggregation, it offers a complete view of the customer journey within workflows. The implementation ensures that performance tracking is both accurate (through DDD entities) and highly performant (via pre-aggregation and caching).