Event Ingestion Endpoints

Relevant source files

The following files were used as context for generating this wiki page:

• app/api/controller/event.controller.ts
• packages/shared/zod.schema.ts
• packages/analytics-web/transport/sender.js
• app/workers/eventWorker.ts
• PROJECT_OUTLINE.md
• packages/shared/lib/getIp.ts
• test/fire_event.sh

The Event Ingestion Endpoints serve as the primary entry point for all tracking data sent from client-side applications to the Glimpse platform. This system is designed to receive, validate, and queue high volumes of analytics data asynchronously to ensure minimal latency for the end-user while maintaining data integrity.

The ingestion layer utilizes the Hono framework for high-performance HTTP routing and Zod for strict schema validation. Once a request is validated, it is enriched with metadata (such as IP address and User-Agent) and passed to a BullMQ queue for background processing by the worker system.

Sources: PROJECT_OUTLINE.md:58-69, app/api/controller/event.controller.ts:1-5

Endpoint Specification

POST /event

The primary endpoint for submitting analytics events. It accepts a JSON payload representing a single analytics event and returns an immediate acknowledgment.

Aspect	Details
Method	POST
URL	/event
Content-Type	application/json
Success Response	200 OK with { status: 'accepted' }
Error Response	400 Bad Request with validation error details

Sources: app/api/controller/event.controller.ts:11-54, test/fire_event.sh:6-8

Ingestion Data Flow

The flow of an event from the client to the persistence layer involves multiple stages of validation and enrichment.

Request Lifecycle Diagram

This diagram illustrates the sequence of operations from the moment a client sends an event until it is accepted into the processing queue.

Data Validation and Schema

The ingestion service enforces a strict schema using EventSchema to ensure that only well-formatted data enters the system.

Core Event Fields

The following fields are extracted and validated from the request body:

Field	Type	Description
projectId	String	Unique identifier for the project being tracked.
event	String	The name of the action (e.g., "page_view", "click").
timestamp	Number	Unix timestamp of when the event occurred.
sessionId	String	Identifier for the current user session.
anonymousId	String	Identifier for the anonymous user.
context	Object	Metadata including URL, referrer, and device info.

Sources: packages/shared/zod.schema.ts:3-56

Metadata Enrichment

Upon successful validation, the controller enriches the event with server-side metadata before queuing:

• IP Address: Extracted using the getIP utility, supporting headers like X-Forwarded-For.
• User-Agent: Retrieved from the User-Agent request header.

Sources: app/api/controller/event.controller.ts:44-50, packages/shared/lib/getIp.ts

Transport and Buffering

Events are sent from the client-side analytics-web SDK using a buffered approach to reduce HTTP overhead.

Client-Side Sender Logic

The sender.js module manages an internal buffer and implements a retry mechanism:

1. Buffering: Events are enqueued locally until the BUFFER_SIZE (10) is reached or a FLUSH_INTERVAL (5000ms) expires.
2. Retry Logic: Failed requests are retried up to 3 times with an exponential delay.
3. Cors/Mode: Requests use mode: 'cors' and credentials: 'omit'.

Sources: packages/analytics-web/transport/sender.js:1-67

Server-Side Processing

Once the API accepts the event, it is placed in a BullMQ named eventQueue. The eventWorker then processes these jobs by:

1. Generating missing IDs (Session/Anonymous) via crypto.randomUUIDv7().
2. Performing GeoIP lookups based on the enriched IP address.
3. Updating real-time active session counts in Redis with a 300s TTL.
4. Adding the event to a server-side memory buffer (eventBuffer) for batch insertion into the database.

Sources: app/workers/eventWorker.ts:88-146, PROJECT_OUTLINE.md:71-92

Error Handling

The ingestion controller provides structured feedback when validation fails. If the JSON is malformed or missing required fields defined in the EventSchema, the API returns a list of issues.

{
    "status": "error",
    "errors": [
        {
            "path": "projectId",
            "message": "Required"
        }
    ]
}

Sources: app/api/controller/event.controller.ts:25-37

Summary

The Event Ingestion Endpoints provide a robust gateway for the Glimpse analytics platform. By separating the synchronous HTTP acknowledgment from the heavy lifting of geolocation lookup and database persistence via BullMQ, the system ensures high performance and scalability. The combination of Zod schema validation and automatic metadata enrichment provides a clean, reliable data stream for the downstream analytics pipeline.