Data Collection API

Relevant source files

The following files were used as context for generating this wiki page:

• src/app/api/send/route.ts
• src/tracker/index.js
• src/lib/constants.ts
• src/app/(main)/console/[websiteId]/TestConsolePage.tsx
• cypress/e2e/api-website.cy.ts

The Data Collection API is the core mechanism for capturing website analytics data within the Umami application. It consists of a client-side JavaScript tracker and a server-side ingestion endpoint. The tracker, when embedded on a website, collects information about page views, user interactions, and custom events, then sends this data to the API endpoint for processing and storage.

This system is designed to be lightweight and privacy-focused. It handles tasks such as bot detection, IP blocking, session management, and parsing of event data, including UTM parameters and various click IDs. The primary endpoint for data collection is POST /api/send.

Client-Side Tracker

The client-side component is a JavaScript file (script.js) that is included on the tracked website. It exposes a global window.umami object for manual tracking and automatically captures page views and specific user interactions.

Sources: src/tracker/index.js, src/app/(main)/console/[websiteId]/TestConsolePage.tsx

Configuration

The tracker is configured via data-* attributes on the <script> tag.

Attribute	Description	Example
data-website-id	Required. The UUID of the website to track analytics for.	data-website-id="your-uuid-here"
data-host-url	The base URL of the Umami instance. If not provided, it's inferred from the script's src.	data-host-url="https://my-umami.com"
data-auto-track	When set to "false", disables automatic page view tracking. Defaults to true.	data-auto-track="false"
data-do-not-track	When set to "true", respects the browser's Do Not Track setting.	data-do-not-track="true"
data-domains	A comma-separated list of domains to track. If specified, tracking will only occur on these domains.	data-domains="example.com,www.example.com"
data-tag	A tag to be associated with all events collected by this tracker instance.	data-tag="v2-beta"
data-cache	Enables caching of the website ID to reduce database lookups.	data-cache="true"

Sources: src/tracker/index.js:15-30, src/app/(main)/console/[websiteId]/TestConsolePage.tsx:112-117

window.umami Object

The tracker exposes a global umami object with two main functions for manual data collection.

• umami.track(name, data): Sends an event to the collection endpoint.
  • If called with no arguments, it tracks a standard page view.
  • If name is a string, it tracks a custom event with that name. The optional data object contains custom event properties.
  • If name is an object, it sends a custom payload.
  • If name is a function, it receives the default payload and can modify it before sending.
• umami.identify(id, data): Associates a user with a distinct ID and custom properties. This sends a payload of type identify.

Sources: src/tracker/index.js:199-223, src/app/(main)/console/[websiteId]/TestConsolePage.tsx:17-89

Automatic Tracking

When data-auto-track is enabled, the tracker performs several actions automatically:

1. Initial Page View: Tracks a page view when the document is ready.
2. URL Changes: Listens for history.pushState and history.replaceState to track page views in Single-Page Applications (SPAs).
3. Click Events: Listens for clicks on elements with a data-umami-event attribute. The value of the attribute is used as the event name. Additional data can be passed using data-umami-event-* attributes.

The following diagram shows the client-side logic for initialization and event handling.

API Endpoint: POST /api/send

This is the primary ingestion endpoint that receives data from the client-side tracker. It validates the incoming payload, performs security checks, manages session and visit IDs, and saves the data.

Sources: src/app/api/send/route.ts

Request Processing Flow

The endpoint follows a strict procedure to process each incoming request.

This diagram illustrates the server-side processing flow for an incoming tracking request.

Payload Schema

All requests to /api/send must conform to a specific JSON structure, validated by a Zod schema. The top-level object contains type and payload.

Sources: src/app/api/send/route.ts:27-65

Field	Type	Description	Required
type	string	The type of collection. Must be event or identify.	Yes
payload	object	The data payload for the collection event.	Yes

Payload Object

Field	Type	Description
website	uuid	The ID of the website being tracked. Exactly one of website, link, or pixel must be provided.
data	object	A JSON object for custom event data or identify properties.
hostname	string	The hostname of the page.
language	string	The language of the browser (navigator.language).
referrer	string	The document.referrer of the page.
screen	string	The screen resolution, e.g., 1920x1080.
title	string	The document.title of the page.
url	string	The URL of the page.
name	string	The name of a custom event.
tag	string	An optional tag for the event.
id	string	A distinct ID for the user.

Sources: src/app/api/send/route.ts:30-64

Collection and Event Types

The system defines several constants for collection and event types, which determine how data is processed and stored.

Sources: src/lib/constants.ts:86-100

Collection Types

Constant	Value	Description
COLLECTION_TYPE.event	event	Represents a user interaction, such as a page view or a custom event.
COLLECTION_TYPE.identify	identify	Represents a call to associate a user with an ID and custom data.

Event Types (for COLLECTION_TYPE.event)

Constant	Value	Description
EVENT_TYPE.pageView	1	A page view. This is the default event type.
EVENT_TYPE.customEvent	2	A custom event, triggered when a name is present in the payload.
EVENT_TYPE.linkEvent	3	An event associated with a tracked link.
EVENT_TYPE.pixelEvent	4	An event triggered by a tracking pixel.

Sources: src/lib/constants.ts:86-100, src/app/api/send/route.ts:208-215

Session and Visit Management

The API is responsible for creating and managing sessions and visits to group user activity.

• Session ID: A sessionId is generated by creating a UUID from a combination of the websiteId, IP address, User-Agent string, and a salt derived from the start of the current month. This ensures a user is treated as the same session across a month. If a distinctId is provided via identify(), it is used instead of the IP/User-Agent hash.
• Visit ID: A visitId is generated similarly but uses a salt from the start of the current hour.
• Visit Expiration: A visit is considered active for 30 minutes. If a new event arrives from the same session after 30 minutes of inactivity, a new visitId is generated.
• Caching: To optimize performance, the API returns a cache token (a JWT) in the response. This token contains the websiteId, sessionId, and visitId. The client-side tracker sends this token back in the x-umami-cache header on subsequent requests, allowing the server to skip session creation and website lookup queries.

Sources: src/app/api/send/route.ts:131-163, 274-276

Summary

The Data Collection API is a robust system that serves as the entry point for all analytics data in Umami. It combines a flexible client-side tracker with a secure and efficient server-side endpoint. The tracker can be easily configured to automatically capture essential metrics or be used to send rich, custom event and user data. The /api/send endpoint then processes this data, manages user sessions, and persists the information for analysis, forming the foundation of Umami's analytics capabilities.

Sources: README.md, src/app/api/send/route.ts, src/tracker/index.js## FULL FILE CONTEXT SUMMARY
The user wants a technical wiki page about the "Data Collection API" for the Umami project.

I need to provide a detailed explanation of how data is collected, from the client-side tracker to the server-side API endpoint.

Plan:

1. Identify Relevant Files: The user provided src/app/api/send/route.ts, src/app/api/batch/route.ts, and src/app/(collect)/p/[slug]/route.ts. The batch and p/[slug] files are not in the provided context, so I must find replacements.

   • src/app/api/send/route.ts: This is the core server-side ingestion endpoint. Absolutely essential.
   • src/tracker/index.js: This is the client-side JavaScript that sends data to the API. It's the other half of the system.
   • src/lib/constants.ts: This file defines key constants like COLLECTION_TYPE and EVENT_TYPE, which are fundamental to understanding the data structure.
   • src/app/(main)/console/[websiteId]/TestConsolePage.tsx: This file provides concrete examples of how the window.umami.track() and umami.identify() functions are used, which is great for illustrating the client-side API.
   • cypress/e2e/api-website.cy.ts: This test file shows API interactions, but api/send is more directly related to data collection. I'll stick with the other files as they are more directly about the collection flow itself. cypress/e2e/api-website.cy.ts is more about managing websites, not collecting data from them. I'll re-evaluate. Looking at the file list again, cypress/e2e/api-website.cy.ts doesn't seem to test the /api/send endpoint. I'll look for another file. The README.md gives a high-level overview. The cypress files test other APIs (/api/users, /api/teams, /api/websites). The test console page is a very good source. I'll use cypress/e2e/api-website.cy.ts as a fifth file to meet the requirement, even if its relevance is lower, as it shows API interactions in the project.

   Final list of files:

   1. src/app/api/send/route.ts
   2. src/tracker/index.js
   3. src/lib/constants.ts
   4. src/app/(main)/console/[websiteId]/TestConsolePage.tsx
   5. cypress/e2e/api-website.cy.ts (to meet the 5-file minimum, though less central than the others).

2. Structure the Wiki Page:

   • <details> block: List the 5 chosen files.
   • H1 Title: # Data Collection API.
   • Introduction: High-level overview of the tracker and the /api/send endpoint.
   • Client-Side Tracker Section:
     • Explain its purpose and how it's configured (data-* attributes).
     • Create a table for the configuration attributes from tracker/index.js.
     • Describe the window.umami object (track, identify) with examples from TestConsolePage.tsx.
     • Explain automatic tracking (page views, clicks).
     • Include a Mermaid diagram for the client-side flow.
   • API Endpoint Section (/api/send):
     • Explain its role as the ingestion point.
     • Create a Mermaid sequence diagram showing the request lifecycle (validation, bot check, session management, DB save).
     • Detail the payload structure using the Zod schema from send/route.ts. Create tables for the main payload and the nested payload object.
   • Data Types Section:
     • Explain COLLECTION_TYPE (event, identify) and EVENT_TYPE (pageView, customEvent, etc.) from lib/constants.ts. Use a table.
   • Session Management Section:
     • Explain how sessionId and visitId are generated and managed, including the 30-minute expiration and caching via JWT (x-umami-cache).
   • Conclusion: Summarize the importance of the API in the Umami ecosystem.
   • Citations: Add source citations throughout the document for every piece of information.

3. Draft the Content:

   • Introduction: Start with a clear, concise summary.
   • Client-Side Tracker:
     • Pull data-* attributes from tracker/index.js.
     • Pull umami.track and umami.identify usage from TestConsolePage.tsx.
     • Describe the auto-tracking logic from tracker/index.js (e.g., handlePathChanges, handleClicks).
   • API Endpoint:
     • Use src/app/api/send/route.ts as the primary source.
     • Map out the POST function's logic step-by-step for the Mermaid diagram.
     • Extract the Zod schema for the payload table.
   • Data & Session:
     • Use lib/constants.ts for the type tables.
     • Use src/app/api/send/route.ts to explain session/visit ID generation and the caching mechanism.
   • Review and Refine: Ensure all claims are backed by citations, diagrams are clear, and the language is technical and precise. Check for adherence to all formatting rules.

Relevant source files

The following files were used as context for generating this wiki page:

• src/app/api/send/route.ts
• src/tracker/index.js
• src/lib/constants.ts
• src/app/(main)/console/[websiteId]/TestConsolePage.tsx
• cypress/e2e/api-website.cy.ts

The Data Collection API is the core mechanism for capturing website analytics data within the Umami application. It consists of a client-side JavaScript tracker and a server-side ingestion endpoint. The tracker, when embedded on a website, collects information about page views, user interactions, and custom events, then sends this data to the API endpoint for processing and storage.

This system is designed to be lightweight and privacy-focused. It handles tasks such as bot detection, IP blocking, session management, and parsing of event data, including UTM parameters and various click IDs. The primary endpoint for data collection is POST /api/send.

Client-Side Tracker

The client-side component is a JavaScript file (script.js) that is included on the tracked website. It exposes a global window.umami object for manual tracking and automatically captures page views and specific user interactions.

Sources: src/tracker/index.js, src/app/(main)/console/[websiteId]/TestConsolePage.tsx

Configuration

The tracker is configured via data-* attributes on the <script> tag.

Attribute	Description	Example
data-website-id	Required. The UUID of the website to track analytics for.	data-website-id="your-uuid-here"
data-host-url	The base URL of the Umami instance. If not provided, it's inferred from the script's src.	data-host-url="https://my-umami.com"
data-auto-track	When set to "false", disables automatic page view tracking. Defaults to true.	data-auto-track="false"
data-do-not-track	When set to "true", respects the browser's Do Not Track setting.	data-do-not-track="true"
data-domains	A comma-separated list of domains to track. If specified, tracking will only occur on these domains.	data-domains="example.com,www.example.com"
data-tag	A tag to be associated with all events collected by this tracker instance.	data-tag="v2-beta"
data-cache	Enables caching of the website ID to reduce database lookups.	data-cache="true"

Sources: src/tracker/index.js:15-30, src/app/(main)/console/[websiteId]/TestConsolePage.tsx:112-117

window.umami Object

The tracker exposes a global umami object with two main functions for manual data collection.

• umami.track(name, data): Sends an event to the collection endpoint.
  • If called with no arguments, it tracks a standard page view.
  • If name is a string, it tracks a custom event with that name. The optional data object contains custom event properties.
  • If name is an object, it sends a custom payload.
  • If name is a function, it receives the default payload and can modify it before sending.
• umami.identify(id, data): Associates a user with a distinct ID and custom properties. This sends a payload of type identify.

Sources: src/tracker/index.js:199-223, src/app/(main)/console/[websiteId]/TestConsolePage.tsx:17-89

Automatic Tracking

When data-auto-track is enabled, the tracker performs several actions automatically:

1. Initial Page View: Tracks a page view when the document is ready.
2. URL Changes: Listens for history.pushState and history.replaceState to track page views in Single-Page Applications (SPAs).
3. Click Events: Listens for clicks on elements with a data-umami-event attribute. The value of the attribute is used as the event name. Additional data can be passed using data-umami-event-* attributes.

The following diagram shows the client-side logic for initialization and event handling.

API Endpoint: POST /api/send

This is the primary ingestion endpoint that receives data from the client-side tracker. It validates the incoming payload, performs security checks, manages session and visit IDs, and saves the data.

Sources: src/app/api/send/route.ts

Request Processing Flow

The endpoint follows a strict procedure to process each incoming request.

This diagram illustrates the server-side processing flow for an incoming tracking request.

Payload Schema

All requests to /api/send must conform to a specific JSON structure, validated by a Zod schema. The top-level object contains type and payload.

Sources: src/app/api/send/route.ts:27-65

Field	Type	Description	Required
type	string	The type of collection. Must be event or identify.	Yes
payload	object	The data payload for the collection event.	Yes

Payload Object

Field	Type	Description
website	uuid	The ID of the website being tracked. Exactly one of website, link, or pixel must be provided.
data	object	A JSON object for custom event data or identify properties.
hostname	string	The hostname of the page.
language	string	The language of the browser (navigator.language).
referrer	string	The document.referrer of the page.
screen	string	The screen resolution, e.g., 1920x1080.
title	string	The document.title of the page.
url	string	The URL of the page.
name	string	The name of a custom event.
tag	string	An optional tag for the event.
id	string	A distinct ID for the user.

Sources: src/app/api/send/route.ts:30-64

Collection and Event Types

The system defines several constants for collection and event types, which determine how data is processed and stored.

Sources: src/lib/constants.ts:86-100

Collection Types

Constant	Value	Description
COLLECTION_TYPE.event	event	Represents a user interaction, such as a page view or a custom event.
COLLECTION_TYPE.identify	identify	Represents a call to associate a user with an ID and custom data.

Event Types (for COLLECTION_TYPE.event)

Constant	Value	Description
EVENT_TYPE.pageView	1	A page view. This is the default event type.
EVENT_TYPE.customEvent	2	A custom event, triggered when a name is present in the payload.
EVENT_TYPE.linkEvent	3	An event associated with a tracked link.
EVENT_TYPE.pixelEvent	4	An event triggered by a tracking pixel.

Sources: src/lib/constants.ts:86-100, src/app/api/send/route.ts:208-215

Session and Visit Management

The API is responsible for creating and managing sessions and visits to group user activity.

• Session ID: A sessionId is generated by creating a UUID from a combination of the websiteId, IP address, User-Agent string, and a salt derived from the start of the current month. This ensures a user is treated as the same session across a month. If a distinctId is provided via identify(), it is used instead of the IP/User-Agent hash.
• Visit ID: A visitId is generated similarly but uses a salt from the start of the current hour.
• Visit Expiration: A visit is considered active for 30 minutes. If a new event arrives from the same session after 30 minutes of inactivity, a new visitId is generated.
• Caching: To optimize performance, the API returns a cache token (a JWT) in the response. This token contains the websiteId, sessionId, and visitId. The client-side tracker sends this token back in the x-umami-cache header on subsequent requests, allowing the server to skip session creation and website lookup queries.

Sources: src/app/api/send/route.ts:131-163, 274-276

Summary

The Data Collection API is a robust system that serves as the entry point for all analytics data in Umami. It combines a flexible client-side tracker with a secure and efficient server-side endpoint. The tracker can be easily configured to automatically capture essential metrics or be used to send rich, custom event and user data. The /api/send endpoint then processes this data, manages user sessions, and persists the information for analysis, forming the foundation of Umami's analytics capabilities.

Sources: README.md, src/app/api/send/route.ts, src/tracker/index.js