Seeding Data for Development

Relevant source files

The following files were used as context for generating this wiki page:

• scripts/seed/index.ts
• scripts/seed/generators/sessions.ts
• scripts/seed/sites/blog.ts
• scripts/seed-data.ts
• prisma/schema.prisma
• scripts/seed/sites/saas.ts
• scripts/seed/generators/events.ts
• scripts/seed/generators/revenue.ts

The Umami Sample Data Generator is a command-line tool designed to populate the database with realistic analytics data for local development and testing purposes. This utility is crucial for developers to test application features on a dataset that mimics real-world usage, without needing to generate traffic manually. The script is explicitly blocked from running in production or cloud environments to prevent accidental data corruption.

The seeder creates two pre-configured demo websites, each with distinct traffic patterns: a low-traffic "Demo Blog" and an average-traffic "Demo SaaS" site that includes revenue data. This allows for testing a range of scenarios, from basic page views to more complex event and e-commerce tracking. The amount of historical data generated can be configured by the user.

Sources: scripts/seed-data.ts:5-12, 85-96, scripts/seed/index.ts:31-43

Usage

The data seeding script is executed via an npm command. It supports several command-line arguments to customize its behavior.

Sources: scripts/seed-data.ts:14-20

# Generate 30 days of data (default)
npm run seed-data

# Generate 90 days of data
npm run seed-data -- --days 90

# Clear existing demo data before generating new data
npm run seed-data -- --clear

# Show detailed progress during generation
npm run seed-data -- --verbose

Sources: scripts/seed-data.ts:14-20

Configuration Options

The script's behavior can be modified using the following arguments:

Argument	Alias	Description	Default
--days <number>		Number of past days of data to generate.	30
--clear		If present, clears all existing data for the demo websites before seeding.	false
--verbose	-v	If present, shows detailed progress logs, including batch insertion counts.	false
--help	-h	Shows the help message and exits.	N/A

Sources: scripts/seed-data.ts:24-54

Architecture and Execution Flow

The seeding process is orchestrated by the main seed function in scripts/seed/index.ts, which is called from the scripts/seed-data.ts entry point. The process involves environment checks, configuration parsing, database connection, optional data clearing, and iterative data generation for each defined site.

Sources: scripts/seed-data.ts:98-107, scripts/seed/index.ts:300-369

The overall execution flow is as follows:

This diagram shows the main steps of the seed script, from argument parsing to the final summary.

Sources: scripts/seed-data.ts:98-107, scripts/seed/index.ts:300-369

Key Components

• seed-data.ts: The main entry point. It handles parsing command-line arguments and calling the core seed function. It also performs critical environment checks to prevent execution in production.
• seed/index.ts: Contains the core logic for the seeding process. It manages the overall workflow, including database interactions and orchestrating data generation for each site.
• Site-specific modules (sites/blog.ts, sites/saas.ts): These files define the configuration for each demo website, including name, domain, traffic volume (sessionsPerDay), and specific user journey patterns.
• Generator modules (generators/*.ts): These modules are responsible for creating specific data types like sessions, events, and revenue, based on the site configurations.

Sources: scripts/seed-data.ts, scripts/seed/index.ts, scripts/seed/sites/blog.ts, scripts/seed/sites/saas.ts

Data Generation Process

Data generation is handled by the generateSiteData function. For each configured demo site, it performs a series of steps to create and link related analytics data.

Sources: scripts/seed/index.ts:200-269

The process for generating data for a single site is as follows:

This diagram illustrates the nested loops for days and sessions to generate a comprehensive dataset for a single website.

Sources: scripts/seed/index.ts:200-269

Demo Site Configurations

The seeder generates data for two predefined websites with different characteristics to simulate a varied environment for testing.

Property	Demo Blog	Demo SaaS
Name	Demo Blog	Demo SaaS
Domain	demo-blog.umami.is	demo-saas.umami.is
Traffic	Low (~90 sessions/month)	Average (~500 sessions/day)
Revenue Data	No	Yes

Sources: scripts/seed-data.ts:80-81, scripts/seed/index.ts:31-43, scripts/seed/sites/blog.ts:5-7, scripts/seed/sites/saas.ts:5-7, 10

Database Interaction

To efficiently insert large volumes of data, the script collects all generated records (sessions, events, etc.) in memory and then uses batch insertion functions to write them to the database. The BATCH_SIZE is set to 1000 records per transaction.

Sources: scripts/seed/index.ts:45, 250-265

The following functions handle batch insertions into their respective tables:

• batchInsertSessions
• batchInsertEvents
• batchInsertEventData
• batchInsertRevenue

Sources: scripts/seed/index.ts:56-117

Data Clearing

If the --clear flag is used, the clearDemoData function is executed before any new data is generated. It finds websites matching the demo site names and deletes all associated data in an order that respects foreign key constraints to avoid database errors.

Sources: scripts/seed/index.ts:153-186

Database Schema

The seeder populates several tables in the database. The relationships between these tables are defined in the Prisma schema and are crucial for the integrity of the analytics data.

The following ER diagram shows the primary tables populated by the seeder and their relationships.

The key models involved are:

Website

Represents a website being tracked.

Field	Type	Description
id	String	Unique identifier (UUID).
name	String	The display name of the website.
domain	String	The domain of the website.
userId	String	Foreign key to the User who owns the website.

Sources: prisma/schema.prisma:79-90

Session

Represents a single user session.

Field	Type	Description
id	String	Unique identifier (UUID).
websiteId	String	Foreign key to the Website.
browser	String	Browser used in the session.
os	String	Operating system used.
device	String	Device type (e.g., desktop, mobile).
country	String	Two-letter country code.

Sources: prisma/schema.prisma:35-51

WebsiteEvent

Represents an event, such as a page view or a custom event.

Field	Type	Description
id	String	Unique identifier (UUID).
websiteId	String	Foreign key to the Website.
sessionId	String	Foreign key to the Session.
urlPath	String	The path of the URL where the event occurred.
eventType	Int	Type of event (e.g., 1 for pageview).
eventName	String	Name of the custom event.

Sources: prisma/schema.prisma:106-131

EventData

Stores custom data associated with a WebsiteEvent.

Field	Type	Description
id	String	Unique identifier (UUID).
websiteEventId	String	Foreign key to the WebsiteEvent.
dataKey	String	The key for the custom data field.
stringValue	String	The string value of the data.
numberValue	Decimal	The numerical value of the data.
dataType	Int	The type of data stored (string, number, etc.).

Sources: prisma/schema.prisma:151-163

Revenue

Stores revenue data associated with an event.

Field	Type	Description
id	String	Unique identifier (UUID).
websiteId	String	Foreign key to the Website.
sessionId	String	Foreign key to the Session.
eventName	String	The name of the event that generated revenue.
revenue	Decimal	The revenue amount.
currency	String	The currency code (e.g., 'USD').

Sources: prisma/schema.prisma:243-254

Summary

The data seeding feature is an essential tool for the Umami project's development lifecycle. It provides a standardized and efficient way to create a populated, realistic database environment. By simulating different traffic levels and data types, including page views, custom events, and revenue, it enables developers to thoroughly test, debug, and validate application functionality across a variety of scenarios, ensuring the robustness and correctness of the analytics platform.

Sources: scripts/seed-data.ts:5-12, README.md