Database Schema Overview

Relevant source files

The following files were used as context for generating this wiki page:

• eval_platform/models.py
• eval_platform/db.py
• eval_platform/api/endpoints/evaluations.py
• eval_platform/schemas/evaluation.py
• README.md

The Database Schema Overview provides a detailed blueprint of the data structures and relationships within the eval-platform. The database stores information regarding language model evaluations, including datasets, model configurations, and resulting metrics.

This system uses a relational database to track the lifecycle of evaluation tasks. It manages the connection between input parameters, the execution status of evaluations, and the quantitative outputs produced by different evaluation frameworks.

Core Data Models

The platform defines several key entities to manage evaluation metadata and results. These models are implemented using an Object-Relational Mapper (ORM) to interface with the underlying database.

Evaluation Model

The Evaluation model serves as the central entity. It records the specific instance of an evaluation run, linking datasets to models and capturing execution progress.

Field	Type	Description
id	UUID	Primary key for the evaluation.
status	String	Current state (e.g., PENDING, RUNNING, COMPLETED, FAILED).
model_name	String	The identifier of the AI model being evaluated.
dataset_id	String	Reference to the dataset used for the evaluation.
created_at	DateTime	Timestamp when the evaluation was initialized.
completed_at	DateTime	Timestamp when the evaluation finished.

Sources: eval_platform/models.py:10-25, eval_platform/schemas/evaluation.py:5-15

Metrics Model

The Metric model stores the numerical outcomes of an evaluation. Each evaluation can have multiple metrics associated with it, such as accuracy, F1 score, or latency.

Field	Type	Description
id	Integer	Primary key for the metric record.
evaluation_id	UUID	Foreign key referencing the parent Evaluation.
name	String	The name of the metric (e.g., "exact_match").
value	Float	The numerical result of the metric.

Sources: eval_platform/models.py:30-40

Entity Relationships

The following diagram illustrates the relationships between the primary database entities. An Evaluation acts as a parent container for multiple Metrics.

The diagram shows a one-to-many relationship where one evaluation record correlates to zero or more metric records.
Sources: eval_platform/models.py:10-45

Data Flow and Persistence

The application manages database connections and sessions through a centralized utility. It initializes the engine and provides a session generator for API endpoints to perform CRUD operations.

Connection Management

The system configures the database engine using environment variables. It ensures that the schema is created if it does not exist during application startup.

# eval_platform/db.py
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

engine = create_engine(DATABASE_URL)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)

def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()

Sources: eval_platform/db.py:5-20

Evaluation Lifecycle Flow

The sequence below demonstrates how data is persisted when a user triggers a new evaluation through the API.

Configuration and Initialization

The database layer relies on specific configurations to ensure data integrity and connectivity.

• Migrations: The schema is managed via declarative base classes, allowing for automatic table generation.
• Validation: Pydantic schemas validate data before it reaches the database models, ensuring that required fields like model_name and dataset_id are present.

Sources: eval_platform/models.py:5-8, eval_platform/schemas/evaluation.py:20-25

Summary

The eval-platform database schema focuses on tracking the execution and results of LLM evaluations. By structuring the relationship between an Evaluation and its Metrics, the system provides a queryable history of model performance across various datasets. The use of UUIDs for primary keys ensures unique identification across distributed evaluation runs, while the status field allows for real-time tracking of asynchronous tasks.