Leaderboards & Analytics
Relevant source files
The following files were used as context for generating this wiki page:
Leaderboards & Analytics provides a centralized system for ranking models based on performance metrics across various datasets. This module aggregates evaluation results to offer comparative insights into model efficacy and historical performance trends.
The system processes raw evaluation outputs into structured rankings. Users view these rankings through a web interface that supports filtering by task type, model architecture, and evaluation dates.
Sources: README.md, frontend/src/views/leaderboard/index.vue
System Architecture
The leaderboard architecture follows a traditional three-tier structure. The frontend requests ranked data, the backend API processes these requests via CRUD operations, and the database stores persistent model scores and metadata.
This diagram illustrates the data flow from the user interface through the backend services to the data persistence layer.
Sources: backend/app/api/v1/endpoints/leaderboard.py, backend/app/crud/crud_leaderboard.py
Data Models and Schemas
The system uses specific Pydantic schemas and SQLAlchemy models to manage leaderboard entries. Each entry represents a unique combination of a model, a dataset, and the resulting performance score.
Leaderboard Entry Fields
| Field | Type | Description |
|---|---|---|
id | Integer | Unique identifier for the leaderboard record. |
model_name | String | Name of the evaluated model. |
dataset_id | Integer | Foreign key linking to the source dataset. |
score | Float | Primary performance metric (e.g., Accuracy, F1). |
created_at | DateTime | Timestamp of when the evaluation result was recorded. |
Sources: backend/app/models/leaderboard.py:10-25, backend/app/schemas/leaderboard.py:5-15
Backend API Endpoints
The backend exposes RESTful endpoints to retrieve and filter leaderboard data. These endpoints support pagination and sorting to handle large datasets efficiently.
Get Leaderboard Rankings
GET /api/v1/leaderboard/
Retrieves a list of ranked models based on specific query parameters.
| Parameter | Type | Required | Description |
|---|---|---|---|
dataset_id | int | No | Filter rankings by a specific dataset. |
limit | int | No | Number of records to return (default: 100). |
offset | int | No | Number of records to skip for pagination. |
sort_by | string | No | Metric to sort by (default: score). |
Sources: backend/app/api/v1/endpoints/leaderboard.py:15-40
Leaderboard Logic and Scoring
The CRUDLeaderboard class handles the business logic for calculating ranks. It performs SQL aggregations to determine the relative position of a model performance record within a specific dataset category.
The sequence shows how raw scores are transformed into a ranked list for display.
Sources: backend/app/crud/crud_leaderboard.py:20-45
Frontend Presentation
The frontend utilizes Vue.js components to render the leaderboard table. It includes dynamic columns that adjust based on the selected task or dataset type.
Key Components
- LeaderboardTable: Displays ranked rows with model names, scores, and rank change indicators.
- FilterPanel: Allows users to toggle between different evaluation metrics and timeframes.
Sources: frontend/src/views/leaderboard/index.vue:5-120
Summary
Leaderboards & Analytics serves as the primary evaluation visualization tool for the platform. It transforms discrete evaluation results into actionable comparative data, enabling developers to track model progress and benchmark against state-of-the-art results. The system ensures data integrity through structured schemas and provides efficient retrieval via a dedicated API layer.