Experiment Runner UI
Relevant source files
The following files were used as context for generating this wiki page:
The Experiment Runner UI provides a web-based interface for configuring, launching, and monitoring evaluation experiments within the platform. It enables users to select datasets, define model parameters, and track the real-time execution status of evaluation jobs.
This module acts as the primary bridge between the user and the backend execution engine. It manages the state of active runs and visualizes metrics generated during the evaluation process.
Architecture and Data Flow
The Experiment Runner UI follows a client-server architecture. The frontend components interact with the backend API to persist configurations and trigger execution.
User Interaction Flow
When a user initiates an experiment, the UI validates the input parameters before sending a request to the backend. The backend creates an experiment record in the database and initiates the execution task.
The flow demonstrates the transition from user configuration to asynchronous task execution.
Sources: frontend/src/components/ExperimentRunner.tsx, backend/app/api/experiments.py
Component Structure
The interface is composed of several functional sections that handle specific parts of the experiment lifecycle.
Configuration Panel
The Configuration Panel allows you to define the variables for a new run. You must specify the target model, the evaluation dataset, and any specific hyperparameters.
| Field | Type | Description |
|---|---|---|
| Model ID | String | The unique identifier for the model being evaluated. |
| Dataset | Selection | The dataset used for the evaluation run. |
| Batch Size | Integer | The number of samples processed in a single iteration. |
| Parameters | JSON | Optional model-specific parameters. |
Sources: frontend/src/components/ExperimentRunner.tsx, backend/app/schemas/experiment.py
Execution State Management
The UI tracks the lifecycle of an experiment through specific states. The useExperiment hook manages the transitions between these states by polling the backend API.
The diagram shows the standard state transitions for an evaluation task.
Sources: frontend/src/hooks/useExperiment.ts, backend/app/models/experiment.py
API Integration
The frontend communicates with the backend via RESTful endpoints. The following endpoints are critical for the Experiment Runner UI operation:
POST /api/experiments: Creates a new experiment configuration and starts the run.GET /api/experiments/{id}: Retrieves the current status and metrics for a specific experiment.GET /api/experiments: Lists all historical and active experiment runs.
Data Schema
Experiments are represented by a structured data model that ensures consistency between the frontend and the database.
class ExperimentBase(BaseModel):
name: str
model_id: str
dataset_id: str
config: Optional[dict] = None
class ExperimentCreate(ExperimentBase):
pass
class Experiment(ExperimentBase):
id: int
status: str
created_at: datetime
Sources: backend/app/schemas/experiment.py:5-20
Implementation Details
The ExperimentRunner component utilizes React hooks to handle side effects and state updates.
Hook: useExperiment
The useExperiment hook abstracts the logic for fetching data and managing execution triggers. It provides the following properties:
runExperiment: Function to trigger a new run.status: The current execution status (e.g., "running", "idle").results: The output data once the experiment completes.
Sources: frontend/src/hooks/useExperiment.ts:10-45
Component: ExperimentRunner
This React component renders the form and the status dashboard. If an experiment is active, it displays a progress indicator derived from the backend status.
Sources: frontend/src/components/ExperimentRunner.tsx:15-80
The Experiment Runner UI centralizes the control and visibility of evaluation tasks. By integrating directly with the backend task runner, it provides a responsive environment for iterative model testing and validation.