REST API Reference
Relevant source files
The following files were used as context for generating this wiki page:
The eval-platform REST API provides a programmatic interface for managing evaluation tasks and model configurations. It allows users to submit evaluation requests, monitor progress, and retrieve results for various Large Language Model (LLM) performance metrics.
The API follows RESTful principles and uses JSON for request and response bodies. It integrates with the core evaluation engine to execute benchmarks across different model architectures.
API Architecture and Routing
The application uses FastAPI to manage routing and request handling. The base path for all version 1 endpoints is /api/v1.
The diagram shows the request flow from a client through the FastAPI router to specific endpoint modules.
Sources: main.py:1-25, api/v1/endpoints/eval.py:1-10
Evaluation Endpoints
The /eval endpoints manage the lifecycle of an evaluation job. Users submit configurations defining which models and datasets to evaluate.
Submit Evaluation Task
POST /api/v1/eval/submit
Starts a new evaluation process based on the provided configuration schema.
| Parameter | Type | Description |
|---|---|---|
model_id | String | Unique identifier of the model to test |
dataset_name | String | Name of the evaluation dataset |
metrics | List | List of metrics to compute (e.g., accuracy, bleu) |
Sources: api/v1/endpoints/eval.py:15-40, schemas/eval.py:5-20
Get Task Status
GET /api/v1/eval/status/{task_id}
Retrieves the current state and results of a specific evaluation task.
The sequence shows the polling mechanism used to track background evaluation tasks.
Sources: api/v1/endpoints/eval.py:45-60
Model Management Endpoints
The /models endpoints provide access to the registry of supported models.
List Supported Models
GET /api/v1/models
Returns a list of all models currently available for evaluation in the system.
| Field | Type | Description |
|---|---|---|
id | String | Internal model reference ID |
name | String | Display name of the model |
provider | String | Model source (e.g., HuggingFace, OpenAI) |
Sources: api/v1/endpoints/models.py:10-30
Configuration and Security
The API behavior is governed by environment variables and security settings defined in the core configuration.
| Variable | Default | Description |
|---|---|---|
API_V1_STR | /api/v1 | Base prefix for API routes |
PROJECT_NAME | eval-platform | Application name used in documentation |
ACCESS_TOKEN_EXPIRE_MINUTES | 11520 | JWT token validity duration |
Sources: core/config.py:5-25
Conclusion
The REST API serves as the primary gateway for the eval-platform, enabling automated LLM benchmarking. By providing structured endpoints for task submission and status tracking, it supports integration into CI/CD pipelines and external dashboards.