Model Selection and Tiering

Relevant source files

The following files were used as context for generating this wiki page:

• packages/core/src/workflow/actions/generate/formalizeWorkflow.ts
• packages/shared/src/contracts/models.ts
• packages/models/examples/tier-resolution.ts
• packages/core/README.md
• packages/models/examples/using-configs.ts
• apps/web/src/app/components/agent-dialog/core-config-section.tsx
• apps/web/src/lib/workflow_setup_examples/my-workflow-judge.json

The Model Selection and Tiering system provides a flexible and abstract way to manage and utilize various AI models within the platform. It decouples the application logic from specific model implementations, allowing for dynamic, cost-effective, and performance-driven model choices. This is achieved through two primary mechanisms: a semantic tiering system (e.g., nano, medium, high) and user-specific model configurations.

This system enables developers and users to define model preferences based on task requirements, such as speed, cost, or reasoning capabilities, without hardcoding model names. The framework then resolves these abstract requests into concrete model instances at runtime, supporting strategies like fallbacks, racing, and first-available.

Core Concepts and Data Structures

The foundation of the model system is the ModelEntry schema, which provides a standardized structure for defining every model's properties.

Sources: packages/shared/src/contracts/models.ts

ModelEntry Schema

The modelEntrySchema is the single source of truth for all model definitions, capturing identity, pricing, capabilities, and performance metrics.

The diagram below shows the key Zod schemas that constitute a model's definition.

The table below details the most critical fields in the ModelEntry object.

Field	Type	Description
id	string	Catalog lookup ID in the format <provider>#<model>. Used for internal lookup, not for API calls.
provider	string	The name of the provider (e.g., openai, openrouter), determining which API endpoint to use.
model	string	The model identifier in the provider-specific format. This is the value used for API calls.
input / output	number	Cost per 1 million tokens in USD for input and output.
speed	ModelSpeed	An enum (fast, medium, slow) indicating the model's relative speed.
intelligence	number	A rating from 1 to 10 indicating the model's reasoning capability.
pricingTier	ModelPricingTier	An enum (low, medium, high) categorizing the model's cost.
runtimeEnabled	boolean	If true, the model is eligible for selection during workflow execution.
uiHiddenInProd	boolean	If true, the model is hidden from the provider model discovery UI in production.

Sources: packages/shared/src/contracts/models.ts:46-118

Model Tiering System

The system uses a cost-optimized, three-tier model selection strategy to balance performance and expense. These tiers provide a semantic layer, allowing workflows to request a model based on the complexity of the task rather than a specific model name.

Sources: packages/core/README.md, packages/models/examples/tier-resolution.ts

The available tiers are:

• Nano: For fast, low-cost operations like summaries and validation.
• Medium: For standard processing and workflow improvement tasks.
• High: For complex reasoning and demanding operations like genetic crossover.

Tier Configuration

Tiers are defined in the createModels configuration. Each tier can specify a selection strategy (race, first, fallback, consensus) and a list of models.

// Example from packages/models/examples/tier-resolution.ts
const models = createModels({
  // ... providers
  tiers: {
    // Fast tier - use fastest models
    fast: {
      strategy: "race",
      models: [
        { provider: "openrouter", model: "openrouter#google/gemini-2.5-flash-lite" },
        { provider: "openai", model: "gpt-4o-mini" },
      ],
    },
    // Medium tier - balanced quality/speed
    medium: {
      strategy: "first",
      models: [{ provider: "openrouter", model: "openrouter#openai/gpt-4.1-mini" }],
    },
    // High tier - best quality
    high: {
      strategy: "first",
      models: [{ provider: "openrouter", model: "openrouter#openai/gpt-4.1" }],
    },
  },
  defaultTier: "medium",
  // ...
})

Sources: packages/models/examples/tier-resolution.ts:11-40

A defaultTier can be specified, which is used when a model request does not specify a tier.

Sources: packages/models/examples/tier-resolution.ts:39-39

Model Selection Strategies

When a workflow is generated or updated, a ModelSelectionStrategy determines how model names in the workflow configuration are resolved.

Sources: packages/core/src/workflow/actions/generate/formalizeWorkflow.ts:133-140

Tier Strategy

This is the default strategy. When a node's modelName is a tier keyword (e.g., "low", "medium", "high"), the system maps it to the corresponding model defined in the global tier configuration. The handleWorkflowCompletionTierStrategy function is responsible for this logic.

Sources: packages/core/src/workflow/actions/generate/formalizeWorkflow.ts:135-137

User-Models Strategy

This strategy allows for dynamic model selection from a list of models provided by the user at runtime. It is used to map a tier keyword to the best-suited model from the user's available list. The pickUserModelForTier function implements this policy.

The selection process is as follows:

1. Filter the user's models by the requested pricingTier and ensure they are runtimeEnabled.
2. Limit the filtered list to the top 5 candidates to ensure performance.
3. Rank the candidates by a score calculated from intelligence * 10 + speedScore, where speedScore is 3 for fast, 2 for medium, and 1 for slow.
4. The top-ranked model is selected.
5. If no model matches the tier, it falls back to the best overall model from the user's list.
6. If the user's list is empty or has no enabled models, it falls back to the system's default model.

This flow is visualized in the diagram below.

User-Specific Configurations

In addition to global tiers, the system supports loading user-specific configurations. This allows different users or contexts (e.g., research vs. production) to have entirely separate model definitions, experiments, and strategies.

User configs are loaded via models.loadUserConfig(). Models from these configs are requested using the format user:userId:experimentName.

// Example from packages/models/examples/using-configs.ts

// Load a config for 'researcher-1'
await models.loadUserConfig("researcher-1", researcherConfigPath)

// Use the 'quick' experiment from that user's config
const quickModel = await models.model("user:researcher-1:quick", {
  userId: "researcher-1",
  // ...
});

// If the experiment name is omitted, the 'defaults.experiment' from the config is used
const defaultModel = await models.model("user:researcher-1", {
  userId: "researcher-1",
  // ...
});

Sources: packages/models/examples/using-configs.ts:47-83

Integration

Workflow Node Configuration

Within a workflow definition, each node specifies its desired model via the modelName property. This can be a concrete model ID or a tier keyword.

// Example from apps/web/src/lib/workflow_setup_examples/my-workflow-judge.json
{
  "nodeId": "requirement_analyzer",
  "description": "Analyzes the task requirements...",
  "modelName": "openrouter#openai/gpt-4.1-mini",
  "systemPrompt": "You are a requirement analysis specialist...",
  "handOffs": ["task_output_analyzer"]
}

Sources: apps/web/src/lib/workflow_setup_examples/my-workflow-judge.json:15-25

The normalizeWorkflowModels function is responsible for resolving these names into canonical, provider-specific model IDs before execution, applying the selected ModelSelectionStrategy.

Sources: packages/core/src/workflow/actions/generate/formalizeWorkflow.ts:182-215

UI Integration

In the web interface, the agent configuration dialog presents a dropdown of all runtimeEnabled models.

• The list is populated by getActiveModelNames().
• The display name is cleaned by stripping the provider prefix (e.g., openai#gpt-4o-mini becomes gpt-4o-mini).
• When a model is selected, its input and output pricing per 1M tokens is displayed.

Sources: apps/web/src/app/components/agent-dialog/core-config-section.tsx:24-61

Conclusion

The Model Selection and Tiering system is a cornerstone of the project's flexibility. By abstracting model definitions through tiers and user-specific configurations, it allows the platform to adapt to different use cases, manage costs effectively, and easily integrate new models without requiring changes to the core workflow logic. The clear separation between abstract requests (tiers, user experiments) and concrete model resolution enables robust, maintainable, and cost-aware AI-powered workflows.