Cost and Usage Tracking

Relevant source files

The following files were used as context for generating this wiki page:

• packages/models/src/pricing/pricing-service.ts
• packages/models/src/pricing/catalog.ts
• packages/core/src/utils/spending/pricing.ts
• packages/core/src/messages/api/vercel/pricing/vercelUsage.ts
• packages/core/src/messages/api/vercel/vercelStepProcessor.ts
• packages/mcp-server/README.md
• apps/web/src/app/(protected)/dev/components/PipelineTester.tsx
• packages/core/README.md
• apps/web/src/app/components/agent-dialog/purpose-section.tsx

The Cost and Usage Tracking system is a core component responsible for monitoring the financial cost of AI model interactions within the evolutionary workflow platform. Its primary purpose is to calculate and aggregate the cost of token usage for various AI models, providing essential metrics for performance analysis, budget management, and fitness evaluation in both cultural and genetic programming evolution modes.

This system relies on a centralized pricing catalog, a set of calculation utilities, and integration points within the workflow execution pipeline. It enables real-time cost tracking, which is crucial for optimizing workflows not just for accuracy and speed, but also for financial efficiency. The calculated costs are surfaced in developer tools and are a key factor in the multi-objective fitness functions that guide the evolution process.

Architecture and Core Components

The system is composed of three main parts: a pricing data source, a cost calculation engine, and integration points for tracking and display.

1. Pricing Catalog

The foundation of the cost tracking system is the MODELS_CATALOG, which serves as the single source of truth for all AI model information, including pricing.

Sources: packages/models/src/pricing/catalog.ts

Model Data Structure

Each model in the catalog is defined by the Model interface, which includes details about its provider, pricing, and operational status.

Field	Type	Description
id	string	Unique identifier for the model (e.g., openrouter#openai/gpt-4o).
provider	string	The service provider (e.g., openrouter, openai).
model	string	The specific model name (e.g., gpt-4o).
input	number	Cost per million input tokens in USD.
output	number	Cost per million output tokens in USD.
rpm	number	Rate limit in requests per minute.
tpm	number	Rate limit in tokens per minute.
context_window	number	The maximum context window size.
enabled	boolean	Whether the model is active for runtime use.
runtimeEnabled	boolean	A flag to indicate if the model is enabled in the current runtime environment.

Sources: packages/models/src/pricing/catalog.ts:10-23

Accessing Pricing Information

The getPricingForModel function is the primary utility for retrieving pricing data for a specific model from the catalog.

// packages/models/src/pricing/pricing-service.ts:8-12

export const getPricingForModel = (modelId: string) => {
  const model = MODELS_CATALOG.find(m => m.id === modelId)
  if (!model) return undefined
  return { input: model.input, output: model.output }
}

Sources: packages/models/src/pricing/pricing-service.ts:8-12

2. Cost Calculation

Cost calculation occurs after an AI model interaction is complete. The system uses the token usage data from the AI provider's response and the model's pricing from the catalog to determine the cost in USD.

Calculation Logic

The calculateUsageCost function is central to this process. It takes the usage object from a Vercel AI SDK StepResult and the model ID to compute the cost.

The following diagram illustrates the cost calculation flow.

The core calculation is as follows:

1. Retrieve the model's pricing (cost per million input/output tokens) from the MODELS_CATALOG.
2. Calculate the input cost: (inputTokens / 1,000,000) * inputCost.
3. Calculate the output cost: (outputTokens / 1,000,000) * outputCost.
4. Sum the input and output costs to get the total USD cost for the step.

// packages/core/src/messages/api/vercel/pricing/vercelUsage.ts:10-24
export const calculateUsageCost = (usage: CoreUsage | undefined, modelUsed: string) => {
  if (!usage) return 0

  const modelPricing = getPricingForModel(modelUsed)
  if (!modelPricing) return 0

  const { inputTokens, outputTokens } = usage
  const { input, output } = modelPricing

  const inputCost = (inputTokens / 1_000_000) * input
  const outputCost = (outputTokens / 1_000_000) * output

  const totalCost = inputCost + outputCost
  return totalCost
}

Sources: packages/core/src/messages/api/vercel/pricing/vercelUsage.ts:10-24

3. Integration into Workflow Processing

Cost calculation is integrated directly into the agent step processing logic. The processStepsV2 function iterates through the results of an agent's execution, calculates the cost for each step, and aggregates the total cost.

The sequence diagram below shows how a set of StepResult objects are processed to yield agent steps and a total cost.

This ensures that every AI interaction within a workflow is cost-tracked, and the total cost is available for logging, analysis, and display.

4. Usage and Cost Accumulation

While a specific SpendingTracker.ts file was not available for analysis, its function can be inferred from its usage in tests and mentions in documentation. The system is designed to accumulate costs over multiple operations or entire workflow invocations.

A test case for "cost accumulation" confirms this design pattern, where costs from individual operations are summed to provide a running total.

// packages/core/src/improvement/gp/__tests__/utils.test.ts:208-216
it("should track cost accumulation", () => {
  let totalCost = 0
  const operations = [0.01, 0.02, 0.015, 0.008]

  operations.forEach(cost => {
    totalCost += cost
  })

  expect(totalCost).toBeCloseTo(0.053, 3)
})

Sources: packages/core/src/improvement/gp/tests/utils.test.ts:208-216

The core module's README highlights "Real-time cost tracking" as a feature of the execution engine, indicating that costs are monitored as workflows run.

Sources: packages/core/README.md:329

Configuration

The system includes configuration options for monitoring API credit usage, which is related to overall cost management. These are set via environment variables.

Environment Variable	Default	Description
LUCKY_CREDIT_WARNING_THRESHOLD	1000	The credit level at which to issue a warning.
LUCKY_CREDIT_CRITICAL_THRESHOLD	100	The credit level at which to issue a critical alert.

Sources: packages/mcp-server/README.md:145-146

UI and Display

The calculated costs are surfaced in the application's UI, particularly in development and testing tools.

Pipeline Tester

The PipelineTester component displays the final cost of a pipeline run, formatted to four decimal places. This provides immediate feedback to developers on the cost implications of their workflow configurations.

// apps/web/src/app/(protected)/dev/components/PipelineTester.tsx:300-302
<div className="text-lg font-semibold">
  ${result.result.cost.toFixed(4)}
</div>

Sources: apps/web/src/app/(protected)/dev/components/PipelineTester.tsx:300-302

Agent Dialog

In some UI components, a simplified, hardcoded estimated cost is used as a placeholder, indicating the design intent to display cost information per agent or node.

// apps/web/src/app/components/agent-dialog/purpose-section.tsx:103-107
// Calculate estimated cost per run
const estimatedCost = useMemo(() => {
  // Simple estimate: ~1000 tokens in, ~500 tokens out on average
  // This should be calculated based on actual usage later
  return "$0.02"
}, [])

Sources: apps/web/src/app/components/agent-dialog/purpose-section.tsx:103-107

Summary

The Cost and Usage Tracking system is an essential observability feature of the lucky platform. By leveraging a centralized pricing catalog and integrating cost calculations directly into the workflow processing pipeline, it provides accurate, real-time financial data. This data is critical for developers to understand the economic impact of their agentic workflows and is a fundamental input for the evolutionary algorithms that automatically optimize these workflows for cost-efficiency.