Prompt Budget Checker Tool
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
The Prompt Budget Checker Tool is a critical CI-enforced utility within the Sploosh project designed to ensure that the language's core specification mirrors remain small enough to fit within an LLM's system prompt context. As an AI-native programming language, Sploosh prioritizes a "spec-first" approach where the language definition must be prompt-loadable alongside actual task context to ensure high first-shot accuracy from models like Claude, GPT, and Gemini.
The tool monitors two specific prompt-sized specification mirrors: the Language Core (_CORE) and the Web3 companion (_WEB3). It enforces strict token ceilings using the cl100k_base encoding (Tiktoken). This enforcement is a realization of Sploosh Design Principle §1.7, which mandates that the language core stay within specific economic and attention-quality constraints rather than expanding to fill frontier context windows.
Sources: VISION.md:46-51, README.md:14-19, scripts/check_prompt_budget.py:1-12
Core Logic and Architecture
The tool is implemented as a Python script (scripts/check_prompt_budget.py) that leverages the tiktoken library for accurate token counting. It operates by encoding the target Markdown files and comparing the resulting token count against pre-defined ceilings.
Token Counting Flow
The following diagram illustrates the execution flow of the budget checker script:
The script uses a three-tier rule for reporting: passing silently if well under budget, warning if approaching the limit (default 90%), and failing the build if the ceiling is exceeded.
Sources: scripts/check_prompt_budget.py:7-12, scripts/check_prompt_budget.py:35-42
Configuration and Ceilings
The tool defines specific ceilings for the two primary documentation mirrors. These values are calibrated to ensure prompt portability and attention quality.
| Target | File Path | Default Ceiling (Tokens) | Purpose |
|---|---|---|---|
| _CORE | docs/spec-plans/LANGUAGE_SPEC_PROMPT_CORE.md | 5,600 | Authoritative core language rules. |
| _WEB3 | docs/spec-plans/LANGUAGE_SPEC_PROMPT_WEB3.md | 1,500 | On-chain surface and smart contract rules. |
Sources: scripts/check_prompt_budget.py:21-23, scripts/check_prompt_budget.py:46-48, CONTRIBUTING.md:78-82
Command Line Arguments
The script supports customization of these limits via CLI arguments:
--core-ceiling: Sets the token limit for the core spec (Default: 5600).--web3-ceiling: Sets the token limit for the Web3 spec (Default: 1500).--warn-at: Threshold fraction to trigger a warning (Default: 0.9).
Sources: scripts/check_prompt_budget.py:46-50
Integration and Usage
The Prompt Budget Checker is integrated into the Sploosh development workflow in three ways:
- Continuous Integration (CI): The script runs automatically via GitHub Actions (specifically
prompt-budget.yml) to prevent any PR from merging that violates the token budget. - Local Validation: Developers are encouraged to run the script locally after modifying documentation to ensure compliance before pushing.
- Agent Interaction: Coding agents (like
cubic-dev-ai) are instructed to follow these constraints when suggesting spec amendments.
Local Execution
To run the tool locally, users must install the requirements and execute the script:
pip install -r scripts/requirements.txt
python scripts/check_prompt_budget.py
Sources: README.md:8-10, CONTRIBUTING.md:84-87, scripts/requirements.txt
Functional Implementation
The tool is designed to be allocation-free regarding the project's main Rust crates, relying on a lightweight Python environment.
def count_tokens(path: Path, enc: "tiktoken.Encoding") -> int:
return len(enc.encode(path.read_text(encoding="utf-8")))
def classify(tokens: int, ceiling: int, warn_at: float) -> str:
if tokens > ceiling:
return "fail"
if tokens >= ceiling * warn_at:
return "warn"
return "ok"
Sources: scripts/check_prompt_budget.py:26-33
Conclusion
The Prompt Budget Checker Tool is fundamental to Sploosh's identity as an AI-native language. By enforcing strict token limits on the specification mirrors, the tool ensures the language remains "prompt-loadable," preventing the documentation bloat common in legacy languages. This ensures that AI agents always have the full, authoritative context of the language within their active memory, facilitating one-shot code generation accuracy.
Sources: VISION.md:46-51, README.md:14-19