Full Bundle vs Per-Page Markdown
Relevant source files
The following files were used as context for generating this wiki page:
The claude-code-docs project provides two primary methods for consuming Claude Code documentation: a single "full bundle" and individual "per-page" Markdown files. This dual-structure is designed to optimize context window usage for AI agents, allowing them to choose between a comprehensive but large text blob and specific, granular pages retrieved on-demand.
Sources: README.md:1-10, skills/claude-code-docs/SKILL.md:1-5
Architectural Overview
The system mirrors content from the canonical documentation at code.claude.com/docs. It uses a synchronization script to maintain two distinct formats within the repository: a single bundled file and a structured directory of individual files.
Synchronization Logic
The sync.ts script treats llms.txt as the source of truth for the documentation manifest. It performs the following high-level operations:
- Fetches
llms.txtto identify all valid documentation pages. - Fetches
llms-full.txtto create the full bundle. - Recursively rebuilds the
docs/directory by fetching individual.mdfiles listed in the manifest.
Sources: sync.ts:1-12, README.md:14-17
This flowchart illustrates the data acquisition and distribution process:
The diagram shows how the synchronization process splits the source documentation into a single bundle and a directory of individual pages.
Sources: sync.ts:51-64, README.md:5-9
Full Bundle: llms-full.txt
The full bundle is a single, large Markdown file containing the entire documentation set. While comprehensive, its size (approximately 5MB) makes it inefficient for direct injection into AI context windows.
Characteristics and Use Cases
- Source: Mirrored directly from
https://code.claude.com/docs/llms-full.txt. - Storage: Located at the root of the repository as
llms-full.txt. - Constraint: AI agents are encouraged to skip indexing this file for semantic search to avoid duplicating results found in the
docs/folder.
Sources: README.md:6, README.md:44-46, sync.ts:54
Per-Page Markdown: docs/ Directory
The docs/ directory contains individual Markdown files for every page found in the documentation manifest. This structure supports "lazy" access, where only relevant snippets or pages are provided to the AI agent.
Data Transformation
When pages are downloaded for the docs/ directory, the system rewrites site-absolute links to relative file paths. This ensures that cross-links between documentation pages work correctly when browsing the mirror locally or on platforms like GitHub.
| Feature | Implementation |
|---|---|
| Link Rewriting | rewriteLinks function converts /en/slug to ../slug.md |
| Path Resolution | Uses node:path/posix to calculate relative paths from page depth |
| Anchor Handling | Percent-encoded fragments (e.g., %2F) are cleaned for GitHub compatibility |
Sources: sync.ts:17-41, sync.ts:68-75
Retrieval Flow
For AI agents using tools like ASK (Agent Skills Kit), the per-page structure enables semantic search without loading the entire 5MB bundle.
The sequence diagram demonstrates how the system avoids the full bundle by performing semantic search over individual granular files.
Sources: README.md:37-46, skills/claude-code-docs/SKILL.md:27-31
Feature Comparison
The following table summarizes the differences between the two storage strategies:
| Attribute | Full Bundle (llms-full.txt) | Per-Page (docs/) |
|---|---|---|
| Context Load | High (~5MB) | Low (Individual small pages) |
| Search Method | Linear / Grep | Semantic (via csp) |
| File Structure | Single file | Hierarchical directory |
| Link Integrity | Maintains original structure | Rewritten for local navigation |
| Tooling Support | WebFetch fallback | ASK, CSP, and Skill-based retrieval |
Sources: README.md:5-9, README.md:37-46, sync.ts:17-41, skills/claude-code-docs/SKILL.md:27-42
Dynamic URL Rewriting (Hooks)
The repository includes a Claude Code plugin that dynamically interacts with the "Per-Page" philosophy. A PreToolUse hook intercepts requests to the documentation website and redirects them to the clean Markdown version of that specific page.
# Example of the rewrite logic in hooks/rewrite-docs-url.sh
# Input: https://code.claude.com/docs/en/agents
# Output: https://code.claude.com/docs/en/agents.md
Sources: hooks/rewrite-docs-url.sh:25-33, hooks/hooks.json:1-15
This mechanism ensures that even when an agent attempts to use WebFetch on a standard URL, it receives the granular Markdown source rather than the full HTML application shell or the entire bundle.
Sources: README.md:51-62, hooks/rewrite-docs-url.test.sh:19-25
Conclusion
The distinction between the Full Bundle and Per-Page Markdown is central to the project's goal of providing efficient documentation access to AI agents. By prioritizing the docs/ directory for semantic search and providing a PreToolUse hook for URL rewriting, the system ensures agents consume only the minimum necessary context while maintaining the ability to fall back to the complete llms-full.txt bundle when required.
Sources: README.md:21-25, skills/claude-code-docs/SKILL.md:7-14