Browser & Frontend Skills
Relevant source files
The following files were used as context for generating this wiki page:
Introduction
Browser and Frontend Skills represent a specialized set of capabilities within the project designed to handle web automation, visual engineering, and UI/UX design. These skills enable the primary orchestrator, Sisyphus, to delegate tasks that require complex browser interactions, meticulous interface design, or multimodal analysis of visual media.
The system distinguishes between functional browser automation (via the dev-browser skill) and aesthetic visual engineering (via the frontend-ui-ux skill). While the former focuses on stateful web navigation and data extraction, the latter emphasizes high-fidelity design principles, typography, and motion. Additionally, the multimodal-looker agent provides a specialized tier for analyzing media such as PDFs and diagrams that exceed raw text interpretation.
Sources: sisyphus-prompt.md:32-34, src/agents/sisyphus.ts:16-17, src/features/builtin-skills/skills/frontend-ui-ux.ts:4-6
Dev Browser Skill
The dev-browser skill provides a stateful browser automation environment using Playwright. It is designed to maintain page state across multiple script executions, allowing the agent to perform incremental actions like logging in, navigating complex flows, and extracting data.
Automation Architecture
The system supports two primary modes of operation for browser control: Standalone Mode for fresh Chromium sessions and Extension Mode for connecting to a user's existing authenticated Chrome browser.
The diagram shows the interaction between the agent script, the client SDK, and the underlying browser relay server that maintains persistence.
Sources: src/features/builtin-skills/dev-browser/SKILL.md:21-48, src/features/builtin-skills/skills/dev-browser.ts:44-77
Implementation Modes
| Mode | Use Case | Setup Requirement |
|---|---|---|
| Standalone | Fresh automation sessions, testing web apps, scraping. | Run ./skills/dev-browser/server.sh |
| Extension | Interaction with authenticated sites, user-driven sessions. | Run npm run start-extension and activate Chrome extension |
Sources: src/features/builtin-skills/dev-browser/SKILL.md:25-48, src/features/builtin-skills/skills/dev-browser.ts:44-77
Element Discovery via ARIA Snapshots
Instead of relying solely on brittle CSS selectors, the skill utilizes getAISnapshot() to retrieve a YAML-formatted accessibility tree. This tree provides semantic references (ref=eN) that allow the agent to interact with elements based on their roles and labels.
- banner:
- link "Hacker News" [ref=e1]
- main:
- list:
- listitem:
- link "Article Title" [ref=e8]
Sources: src/features/builtin-skills/dev-browser/SKILL.md:128-144, src/features/builtin-skills/skills/dev-browser.ts:133-149
Frontend UI/UX Skill
The frontend-ui-ux skill is modeled as a "Designer-turned-Developer" role. It prioritizes aesthetic precision, micro-interactions, and spatial composition, ensuring that interfaces are not just functional but visually striking.
Design Principles and Process
The skill follows a strict process of committing to a bold aesthetic direction before implementing code. This avoids "AI slop" and generic layouts.
The process emphasizes intentionality over intensity, requiring the agent to pick a specific aesthetic (e.g., Brutalist, Art Deco, or Luxury) before coding.
Sources: src/features/builtin-skills/frontend-ui-ux/SKILL.md:29-45, src/features/builtin-skills/skills/frontend-ui-ux.ts:31-47
Aesthetic Guidelines
The skill mandates specific technical choices to ensure high-quality output:
- Typography: Avoids common system fonts like Arial or Inter; pairs distinctive display fonts with refined body text.
- Color: Uses cohesive CSS variable palettes; avoids purple-on-white gradients.
- Spatial Composition: Employs asymmetry, grid-breaking elements, and generous negative space.
- Motion: Focuses on high-impact moments (staggered reveals) rather than scattered micro-interactions.
Sources: src/features/builtin-skills/frontend-ui-ux/SKILL.md:53-75, src/features/builtin-skills/skills/frontend-ui-ux.ts:55-77
Multi-Model Orchestration
Tasks related to browser and frontend work are typically delegated to specialized models via the visual-engineering category.
Delegation Workflow
When the primary agent (Sisyphus) identifies a frontend task, it must evaluate and load relevant skills.
Specialized Agent: Multimodal Looker
For visual analysis tasks that require more than DOM scraping, the multimodal-looker (using Gemini 3 Flash) is utilized.
| Capability | Description |
|---|---|
| Media Analysis | Interpretation of PDFs, images, and complex diagrams. |
| Visual Validation | Verifying UI changes against screenshots or design mocks. |
| UI Extraction | Analyzing visual layouts to suggest styling improvements. |
Sources: sisyphus-prompt.md:33-35, src/agents/multimodal-looker.ts:14-16
Summary
The Browser & Frontend Skills ecosystem provides a comprehensive framework for both functional web automation and high-fidelity UI/UX design. By leveraging the dev-browser skill for stateful interaction and the frontend-ui-ux skill for aesthetic engineering, the project ensures that agents can operate effectively in complex web environments while maintaining production-grade design standards. All browser-related tasks are strictly verified through automated tools like Playwright to ensure zero human intervention is required for task completion.