[Feature] shadcn-audit: check component PRs against shadcn Base UI + vega

Merged
CI
4d ago
+5339-2

Related issue

No issue. Started from reviewing #572, where checking a PR against shadcn by hand was too slow to actually happen.

Description

RubyUI mirrors shadcn/ui, but nothing checked that on PRs. This adds /shadcn-audit <PR | branch | component>, a Claude Code skill backed by a small Ruby CLI, that compares the components a PR touches with shadcn and leaves a checklist on the PR.

Which shadcn

The reasoning is in design/2026-10-26-shadcn-audit-design.md.

  • Behavior, HTML, ARIA and props: the Base UI base (apps/v4/registry/bases/base/ui/), shadcn's default since July 2026. React Aria is only a second opinion for accessibility.
  • Look: the vega style (styles/style-vega.css), which shadcn calls "the classic shadcn/ui look" and is closest to RubyUI today. Not nova (the create default) and not new-york-v4 (legacy).
  • Theme: neutral. RubyUI's tokens already match it except chart-1..5.

What a run does

  1. Pins the PR's base and head SHAs and the shadcn SHA, and works from a git archive checkout of the head. The main working tree is never touched.
  2. Per touched component, compares classes (Base UI .tsx + vega cn-* rules), HTML, props, ARIA, behavior and animation against shadcn. Both sides are driven in a browser with identical fixture content: a local shadcn sandbox (Base UI + vega + neutral) and RubyUI's docs app at the PR head.
  3. Checks shadcn against HTML, WAI-ARIA and ARIA in HTML (APG and MDN carry less weight), and flags JS that a native feature such as <dialog> or popover could replace.
  4. Prints a checklist (✅ ⚠️ ❌ ⏸, each line with values from both sides and a source) and writes an HTML page with screenshots side by side.
  5. Posts the checklist as one comment on the PR, only after an explicit ok, only on an open PR whose head hasn't moved, and only over the runner's own earlier comment.

What's in the PR

  • audit/: new subproject (CLI exe/shadcn-audit, Minitest + standardrb, CI job on Ruby 3.3 and 3.4). The CI job tests the tool itself. It doesn't audit PRs.
  • .claude/skills/shadcn-audit/: the procedure, plus references for grading, interactions per component kind, spec sources, and native HTML.
  • audit/component_map.yml and fixtures for Dialog, AlertDialog, Sheet and Command. Other components get ⏸ lines until mapped.
  • audit/divergences.yml: deliberate differences, registered in the PR that introduces them.
  • gem/AGENTS.md and CLAUDE.md: agents writing components start from Base UI + vega and prefer native HTML.

Acceptance on #572

Recorded in audit/test/fixtures/acceptance_572.md. The run found everything we found by hand. The overlay matches; the Dialog content still has the new-york look; exit animations are missing; AlertDialog, Sheet and Command are <div>s that could be native <dialog>; and there's no aria-labelledby/aria-describedby. It also found a dark-mode bug: the Dialog has no text-foreground, so its text renders black on near-black.

Each run tracks shadcn main and records the SHA it used. We can pin the reference later if drift between runs becomes a problem.

Testing instructions

  1. cd audit && bundle exec rake (118 runs, 0 failures; standardrb clean).
  2. bundle exec exe/shadcn-audit workspace then bundle exec exe/shadcn-audit reference dialog: the parts list shows vega classes such as rounded-xl and ring-1 on dialog-content.
  3. In Claude Code, run /shadcn-audit 572. It needs gh authenticated, Node/pnpm, and the chrome-devtools MCP. It asks before posting anything.

🤖 Generated with Claude Code