fix(orchestrator): create /dev/fuse for dockerd; runner parity refinements

Closed
BN
2mo ago
+18-3

Follow-ups to the conformance campaign fixes (PR #151).

  • docker: the krunfw guest kernel has fuse built in, but /dev boots as a plain tmpfs with no device nodes, so fuse-overlayfs (dockerd's fallback when its overlay probe fails) dies with fuse: device not found. The hook now creates /dev/fuse when the kernel lists fuse, letting dockerd auto-pick fuse-overlayfs (CoW) — on this kernel dockerd's overlay2 probe mount gets EINVAL and overlay2 is never viable, so the earlier fix was falling back to vfs. vfs is now forced only when overlay fails AND fuse is absent.
  • RLIMIT_NOFILE: 524288 instead of 1048576 — systemd's built-in hard default, which is what GitHub's runner service inherits (exact parity).
  • guest PATH: cargo bin dir matches the runner user (/home/<user>/.cargo) instead of hardcoded /root/.cargo/bin, which the unprivileged runner cannot stat (nodejs/ci EACCES).
  • multiarch shim: add libsystemd0:amd64 (valkey's x86_64 tarballs link libsystemd.so.0).

Verified live on the golden VM: hook creates /dev/fuse, dockerd 28.0.4 reports Storage Driver: fuse-overlayfs, and hello-world runs. 61 orchestrator tests pass.

Summary by cubic

Creates /dev/fuse in krun guests so dockerd can auto-select fuse-overlayfs and keep copy-on-write. Previously we forced vfs when the overlay2 probe failed; now we only force vfs if overlay is unusable and /dev/fuse is missing, and we write daemon.json only when a driver is forced.

  • Create /dev/fuse when the kernel supports fuse; leave DRIVER unset when the overlay2 probe succeeds or fails but fuse exists; force DRIVER=vfs only when the probe fails and fuse is absent.
  • Migration: if you rely on a custom Docker data-root without forcing a driver, it will no longer be written; configure it elsewhere.
  • Runner parity: set RLIMIT_NOFILE=524288, add /home/<user>/.cargo to PATH, and install libsystemd0:amd64 for the multiarch shim.

[!NOTE]

Create /dev/fuse for dockerd and stop forcing overlay2/vfs storage drivers

  • Creates /dev/fuse (via mknod c 10 229) when /proc/filesystems reports fuse support and the node is missing, in docker_start_command in lib.rs
  • On successful overlay probe, no longer forces overlay2; leaves DRIVER empty so dockerd selects its own driver
  • On failed overlay probe, only forces vfs when /dev/fuse is absent; otherwise leaves DRIVER empty to allow dockerd's fallback
  • Writes /etc/docker/daemon.json (including data-root and storage-driver) only when DRIVER is non-empty
  • Risk: data-root is no longer written to daemon.json when no explicit driver is selected; any setup relying on a custom data-root without a forced driver will lose that configuration
📊 Macroscope summarized 64d0055. 1 file reviewed, 2 issues evaluated, 0 issues filtered, 1 comment posted

🗂️ Filtered Issues