refactor: play everything through Vivid's own player bridge and Lucid Engine

Merged
BL
4d ago
+59588-6870

[!NOTE]
Every commit here carries [skip release]. I'll choose the release marker for the merge commit when I merge.

Problem and reproduction

Vivid's player rested on two pieces of other people's work. The Swift bridge between the player and libmpv was adapted from another app, and the engine underneath it (libmpv, FFmpeg and the libraries they use) came as a third-party prebuilt package. That package built mpv and FFmpeg as GPL with OpenSSL, and device builds needed an extra step to swap in a separately rebuilt libmpv carrying my audio patch. I couldn't rebuild exactly what shipped, and the player couldn't carry Vivid's own licence and Apple distribution permission all the way through.

There's no user-facing bug to reproduce. The goal is that every part of the player is mine or built by me from upstream source, with nothing from the previous bridge or package left in the tree.

Implementation and impact

  • Player bridge. A new Swift bridge, written from a behaviour specification (docs/playback/player-bridge-spec.md). VividPlayerCoreBase and VividPlayerCore own the mpv context, event loop, ordered delivery, requests and teardown. Conversion, requests, the video surface, Match Content and EDR, external displays and lifecycle observation sit beside them in iosApp/iosApp/Playback/MPV/. LucidTLS turns on certificate verification when FFmpeg is built with Secure Transport. The previous bridge files and their notice are removed.
  • Lucid Engine. My own pipeline in engine/ builds every library from pinned upstream source with my patch series: mpv 0.41.0 and FFmpeg 8.0.1 (both LGPL-2.1-or-later as built, Secure Transport instead of OpenSSL), libass 0.17.5, dav1d, libdovi, FreeType, FriBidi, HarfBuzz, libunibreak, Little CMS, shaderc, MoltenVK, libplacebo, uavs3d, libbluray and uchardet. The patches are:
    • mpv 0010 to 0016: Vivid's AVFoundation audio output (decoded PCM, the bounded tvOS mode, AC-3 and E-AC-3 passthrough, AirPlay start-up supervision and clock anchoring);
    • mpv 0020 to 0025: Vivid's AVFoundation video output (timing and pacing, the frame path, subtitles, layers and observers);
    • FFmpeg 0010 to 0012 and mpv 0040: decoder and Dolby Vision work for VideoToolbox;
    • libdovi 0001 (an error instead of an abort on malformed input), libbluray 0001 (building for iOS and tvOS) and uavs3d 0001 (its version without Git history);
    • FFmpeg 0001 and 0002 (Secure Transport without a private API, Metal-compatible pixel buffers) and mpv 0001 (an unchanged upstream spdif fix) and 0002 (building the audio output for iOS and tvOS);
    • libass parity: upstream commit 057a5334 and upstream pull request #951, both unchanged, plus my 0010, so static subtitles render and cache as before.
  • Package. blurbery/lucid-engine 0.1.0 (commit 53ebeac) serves the 21 frameworks and MoltenVK that run 38458337996 built at fcb6782, with NOTICE, LICENSES/ and a README naming that source commit.
  • Switch-over. Both app targets pin 0.1.0 in iosApp/project-ios.yml, iosApp/project.yml and Package.resolved. The previous package, its patches, its install script, its workflows and its licence texts are removed. Open Source Licences (and the website licences page built from the same files) and THIRD_PARTY_NOTICES.md now list every library as built from upstream source, each patch and its licence, and where the corresponding source lives: engine/ at fcb6782, and the package README.
  • CI. A new Player code checks workflow runs on every pull request and push and fails if the previous code or package comes back or the engine pin drifts. The Lucid Engine build workflow now also runs on pushes to main and on pull requests that change engine/ or the workflow.
  • Tests. New app tests: VividPlayerCoreTests, VividPlayerConversionTests, VividPlayerDisplayCriteriaTests, VividPlayerEngineTests, PlaybackEndRecoveryTests and VividEndOfStreamConfirmationTests (MpvPropertyCoalescingTests went with the old bridge). In the engine pipeline: audio output vectors, video output tests, decoder suites, a malformed Dolby Vision RPU corpus through patched and unpatched libdovi, and libass scenarios.

Affects playback on iPhone, iPad and Apple TV for every server. Deliberately unchanged: the player screens, controls, settings and design, the Silo, Emby and Jellyfin cores, and Vivid's driver options.

Risks: this replaces the whole playback stack in one go, and nothing has been played on a physical device yet. The areas most likely to differ are AirPlay and HDMI audio timing (the new audio output), Match Content, HDR and Dolby Vision on real displays, VideoToolbox decoding, long sessions, and libass cost with heavy styled subtitles now that rendering stays on one thread. Reverting this PR's merge brings the previous player back.

Validation

How parity was established:

  • Specifications first. The bridge and the engine patches were written from behaviour specifications of the previous bridge and engine (bridge, engine), and every review difference was either fixed or recorded there as a deliberate difference.
  • Side-by-side comparisons with the previous engine (patch spec 13.3, on the integrated build at 10faa2a): configuration and symbols match; the audio output gives the same results across 30 scenarios on iOS-like and tvOS-like host builds, apart from free log wording; the decoder matches across 49 scenarios against real macOS VideoToolbox; the video output's timing and picture path show 0 differences (11,744 picture checks); libass is pixel-identical on the parity corpus (23,280 renders), and the libass series added since restores the previous change reporting and font fallback behaviour. No regression was found. Behaviour reviews of the new bridge against the old one found nothing that would regress.
  • Engine host suites in run 38458337996 (the run 0.1.0 was published from): 683 of 683 checks per slice, 140 audio output vectors per slice, 1,920 video output cases, decoder suites of 534, 199,060 and 5,937 checks, 19,381 libdovi cases and 82 libass scenarios, all passing. Every library is checked on every run against the recorded configuration of the previous binaries.
  • CI on this head (1fc0705):
    • Player regression on this PR, run 38466062418: complete iOS suite 1,476 tests, 0 failures (2 skipped, the existing live-server tests); Apple TV suite 211 tests, 0 failures; unsigned Release builds of both apps for generic iOS and tvOS devices, linking 0.1.0's device slices, passed.
    • Optimised run, 38466061836: the iOS suite as a normal build and compiled with the Release optimiser, 1,476 tests each with 0 failures (2 skipped), and the Apple TV suite, 211 tests with 0 failures.
    • Player code checks: 38466062548 (pull request) and 38466056858 (push), both passed.
    • Lucid Engine build on this PR, 38466062631: every job passed with no dependency cache to reuse, so every library was built from source again, and its 22 xcframework zips match 0.1.0's SHA256SUMS byte for byte.

Performance

Not benchmarked. Host comparisons show the same timing and pacing behaviour, but frame times, CPU and memory on devices haven't been measured yet.

Testing status

  • Tested? Partly: CI and local simulators. No physical-device testing yet.
  • Environment: GitHub Actions: iOS and tvOS simulators on macos-15 with Xcode 26.3, unsigned Release device builds, and the engine pipeline on macos-26 with Xcode 26.6. Locally: Xcode 27.0 with the existing iPhone 18 Pro and Apple TV 4K simulators (iOS 27.0 and tvOS 27.0).
  • Revision: 1fc0705.
  • Checks and results: CI as listed under Validation, all passing. On both simulators:
    • testCompletionsCalledOffMainArriveLaterOnMain passed 200 of 200 iterations, the four VividPlayer*Tests classes passed 3 iterations (399 tests each), and the playback test classes passed (334 on iPhone, 116 on Apple TV, 0 failures).
    • Real playback through the engine of H.264, HEVC 8-bit and HEVC 10-bit HDR10 with AAC, AC-3, E-AC-3 and TrueHD, plus the repository's 8 container fixtures: every load played; A/V sync stayed at about 0 ms; no dropped frames in steady play; seeking, audio and subtitle track switches, the compressed audio output, speed changes across 1x and HDR10 detection all behaved.
    • Subtitles: a static boxed SubRip line and a static clipped ASS sign rendered once and were then reused; karaoke re-rendered every frame as it should.
    • The engine reports mpv 0.41.0, FFmpeg 8.0.1 and libass 0.17.5, and the app bundles contain none of the previous bridge's or package's names.
  • Limitations:
    • On the iPhone simulator, video sometimes paused for up to about 0.4 s after resuming from a pause while audio carried on (5 of 15 resumes). I've accepted that for now; the device test will show whether it happens on hardware.
    • On the 27.0 simulators FreeType cannot open the PingFang UI font, so some Simplified Chinese characters can fall back to missing glyphs, and emoji in subtitles show as boxes. These weren't compared against the previous build.
    • Everything hardware-specific is still to test on devices: VideoToolbox decoding and Dolby Vision, passthrough and Atmos, AirPlay, HDR and Match Content, Picture in Picture, external displays, background and lock, and long 4K sessions on Apple TV. That covers the bridge specification's device tests (16.3) and the engine specification's device checklist (9.5, with the checks that 12.7 and 13.7 add), including check 46 (heavy styled ASS on one rendering thread) and checks 65 to 69.

Design impact and approval

  • Design impact: No design changes. Open Source Licences keeps its layout; only its text changes.
  • Approval from blurbery: Not applicable, no design changes.

Documentation impact

  • New: docs/playback/player-bridge-spec.md, docs/playback/lucid-engine-patch-spec.md, docs/cores/lucid-engine-architecture.md, docs/cores/lucid-engine-build.md, docs/cores/lucid-engine-build-facts.md and docs/cores/lucid-engine-libs-build-facts.md.

  • Updated: docs/cores/player-engine.md, docs/README.md, docs/playback/README.md, docs/release/distribution.md, docs/release/versioning.md, CONTRIBUTING.md, AGENTS.md and THIRD_PARTY_NOTICES.md.

  • In-app licences: iosApp/Resources/OpenSourceLicenses/ (overview, every engine library's header, mpv and FFmpeg moved to their LGPL texts, OpenSSL and the previous bridge's licence removed), with scripts/tests/licence-texts-tests.py updated. The website licences page is generated from these files.

  • No privacy policy change: what the app sends and stores is unchanged.

  • I checked the docs this change affects and updated them in this PR, or explained above why none needed changes.

AI disclosure

I implemented these changes and used Claude Opus as a tool.