PR #8721 - manaflow-ai/cmux

This is a large diff, and I know that makes it hard to review. I'm happy to break it apart if you would prefer that. It splits into five pieces that can each stand alone, in this order:

Sign-in on reconnect (what #8555 was).
Connection robustness: reattach after a transport drop, and a clean detach at quit.
The transport seam and named brokers in cmux.json.
EternalTerminal as a transport (--transport et).
One shared connection per host, behind its beta setting.

Tell me if you want that and I'll send them as a stack.

This is the integration branch for the remote-tmux transport line. It carries #8428 (the session multiplexer), #8555 (the reconnect login), and #8556 (the transport seam), and adds what was still missing on top of them: an attach in multiplexed mode opens exactly one connection, a mirror that has nothing to show says so instead of presenting an empty workspace, and a second transport — EternalTerminal — is selectable and proven against a real broker.

One connection per attach

Multiplexed mode exists so a host's sessions ride one shared control stream, but the attach still opened extra connections inherited from the per-session path: a discovery one-shot whose result was discarded, a ControlMaster warm-up for a burst this mode doesn't have, and four pre-attach one-shots the view ran to find, reap, create, and size its own session. Over plain ssh those are nearly free. Over a transport that authenticates per connection they are fatal — measured against a corporate ssh broker, each invocation performed its own MFA, nothing persisted, and the view stream died against a passcode prompt it had no tty to show. The stream is now the only connection: it opens with new-session -A, which attaches the view when it exists and creates it at the right size when it doesn't.

cmux ssh-tmux <host> --new-window did not work over the shared connection. The attach picked its destination from windows that already existed, a dedicated attach for a host with no mirror window has none, and it failed as "app not ready" before opening a connection. It now creates the window, closes the window's first local workspace once the mirrors are in, and discards the window if nothing was mirrored.

An empty mirror tells the truth

Reaching control mode is not the same as having something to mirror. Measured on a real host: the stream sent a complete attach block and then %exit without ever publishing a window, and cmux reported that as success — a workspace appeared whose only surface was a local placeholder shell. A mirror now counts only once its connection has published a topology; attachHost waits for every mirror concurrently under one deadline and drops the ones that never arrive, saying why the tab went away. %exit alone is also no longer trusted for a transport whose remote half outlives its client.

EternalTerminal, selectable per host

cmux ssh-tmux --transport et <host> (with --transport-port and --broker; documented in --help). The transport is a property of the host, not a global switch, because one host can be reachable over a session-preserving transport while another is plain ssh. ssh stays the default with unchanged argv.

Two faults kept et from carrying tmux -CC at all, both found by running the end-to-end harness against a real etserver rather than reading argv. et types its --command into a login shell through a pty in canonical mode, which delivers at most MAX_CANON (1024) bytes per line — the PATH-resolver command ssh needs is ~1113 bytes, so the line never completed and the stream sat silent until the attach timed out. et now gets plain tmux; a login shell already has the user's PATH. And the control-mode parser recognised tmux's ESC P 1000 p only at line start, while the login shell leaves echo and OSC title sequences ahead of it with no newline, so the mirror waited forever on a stream that was working. The scan now finds the sequence anywhere in the line.

Authentication that can't reach a terminal

A transport that authenticates itself never reports a failure — it prints a prompt to a tty it doesn't have and waits. The reconnect classifier only read stderr, so a broker whose passcode prompt produces no stderr came back "transient" and retried forever. An unanswered prompt in the pre-control region (everything before the first %begin) is now classified as needing a login, feeding the same parked-login flow the reconnect path uses; the markers are generic because the wording belongs to whatever broker a site runs.

A reconnect attempt that stops at such a prompt is still running when the next attempt is scheduled. The retry used to start over it without ending it, so a transport that signs in by itself left one client behind on every retry. The attempt is now torn down before the retry is scheduled.

An attach waits for a login that is still working

cmux ssh-tmux <host> failed at exactly 30 seconds with "could not mirror any tmux session" on a host whose login needs a hardware-key tap, and again on the retry, where one connection waited 19 seconds for the host's single session slot. Both logins were still working when the attach gave up. The attach over the shared connection had 30 seconds in total to show its first sessions, and giving up also stopped the transport, so the retry started the login over.

The wait has no total limit now. It ends when the stream publishes its sessions, when the transport exits, when the host asks for a login, or when the transport has printed nothing for longer than its phase allows: 300 seconds before tmux answers, because a person may be part of a login, and 30 seconds after. Anything the transport prints moves that limit out. The failure says which of these happened and carries the transport's own last line, for example the connection to dev ended before tmux answered: ssh: connect to host dev port 22: Connection refused.

While it waits, cmux ssh-tmux prints where the attach stands every 15 seconds (still logging in: the connection is running and has been quiet for 34 s), so a long login reads as one that is still running. The socket call and CLI let the attach own its inactivity deadline, so ongoing login progress is not cut off by an independent wall-clock timeout.

When the shared stream goes away, and when it shouldn't

Closing a host's last workspace left the shared stream, its hidden view session and the remote master running. The workspace handler looked up a tab manager before doing anything else, and once the last workspace is gone the mirror is gone with it and the window the attach came from may be closed too, so that lookup found nothing and returned early — past the teardown it was there to reach. Teardown needs no tab manager, so it now runs first.

The opposite fault is dropping a stream that is answering. A raw query used to end two ways, lines or nothing, so a %error reply and a timeout were indistinguishable. The reconcile's bounded retry re-sent a command the server had already rejected, and that retry asks to reconnect when it times out, which throws away a control stream that was working fine. A query now reports lines, an error, or silence, and only silence is worth asking again.

The reconcile can also be handed an answer that is not its own. Replies are matched to commands by position, and when one lands on the wrong command the session or window list comes back empty or foreign. The plan read that as a host with nothing on it and closed every workspace, with the stream still connected. The stream is attached to the view session, so a real answer to either list always names that session. A snapshot that does not is now dropped, and the stream reconnects, which is what puts replies back in step.

What this leaves out

New Workspace in a mirror window still creates a local workspace on this branch. Sending it to the mirror's host is #7214. Creating that session over the shared connection, which a host that allows only one connection needs, comes after both are in.

Socket parameters

One read-only socket method is added, remote.tmux.attach_progress, and the relay allowlist is untouched, so the method does not work through cmux ssh. For a host the caller names it returns whether an attach is in progress, which phase it is in (logging_in or in_tmux), how many seconds the transport has been quiet and the limit for that phase. It returns nothing the transport printed and no session or window names, runs no command, and changes nothing. It runs on the socket worker lane like the other remote.tmux.* methods.

The existing remote.tmux.* methods build their host from the request, and that builder accepts four more parameters:

transport is parsed against a closed set, ssh or et. Anything else refuses the host.
transport_port must be an integer from 1 to 65535.
transport_helper_path optionally names an absolute remote helper path for direct ET. It is omitted by default, so ET uses its own lookup. cmux ssh-tmux --transport et --transport-helper-path /usr/local/bin/etterminal <host> selects that installation explicitly. SSH and named brokers reject the override; brokers own helper resolution.
transport_broker is a name. It is looked up in remoteTmux.brokers in the user's own cmux.json, and the executable and arguments come from that entry. A request cannot supply a command or arguments. A name that does not resolve refuses the host instead of falling back to a direct connection, and a broker named for the ssh transport is refused.

So the only thing a caller on the socket can make cmux run is a broker the user has already declared, with the arguments the user wrote. An attach that names a different broker than the live connection to the same host uses is refused with the route in use, instead of being served over a route it did not ask for. theSocketBoundaryAttachesADeclaredBroker, theSocketBoundaryRefusesAHostWhenABrokerCannotBeResolved, theSocketBoundaryLeavesAHostAloneWhenNoBrokerIsAskedFor, anUndeclaredBrokerNameIsRefusedRatherThanIgnored, aRelativeBrokerExecutableIsRejectedWithItsReason and hiddenCharactersAreRefusedInArgumentsAndInTheRequestedName in RemoteTmuxProxyTransportRetryTests cover it.

Verification

Conformance matrix recorded against et 6.2.11 and 7.0.0 (docs/ in this branch), measured on a live etserver, not inferred.
Capstone multiplex fuzzer (seeded bidirectional session churn) extended to assert no mirror/channel leaks; attach modes, readiness, and the one-connection view covered in cmuxTests.
RemoteTmuxRawQueryOutcomeTests drives the three ways a raw query can end through the real correlation path and asserts that only silence triggers the reconcile retry.
RemoteTmuxReconnectAttemptTeardownTests runs a fake client that prints a prompt and waits on every reconnect, and expects the second attempt to be signalled to stop before the third starts. It fails at e50d88465ec and passes at 95dc82798dc.
RemoteTmuxViewUnlinkedWindowTests.listReplyWithoutTheViewSessionClosesNothing answers the reconcile's list-sessions, then its list-windows -a, with an empty block. Both cases fail at 869315ae784, where the host's workspaces are emptied, and pass at 0ad26e1be97.
RemoteTmuxMultiplexedDedicatedWindowTests runs a dedicated attach against an ssh stand-in that refuses the connection. It fails at 7aa0002b2c4 with "app not ready" and no connection attempted, and passes at 480f19d44c7, where the host is tried and no window is left behind.
End-to-end: a real attach through a corporate-style broker over et, one authentication prompt total, sessions mirrored and surviving a network change.
RemoteTmuxMultiplexedLoginWaitTests attaches to a host that refuses the connection the way ssh does when it needs a sign-in. It fails at 85879d70b91, where the attach hands back the login with its shared stream already stopped, and passes at 4d0ac06b947.
The session digest subscription names no pane. Measured on tmux 3.7b with pane %0 killed: a %0 subscription reports nothing when a session is created or renamed, and the empty target reports both.
RemoteTmuxAttachProgressTests covers the wait. One test runs an ssh stand-in that fails after 3 seconds with the after-tmux limit set to 1 second, and expects the transport's own reason back. Another runs one that never answers and expects the stall to be named and the stream stopped. With one limit for both phases put back, which is the old behavior, eachPhaseIsHeldToItsOwnQuietLimit and aSlowLoginIsWaitedForAndItsOwnFailureIsReported fail.
scripts/remote-tmux-attach-wait-harness.sh checks the same thing end to end: the real cmux ssh-tmux, a tagged app, a loopback ssh host, and an ssh stand-in that delays or breaks only the connection. A transport that fails 8 seconds in is reported at 9 seconds with Connection refused. One that never answers ends at 300 seconds, is named as a quiet login, and is stopped. One that connects after 45 seconds is mirrored, with progress lines at 2, 18 and 34 seconds.
At 6aff58beca6, merged with main 74cfb96d6c1, the remote-tmux unit suites and the settings-file suite pass locally: 652 tests in 67 suites.

Later work from the PRs this carries

This branch was cut from #8428, #8555 and #8556, and those branches kept going. Their later commits are now cherry-picked here, 22 in all, each resolved against this branch's changes, so every commit on those three branches has a counterpart here. Where both lines had written the same thing, this branch's version stayed: the observers initializer that requires every member, beginReconnecting(preservingBackoff:), and [$id] tagging for channel events. setMirrorEnvironment is now a session-source requirement, so the controller pushes the mirror identity through the protocol, and a multiplexed channel publishes it into its own session instead of skipping it.

Changelog

Added: Remote tmux can connect over EternalTerminal, and a beta setting shares one connection per host across all of its sessions

Summary by CodeRabbit

New Features
- Added EternalTerminal (et) as an option for remote tmux connections, with configurable ports and brokers.
- Added an optional beta mode that shares one connection across sessions on the same host.
- Added interactive login prompts when a remote connection needs authentication, with automatic reconnection after login.
- Added ssh-tmux options for selecting a transport, port, and broker.
Bug Fixes
- Improved recovery from interrupted connections and handling of remote session and window changes.
- Improved connection cleanup when detaching or closing the app.

[!NOTE]
Medium Risk
Changes remote connection routing, long-lived socket I/O, and quit-time detach for shared tmux transports; mis-handling could affect authentication, attach timeouts, or leave remote sessions pinned, but brokers are config-gated and CLI validation limits executable choice.

Overview
Remote tmux CLI and control path gains selectable transports and named brokers: cmux ssh-tmux accepts --transport ssh|et, --transport-port, --transport-helper-path, and --broker (a key from remoteTmux.brokers in global cmux.json, not an arbitrary command). Mirror/attach calls use waitUntilCompletion on the control socket so login can outlive the old fixed timeout, with clearer “still connecting” messaging and a background RemoteTmuxAttachProgressReporter that polls remote.tmux.attach_progress every ~2s. Interactive SSH for tmux sets CMUX_REMOTE_TMUX_AUTH=1 so site ssh_config hooks treat the login as cmux’s own.

Configuration and policy: generated schema adds remoteTmux.brokers and a remoteTmux.multiplexer.beta toggle; settings/catalog wire the multiplexer beta key. CmuxConfig loads brokers from global config only into a socket-trusted snapshot. remote.tmux.attach_progress is allowed on the socket worker lane.

Robustness on quit and file watch: app terminate defers until detachAllAwaitingExit() runs for mirrors that still owe tmux a goodbye. FileWatcher exposes isWatchingAncestorDirectory so reconnect logic can fall back when ancestor open(O_EVTONLY) fails (tests added).

The diff also carries extensive localization for new ssh-tmux help, transport/broker socket errors, reconnect sign-in strings, and related UI copy.

^{Reviewed by Cursor Bugbot for commit 8e7c5fc9d89a26b422e710f4308e7cb97db07fa8. Bugbot is set up for automated code reviews on this repo. Configure here.}

Review follow-up

Updated head: e1a477f71b922aeaeaeb8f3dfe61cfc60c0d431c. The earlier merge from origin/main is retained and GitHub reports MERGEABLE. All 31 inline review threads are resolved; this pass also addresses findings in CodeRabbit's summary comment.

Authentication failure classification now reads the host ledger alone. The active shared view publishes login-required and authenticated transitions; callbacks from a replaced view cannot change the current host state. A held old view cannot turn a later network failure into another login prompt.
Process-tree teardown owns its bounded SIGKILL task until cleanup finishes, including after the mirror releases its connection. Signals check each process's birth timestamp to avoid targeting a reused PID. Query deadlines use the existing cancellation-aware timer.
Host-wide snapshot parsing and planning run off the UI actor, with connection/generation checks before applying the result. Readiness and pre-ID rename matching use indexed lookups. Tests cover 1,000 renames and ambiguous window ownership.
Broker configuration publishes only to the socket snapshot instead of invalidating SwiftUI. Pure classification helpers are nonisolated, reconnect logs redact host identity, and the debug-only pre-control accessor is removed.

Defaults remain unopinionated: SSH argv and ordinary socket deadlines are unchanged; direct ET uses native lookup unless --transport-helper-path is supplied. Named brokers own helper resolution. Endpoint reuse still rejects a conflicting route instead of opening another connection to the same host.

Scope decisions: retain EJ's accepted follow-ups for per-session map caching, broker-snapshot ownership, and the eleven additional locales. Extracting the pure planners into a new Swift package belongs in a separate PR so this transport integration does not also change package/build boundaries. Existing CLI validation and machine-readable schema text retain their current conventions. The nine required catalog locales pass validation. Docstrings were added for changed ownership and execution boundaries; no unrelated documentation expansion was made to meet a percentage.

Verification:

Runtime commit 8e7c5fc9d89 also validates the stream generation immediately after bootstrap. Final commit e1a477f71b9 changes only the scale-test fixture: an explicit Int prevents Swift from generating invalid window IDs such as @Optional(0). All assertions remain intact.

All 17 local static checks passed; Swift syntax passed for all 14 edited files.
Final-head focused native run passed all 45 tests across seven suites at e1a477f71b9, including the held-view stale-login regression, process cleanup after releasing the connection, first-attach/login handoff, view reconciliation, transport defaults, and the 1,000-mirror rename case. The tests executed in 8.123 seconds.
Pre-fix regression run reproduced the stale-login bug at 8ceb8c24c1e: 14 tests executed; the 13 existing tests passed and only the new held-view regression failed.
Tagged build pr-8721-review-v5 passed at runtime commit 8e7c5fc9d89 (job 26134643593d0bf4d7a4e064); the final commit changes only a test fixture; its backend readiness and tagged bundle identity are verified. This is build evidence, not a new live corporate-broker or ET-server test.
Final-head PR CI passed at e1a477f71b9, including native compilation, all seven app-host unit-test shards, CLI product tests, and the static/workflow checks. GitHub reports no merge conflicts or unresolved review threads.

The earlier head 5dc74481034 passed full PR CI, including all seven native shards and CLI product tests. That is prior-head evidence, not verification of this update. The earlier supplementary runner provisioning problem remains tracked in HQ #1545.