Force the exit on a repeated Ctrl-C while a prod lifespan cleanup is stuck

FA
1d ago
+289-11

All Submissions:

  • Have you followed the guidelines stated in CONTRIBUTING.md file?
  • Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Changes To Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

Problem

reflex run --env prod with a lifespan task whose cleanup takes long (the repro in #7834, a 60 s sleep after yield) ignores every Ctrl-C after the first: the process waits out the whole cleanup and prints nothing. Reproduced on main (8e5bb8451) with --env prod --backend-only: SIGINT to the process group at 0 s, 3 s and 6 s, still running 24 s later.

Cause: in prod granian runs in the CLI process and installs its own SIGINT/SIGTERM handlers, which only set an "interrupt" flag. After the first signal it sits in shutdown() joining the workers with no timeout, and a repeat sets the flag again. _run_prod never used _shutdown_signals(), the force path _run_dev has.

Fix

  • _run_prod runs a granian backend under _shutdown_signals(), the same context manager as dev, and passes its begin_cleanup down as on_shutdown.
  • run_granian_backend_prod calls on_shutdown at the start of granian's shutdown(), before the workers are joined. begin_cleanup starts the repeat window, records the worker tree and puts the forcing handler back, as it already does for the handlers granian leaves behind in dev.

The first signal is unchanged: granian still takes it and shuts down gracefully, lifespan cleanup included. A repeat within _REPEAT_IGNORE_SECONDS of the shutdown starting is ignored; a later one kills the workers, prints Forced exit: the cleanup did not finish. and exits 128 + signum.

With the fix the same repro exits 130 right after the second SIGINT (3 s).

Deliberately not changed

  • The gunicorn/uvicorn prod path (REFLEX_USE_GRANIAN=false): that server is a subprocess, so the run keeps the signal handlers it had and gets no forced exit. gunicorn is not installed in my environment, so I left its behaviour alone.
  • No new "waiting for lifespan cleanup" line after the first signal (marked optional in the issue).
  • reflex run --env prod --frontend-only (run_frontend_prod): it serves static files and has no lifespan cleanup to get stuck.
  • _run_preview, and the Windows branch of _shutdown_signals.
  • No timeout on granian's graceful stop in prod: a slow cleanup still gets all the time it needs unless the user asks again.

To check in review

  • On Windows the main thread waits for a worker in a call Python signal handlers cannot interrupt, so the repeat may only be handled once that worker exits. The new signal tests are POSIX-only, like the existing ones.

Tests

  • tests/units/test_reflex.py::test_run_prod_force_exits_on_a_later_repeat (SIGINT and SIGTERM): runs _run_prod in a subprocess around a backend that behaves like granian (own handlers, stuck worker); checks the early repeat is ignored, the later one exits 128 + sig, stderr is exactly the "Forced exit" line and the worker is gone. Fails on main.
  • tests/units/test_reflex.py::test_run_prod_takes_signals_for_granian_only: the handlers and callback are in place for granian and untouched for gunicorn/uvicorn.
  • tests/units/utils/test_exec.py: on_shutdown runs before the workers are stopped, and run_backend_prod hands it to granian only.
  • tests/integration/test_run_shutdown.py::test_prod_repeated_stop_forces_exit: real reflex run --env prod --backend-only (granian forced) with a 600 s lifespan cleanup; once the worker has entered the lifespan, a second group SIGINT 2 s into the cleanup must end it within 5 s with exit 130, no survivors and "Forced exit" as the last line. Fails on main (verified with the source change stashed). The shared test app gained the lifespan task, inert unless the new case sets its env var.

Run locally (Linux, Python 3.14):

  • uv run pytest tests/units/test_reflex.py tests/units/utils/test_exec.py tests/integration/test_run_shutdown.py -q — 161 passed
  • uv run ruff check . and uv run ruff format . — clean
  • uv run pyright on the five changed Python files — 0 errors

closes #7834

https://claude.ai/code/session_01LwJUCRwVzoegAwG3Hp1F9J