Speed up exception handling
AK
3mo ago+162+162+012345678901234567890123456789-148-148-012345678901234567890123456789
Summary
@contextmanager is slow, and it's used in hot paths for every request both in httpcore2 and httpx2.
This PR replaces uses of map_exceptions and map_httpcore_exceptions with faster alternatives.
On my machine, a pytest-benchmark test shows a nearly 10% speed increase on a microbenchmark that retrieves a file from a local Caddy (see source in the details block below)
----------------- benchmark: 2 tests, 2 sources ------------------
Name (time in ms) 0010_82b9e2d Min 0042_384ae31 Min ΔMin
------------------------------------------------------------------
test_connpool 3.8101 3.4641 -9.1%
test_client 4.0599 3.7075 -8.7%
------------------------------------------------------------------
test_bench1.py
import pytest
from httpx2 import Client
from httpcore2 import ConnectionPool
def test_connpool(benchmark):
with ConnectionPool() as pool:
def b():
resp = pool.request(
method="GET",
url="http://127.0.0.1:9339/bigfile2",
)
assert len(resp.content) == 10 * 1024 * 1024
benchmark(b)
def test_client(benchmark):
with Client() as c:
def b():
resp = c.request(
method="GET",
url="http://127.0.0.1:9339/bigfile2",
)
assert len(resp.content) == 10 * 1024 * 1024
benchmark(b)
and a hyperfine check for the same says ~5%, but that's with all of the overhead:
$ hyperfine --warmup=5 --min-runs=20 'git checkout main && uv sync && uv pip install ../pytest-benchmark && pytest --benchmark-autosave test_bench1.py' 'git checkout retire-map-exceptions-hx2 && uv sync && uv pip install ../pytest-benchmark && pytest --benchmark-autosave test_bench1.py'
[...]
Summary
git checkout retire-map-exceptions-hx2 && uv sync && uv pip install ../pytest-benchmark && pytest --benchmark-autosave test_bench1.py ran
1.05 ± 0.06 times faster than git checkout main && uv sync && uv pip install ../pytest-benchmark && pytest --benchmark-autosave test_bench1.py
Rebase of
Checklist
- I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
- I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- No test changes needed. Some pragma: no covers were added to the exception branches that were actually not covered originally either.
- I've updated the documentation accordingly.
- No public documentation changes needed, I think.