Skip to content

Avoid quadratic connector waiter scheduling across hosts - #14379

Open
PLPeeters wants to merge 1 commit into
aio-libs:masterfrom
PLPeeters:bugfix/quadtratic-connector-wait
Open

PLPeeters wants to merge 1 commit into
aio-libs:masterfrom
PLPeeters:bugfix/quadtratic-connector-wait

Conversation

@PLPeeters

@PLPeeters PLPeeters commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

What do these changes do?

They replace per-release copying and shuffling of waiting host keys with rotation through an OrderedDict. Draining one waiter from each of N eligible host queues now takes O(N) scheduling work instead of O(N²).

Local benchmarks showed a 33x speedup for 100 hosts and a 535x speedup for 1600 hosts (yes, you read that correctly).

Are there changes in behavior for the user?

Cross-host scheduling changes from randomized selection to round-robin rotation. Connection limits, FIFO order within each host, and cancellation handling are preserved.

Clients queuing requests across many distinct hosts should spend less time scheduling waiting connections.

Is it a substantial burden for the maintainers to support this?

No new dependencies or public APIs are introduced. A single ordered mapping holds the host queues and their scheduling order, avoiding duplicate bookkeeping.

The main maintenance concern is preserving queue cleanup and rotation during cancellation and retries; regression tests cover these paths.

Related issue number

N/A

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • If you provide code modifications, please add yourself to
    CONTRIBUTORS.txt
  • Add a new news fragment into the CHANGES/ folder

@PLPeeters
PLPeeters requested a review from asvetlov as a code owner November 14, 2026 08:34
@PLPeeters
PLPeeters force-pushed the bugfix/quadtratic-connector-wait branch from 08e9a16 to c1720ba Compare November 14, 2026 08:36
@PLPeeters
PLPeeters requested a review from webknjaz as a code owner November 14, 2026 08:36
@pyba-chronographer pyba-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Nov 14, 2026
@codecov

codecov Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.12%. Comparing base (fe09949) to head (c1720ba).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #14379      +/-   ##
==========================================
- Coverage   99.12%   99.12%   -0.01%     
==========================================
  Files         135      136       +1     
  Lines       53661    53713      +52     
  Branches     2737     2741       +4     
==========================================
+ Hits        53191    53242      +51     
  Misses        356      356              
- Partials      114      115       +1     
Flag Coverage Δ
Autobahn 21.66% <20.00%> (-0.01%) ⬇️
CI-GHA 99.05% <100.00%> (-0.01%) ⬇️
OS-Linux 98.73% <100.00%> (-0.01%) ⬇️
OS-Windows 97.48% <100.00%> (-0.01%) ⬇️
OS-macOS 98.34% <100.00%> (-0.01%) ⬇️
Py-3.11 98.29% <100.00%> (-0.01%) ⬇️
Py-3.12 98.37% <100.00%> (-0.01%) ⬇️
Py-3.13 98.36% <100.00%> (-0.01%) ⬇️
Py-3.14 98.42% <100.00%> (-0.01%) ⬇️
Py-3.15 98.43% <100.00%> (+<0.01%) ⬆️
Py-3.15t 98.27% <100.00%> (-0.02%) ⬇️
Py-pypy-3.12 96.95% <100.00%> (-0.02%) ⬇️
VM-macos-latest 98.34% <100.00%> (-0.01%) ⬇️
VM-ubuntu-latest 98.73% <100.00%> (-0.01%) ⬇️
VM-windows-latest 97.48% <100.00%> (-0.01%) ⬇️
cython-coverage 83.93% <9.09%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@codspeed

codspeed Bot commented Nov 14, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ 50 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 102 untouched benchmarks
🆕 3 new benchmarks
⏩ 83 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
🆕 test_release_waiters_across_distinct_hosts[1600] N/A 32.2 ms N/A
🆕 test_release_waiters_across_distinct_hosts[400] N/A 8.1 ms N/A
🆕 test_release_waiters_across_distinct_hosts[800] N/A 16.2 ms N/A

Comparing PLPeeters:bugfix/quadtratic-connector-wait (c1720ba) with master (fe09949)

Open in CodSpeed

Footnotes

  1. 83 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@PLPeeters

Copy link
Copy Markdown
Contributor Author

@codspeed Can you run the new bench tests against master so we have a comparison?

@codspeed

codspeed Bot commented Nov 14, 2026

Copy link
Copy Markdown

@codspeed Can you run the new bench tests against master so we have a comparison?

To let the performance wizard handle your request, please sign in to CodSpeed at codspeed.io so we can link your GitHub account, then comment again.

@PLPeeters

Copy link
Copy Markdown
Contributor Author

@codspeed Can you run the new bench tests against master so we have a comparison?

@codspeed

codspeed Bot commented Nov 14, 2026

Copy link
Copy Markdown

@codspeed Can you run the new bench tests against master so we have a comparison?

Only members of the aio-libs organization on CodSpeed can ask the performance wizard for help here. Once you've joined it on CodSpeed, comment again.

@PLPeeters

Copy link
Copy Markdown
Contributor Author

Saw that one coming but had to try haha

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant