Skip to content

fix: retain successful backend results after another engine fails - #489

Open
roli-lpci wants to merge 2 commits into
deedy5:mainfrom
roli-lpci:fix/collect-pending-search-results
Open

roli-lpci wants to merge 2 commits into
deedy5:mainfrom
roli-lpci:fix/collect-pending-search-results

Conversation

@roli-lpci

Copy link
Copy Markdown

When one backend raises before a slower sibling finishes, _search_sync() can discard that sibling's successful results and raise DDGSException. The collector now processes remaining submitted futures before extracting results. The executor was already waiting for those futures during shutdown; the existing result limit still applies. Collecting a late failure preserves any previously recorded error, including timeout classification.

Fixes #427.

Validation:

  • Five offline engine-stub regression cases pass, covering pending successful results, duplicate prevention, all-backend failure, and preservation of earlier timeout/general errors. The primary regression fails against unchanged upstream code; the two error-preservation cases failed before the follow-up correction.
  • make lint passes (Ruff and mypy, 33 source files).
  • Final full pytest run: 18 passed, 4 failed. The same books/video integration tests also fail on unchanged upstream commit 70a5635; their failures are not introduced by this patch. Existing integration tests and their inter-test pauses are unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_search drops results from successful backends when another backend raises first

1 participant