Skip to content

fix: HGraph dirty-vector search dereferences an empty result heap - #2802

Closed
vsag-bot wants to merge 1 commit into
antgroup:0.18from
vsag-bot:codex/ai-fix-2801-2ff00610
Closed

vsag-bot wants to merge 1 commit into
antgroup:0.18from
vsag-bot:codex/ai-fix-2801-2ff00610

Conversation

@vsag-bot

@vsag-bot vsag-bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixed search- and construction-time empty candidate handling. The previously reproducible ASan overflow and dirty-vector result regression now pass under ASan/UBSan and TSAN.

Root Cause

Non-finite vectors can leave route-search and graph-construction candidate heaps empty. HGraph dereferenced those heaps, pruning accessed an empty neighbor vector, and disconnected dirty nodes could replace valid graph entry points.

Changes

  • Guarded empty route results in KNN, range, request, and construction descent paths.
  • Made mutual connection safely represent an empty candidate set.
  • Prevented disconnected nodes from replacing HGraph and Pyramid entry points.
  • Restored all newly created HGraph route levels after an unconnectable insertion.
  • Added deterministic dirty-vector addition/search and empty-pruning regressions.

Validation

  • clang-format-15 --dry-run --Werror <changed C++ files> && git diff --check: passed - Formatting and whitespace checks passed.
  • cmake --build build-tsan-ai-clean --parallel 4: passed - Complete TSAN build passed; final affected sources and targets rebuilt successfully.
  • TSAN focused unit tests: passed - HGraph and pruning regressions passed with 22 assertions and no TSAN diagnostics.
  • TSAN functests '(PR) HGraph Search with Dirty Vector': passed - 283 assertions passed with no TSAN diagnostics.
  • ASAN/UBSAN focused build and unit tests: passed - Both test binaries built; 22 focused assertions passed.
  • ASAN/UBSAN functests '(PR) HGraph Search with Dirty Vector': passed - 283 assertions passed; the prior heap-buffer-overflow no longer occurs.

Residual Risks

  • Native AArch64 execution was unavailable; the reported fault path was reproduced and verified under x86_64 ASan/UBSan.
  • The trusted parent process must update the pull-request status label.

Base branch: 0.18

Fixes #2801

Updated by vsag-bot following maintainer direction from @wxyucs.

@vsag-bot
vsag-bot requested review from inabao and wxyucs as code owners August 28, 2026 09:37
@vsag-bot vsag-bot added created-by-AI The issue is found and created by AI Agent kind/bug Bug fixes, defects, or unexpected behavior 修复程序错误、缺陷或异常行为 labels Aug 28, 2026
@vsag-bot

vsag-bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Collaborator Author

/label status/waiting-for-review
/waiting-on reviewer
/request-review @wxyucs
/request-review @inabao
/request-review @ShawnShawnYou

@pull-request-size pull-request-size Bot added the size/M 30-99 changed lines label Aug 28, 2026
@vsag-bot vsag-bot added the version/0.18 1. Major pyramid upgrade 2. HGraph 3. Observability 1. pyramid大升级 2. HGraph增强 3. 可观测性增强 label Aug 28, 2026
@vsag-bot vsag-bot added the status/waiting-for-ci Waiting for CI; the AI agent handles failures label Aug 28, 2026
@vsag-bot vsag-bot self-assigned this Aug 28, 2026
@vsag-bot

vsag-bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Collaborator Author

AI PR state: completed
Current owner: unassigned
Reason: PR state is closed
Head: ac61bba

Automatic merging is disabled. A maintainer performs the final merge.

@mergify

mergify Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Require linked issue for feature/bug PRs

  • body~=(?im)(?:^|[\s\-\*])(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s*:?\s+(?:#\d+|[\w.\-]+/[\w.\-]+#\d+|https?://github\.com/[\w.\-]+/[\w.\-]+/issues/\d+)

@LHT129 LHT129 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this fix. The change is clean and correct.

Summary of the fix:

  • Three call sites in hgraph.cpp (KnnSearch, RangeSearch, SearchWithRequest) now guard result->Top() with not result->Empty() when descending route graph layers, preventing dereference of an empty result heap caused by non-finite distance filtering (e.g. all-NaN queries).
  • A regression test in hgraph_add_test.cpp exercises the all-NaN query path and verifies the search returns an empty result set rather than crashing.

Review notes:

  • The fix is minimal and surgical — exactly what a bug fix should be.
  • When all route graph layers return empty, search_param.ep retains its initial value (entry_point_id_), which is already validated against INVALID_ENTRY_POINT before the loop. This is correct fallback behavior.
  • The test case is well-scoped: deterministic, self-contained, and directly targets the reported crash path.
  • No correctness, performance, or security concerns found.

@vsag-bot vsag-bot added status/ai-working The AI agent owns the next action and removed status/waiting-for-ci Waiting for CI; the AI agent handles failures labels Aug 28, 2026
@vsag-bot

vsag-bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Collaborator Author

AI PR CI recovery

Item Value
Status rerun-requested
Head ac61bba
Workflow Tsan Build & Test Parallel
Classification unrelated_transient
Confidence 98%
Action Requested a rerun of failed jobs

The failure is unrelated to the pull request. Rerun the same workflow; no repository changes were made.

Reasoning:
The PR changes only HGraph, Pyramid, pruning, and associated tests. It does not modify CI, CMake, Makefiles, external dependencies, or Docker configuration. CI failed while independently building the pinned OpenBLAS 0.3.23 dependency at step 39/1056, before any changed translation unit was reported. The Git ownership and Node.js messages were nonfatal warnings. Although the omitted OpenBLAS sub-log prevents identifying the precise runner fault, the dependency boundary establishes that this was not caused by the PR diff.

@codecov

codecov Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.29630% with 1 line in your changes missing coverage. Please review.

@@            Coverage Diff             @@
##             0.18    #2802      +/-   ##
==========================================
- Coverage   91.84%   91.76%   -0.08%     
==========================================
  Files         335      335              
  Lines       20078    20089      +11     
==========================================
- Hits        18440    18435       -5     
- Misses       1638     1654      +16     
Flag Coverage Δ
cpp 91.76% <96.29%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
common 86.03% <ø> (-0.18%) ⬇️
datacell 93.20% <ø> (-0.37%) ⬇️
index 91.73% <95.83%> (-0.06%) ⬇️
simd 100.00% <ø> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b07774e...ac61bba. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@wxyucs

wxyucs commented Sep 1, 2026 •

Copy link
Copy Markdown
Collaborator

@vsag-bot This PR remains labeled status/ai-working, but Build with TSAN and Test Aarch64 (functests) are still failing and there has been no update since August 28. Please inspect the current failures and continue advancing the PR.

@mergify mergify Bot added the module/index Index algorithms and implementations 索引算法与实现 label Sep 6, 2026
Comment thread src/algorithm/hgraph.cpp
@wxyucs

wxyucs commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

@vsag-bot Please resume this PR. The current head still has failures in Build with TSAN (https://github.com/antgroup/vsag/actions/runs/33160117157/job/98812268706) and Test Aarch64 (functests) (https://github.com/antgroup/vsag/actions/runs/33160117244/job/98841395297). Please inspect the logs, address repository defects, and update the status label to reflect the actual progress or blocker. Review threads are resolved and GitHub currently reports no conflicts.

@vsag-bot

vsag-bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator Author

AI PR continuation

Item Value
Status branch-updated
Requested by @wxyucs
Direction Maintainer comment
Base 0.18 (latest at checkout)
Branch codex/ai-fix-2801-2ff00610
Previous head 261072f
Updated head ac61bba

I rebuilt the work on the latest base, pushed the remaining fix, and CI will run for the updated head.

Summary:
Fixed search- and construction-time empty candidate handling. The previously reproducible ASan overflow and dirty-vector result regression now pass under ASan/UBSan and TSAN.

Root cause:
Non-finite vectors can leave route-search and graph-construction candidate heaps empty. HGraph dereferenced those heaps, pruning accessed an empty neighbor vector, and disconnected dirty nodes could replace valid graph entry points.

Changes:

  • Guarded empty route results in KNN, range, request, and construction descent paths.
  • Made mutual connection safely represent an empty candidate set.
  • Prevented disconnected nodes from replacing HGraph and Pyramid entry points.
  • Restored all newly created HGraph route levels after an unconnectable insertion.
  • Added deterministic dirty-vector addition/search and empty-pruning regressions.

Validation:

Command Result Detail
clang-format-15 --dry-run --Werror <changed C++ files> && git diff --check passed Formatting and whitespace checks passed.
cmake --build build-tsan-ai-clean --parallel 4 passed Complete TSAN build passed; final affected sources and targets rebuilt successfully.
TSAN focused unit tests passed HGraph and pruning regressions passed with 22 assertions and no TSAN diagnostics.
TSAN functests '(PR) HGraph Search with Dirty Vector' passed 283 assertions passed with no TSAN diagnostics.
ASAN/UBSAN focused build and unit tests passed Both test binaries built; 22 focused assertions passed.
ASAN/UBSAN functests '(PR) HGraph Search with Dirty Vector' passed 283 assertions passed; the prior heap-buffer-overflow no longer occurs.

Residual risks:

  • Native AArch64 execution was unavailable; the reported fault path was reproduced and verified under x86_64 ASan/UBSan.
  • The trusted parent process must update the pull-request status label.

Signed-off-by: vsag-bot <276218163+vsag-bot@users.noreply.github.com>
Assisted-by: Codex:codex-default
@vsag-bot
vsag-bot force-pushed the codex/ai-fix-2801-2ff00610 branch from 261072f to ac61bba Compare September 17, 2026 11:00
@pull-request-size pull-request-size Bot added size/L 100-499 changed lines and removed size/M 30-99 changed lines labels Sep 17, 2026
@vsag-bot vsag-bot added status/waiting-for-ci Waiting for CI; the AI agent handles failures and removed status/ai-working The AI agent owns the next action status/waiting-for-ci Waiting for CI; the AI agent handles failures labels Sep 17, 2026
@vsag-bot vsag-bot added the status/ai-working The AI agent owns the next action label Sep 17, 2026
@wxyucs

wxyucs commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

@vsag-bot Thanks for updating the branch. Fresh CI on ac61bba now fails in the bundled OpenBLAS build, including ASan X86 (https://github.com/antgroup/vsag/actions/runs/35213407162/job/105176092608) and Lint (https://github.com/antgroup/vsag/actions/runs/35213407034/job/105177637281); the lint job failed during its prerequisite build, not an identified clang-tidy finding. TSan build also failed (https://github.com/antgroup/vsag/actions/runs/35213406968/job/105179288229). Please inspect detailed dependency logs and continue. Related historical tracking: #2507; #2971 is a 1.0 dependency repair, not proof of the same cause on 0.18. Do not assume an OOM or CPU-detection cause without evidence, or change HGraph code merely to address an unrelated dependency failure.

@wxyucs

wxyucs commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

@vsag-bot Following up on the outstanding dependency-build failures: this PR still has the same head and failed ASan/Lint/TSan builds after the previous continuation request. Please inspect the OpenBLAS detailed logs, determine whether a scoped dependency fix/backport is needed on 0.18, and update the status label with the actual blocker. The 1.0 dependency fix #2971 has merged, but its applicability to these failures must be verified rather than assumed. Do not change HGraph behavior to hide an unrelated build failure.

@wxyucs wxyucs closed this Sep 23, 2026
@vsag-bot vsag-bot removed the status/ai-working The AI agent owns the next action label Sep 23, 2026
@vsag-bot vsag-bot removed their assignment Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

created-by-AI The issue is found and created by AI Agent kind/bug Bug fixes, defects, or unexpected behavior 修复程序错误、缺陷或异常行为 module/index Index algorithms and implementations 索引算法与实现 size/L 100-499 changed lines version/0.18 1. Major pyramid upgrade 2. HGraph 3. Observability 1. pyramid大升级 2. HGraph增强 3. 可观测性增强

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants