Skip to content

Honor non-blocking pinned-memory copies - #35

Merged
voltjia merged 2 commits into
masterfrom
feature/nonblocking-copy-lifetime
Jul 25, 2026
Merged

Honor non-blocking pinned-memory copies#35
voltjia merged 2 commits into
masterfrom
feature/nonblocking-copy-lifetime

Conversation

@voltjia

@voltjia voltjia commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR allows contiguous CPU-to-Infini and Infini-to-CPU copies to return asynchronously only when non_blocking=True, the CPU tensor uses torch-infini pinned memory, the runtime advertises asynchronous memcpy and pinned-host allocation, and the device storage lifetime can be tracked.

It records both host and device storage use on the active stream and waits for recorded work before releasing either allocation. Sliced pinned tensors and alternative DataPtr contexts are resolved through the HostAllocator context and live pointer range contracts.

Copies involving ordinary host memory, unsupported runtime capabilities, or external device storage continue to execute in stream order and synchronize before returning without raising after a successful copy.

It documents the supported contract and adds behavioral tests for early return, immediate storage release, sliced storage, alternative contexts, external allocators, and CPU capability fallback.

This advances #10. This PR is stacked on #32 because the lifecycle tests create enough streams to require the stream registry teardown fix.

Validation

  • Ruff and clang-format 19.1.4 checks passed.
  • PyTorch 2.12.0+cu130 NVIDIA installed-wheel tests passed with 18 tests passed and 1 skipped.
  • PyTorch 2.13.0+cu130 NVIDIA installed-wheel tests passed with 18 tests passed and 1 skipped.
  • The complete installed-wheel CPU fallback suite passed with 143 tests passed, 42 skipped, and 3 expected failures.

Base automatically changed from fix/stream-registry-teardown to master July 25, 2026 06:40
@voltjia
voltjia merged commit 1774313 into master Jul 25, 2026
8 checks passed
@voltjia
voltjia deleted the feature/nonblocking-copy-lifetime branch July 25, 2026 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant