Feat/tensor transport#73
Open
wyeth93 wants to merge 3 commits into
Open
Conversation
added 3 commits
July 16, 2026 15:32
Add direct raw-TCP tensor frames with HTTP/2 compatibility, explicit Python buffer ownership, transport statistics, and actor dispatch support. Include bilingual design documentation, a runnable example, and Rust/Python coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(transport): add TensorMessage fast path
Summary
This PR adds
TensorMessage, a transport-oriented message type for moving anordered set of CPU tensor buffers together with opaque metadata. It introduces
a direct TCP data path for compatible plain connections while preserving the
existing HTTP/2 packed-message path for TLS and explicitly configured legacy
compatibility modes.
The result is a lower-overhead remote tensor path without changing the existing
Message/ RPC API for callers that do not opt in.What changed
Core transport
TensorMessage { version, metadata, buffers }and the associatedencode/decode support.
PTR1).A checked-out raw connection carries one request/response pair and is then
returned to the pool; it is intentionally not multiplexed.
PTM1) for TLS and explicit HTTP/2compatibility modes.
receive side; add pooling, size limits, and transport statistics.
Python API and dispatch
pulsing.TensorMessageandpulsing.tensor_transport_stats()whenthe native binding supports them.
receive_tensordispatch andActorRef.tell/ActorRef.asksupport for tensor messages.
receive_tensorfrom generic RPC dispatch so tensor payloads cannotaccidentally take the normal message handler path.
and attempting to convert a
TensorMessagethrough that binding raises aclear unsupported error instead of failing an exhaustive Rust match.
Compatibility selection
auto,raw, or unset transportPTR1)PTM1)http2,legacy, oroffPTM1)http2until all peers support raw transportThere is deliberately no silent fallback from a selected raw path to HTTP/2:
configuration and rollout policy should make the wire protocol unambiguous.
Semantics and limitations
for staging CUDA tensors to CPU and making non-contiguous tensors contiguous.
or schema information.
raw and HTTP/2 deliveries produce independent, writable receive allocations.
but it is not network zero-copy: the data still traverses application-to-kernel
and kernel-to-final-buffer copies.
tellacknowledgement means transport enqueue, not handler completion ordurability. Use
askwith an application-level acknowledgement when thatguarantee is required.
capability negotiation, automatic downgrade/retry, checksums, compression,
encryption, request IDs, or streaming tensor responses.
Versioning and documentation
0.1.3.TensorMessageand the transport-mode behavior.
Validation
Validated using
/home/guoxu1/transferqueue_2/.venvand Maturin1.11.5:The tensor-specific Python suite passes (
12 passed), including raw transport,HTTP/2 fallback, local lifetime behavior, actor dispatch, and reconnect cases.
The full Python suite also passes on a clean Codex-home configuration; the only
failure observed with a pre-populated local Codex cache is an unrelated existing
Forge plugin-catalog attribute mismatch from
main.