[Selena Yang (selenayang888)]

Summary

Add an authoritative EP/device API for querying total and currently available device memory. This provides capacity information for downstream consumers such as Foundry Local and ONNX Runtime GenAI without requiring provider-specific memory queries in those consumers.

Changes

• Add OrtApi::EpDevice_GetMemoryCapacity and a C++ GetMemoryCapacity() wrapper.
• Add an optional OrtEpFactory::GetDeviceMemoryCapacity callback, with internal factory and provider-bridge forwarding.
• Return ORT_NOT_IMPLEMENTED when an EP does not support the query.
• Preserve output values on failure.
• Implement memory queries in both CUDA factories using cuDeviceTotalMem for device-global capacity and cuMemGetInfo for current free memory.
• Preserve the calling thread's exact CUDA context using context push/pop.
• Avoid allocating string construction in noexcept query error paths.
• Document partition-aware capacity, free-memory snapshot semantics, and possible provider-resource initialization.
• Add example-plugin coverage and deterministic regression tests.

API Semantics

Total memory is the device-global capacity exposed for the specific EP device, not a per-process or per-context allocation budget. For partitioned or virtual devices, it reflects the exposed device rather than the entire physical board.

Available memory is a current free-memory estimate. It is not a reservation or a guarantee that an allocation of that size will succeed.

Validation

• CPU/shared runtime and AutoEP target build succeeded.
• OrtEpLibrary.*: 66 passed, 1 GPU-dependent test skipped.
• No memory leaks detected by the test harness.
• Repository lint and whitespace checks passed.
• Deterministic tests cover context restoration, dynamic free-memory values, device-global totals, and driver failure paths.

CUDA-enabled compilation and real GPU validation remain outstanding because the local environment has no CUDA toolkit. The deterministic tests exercise the shared query logic without a CUDA dependency.

Scope

This PR provides the ORT/EP capacity-reporting foundation. It does not change Foundry Local catalog filtering or migrate GenAI runtime-profile selection to the new API.

[Copilot]

🟡 Changes recommended

The minimal CUDA build lacks the required driver-library dependency, and the new mock capacities overflow on 32-bit targets.

2 open issues

• Link CUDA minimal builds against the CUDA driver library · New
• Use portable mock capacities for 32-bit AutoEP tests · New

What changed in this PR

Adds a provider-independent API for querying device memory capacity, giving downstream consumers access to total and currently available memory through ONNX Runtime.

Changes:

• Adds C and C++ query APIs with an optional execution-provider factory callback.
• Implements CUDA queries through shared logic that manages context switching and preserves outputs on failure.
• Adds example-plugin coverage and deterministic regression tests.

File	Description
onnxruntime/​test/​autoep/​test_registration.cc	Adds API, context, and failure-path tests.
onnxruntime/​test/​autoep/​library/​example_plugin_ep/​ep_factory.h	Declares the example callback.
onnxruntime/​test/​autoep/​library/​example_plugin_ep/​ep_factory.cc	Provides synthetic memory capacities.
onnxruntime/​core/​session/​plugin_ep/​forward_to_factory_impl.h	Forwards queries to internal factories.
onnxruntime/​core/​session/​plugin_ep/​ep_factory_provider_bridge.h	Adds version-checked provider forwarding.
onnxruntime/​core/​session/​plugin_ep/​ep_factory_internal.h	Exposes internal query forwarding.
onnxruntime/​core/​session/​plugin_ep/​ep_factory_internal.cc	Registers the forwarding callback.
onnxruntime/​core/​session/​plugin_ep/​ep_factory_internal_impl.h	Supplies the unsupported-query default.
onnxruntime/​core/​session/​ort_apis.h	Declares the C API implementation.
onnxruntime/​core/​session/​onnxruntime_c_api.cc	Validates, dispatches, and registers queries.
onnxruntime/​core/​providers/​cuda/​shared_inc/​device_memory_capacity.h	Adds testable query and cleanup logic.
onnxruntime/​core/​providers/​cuda/​shared_inc/​cuda_memory_capacity.h	Wraps CUDA Driver API calls.
onnxruntime/​core/​providers/​cuda/​plugin/​cuda_ep_factory.h	Declares the CUDA plugin callback.
onnxruntime/​core/​providers/​cuda/​plugin/​cuda_ep_factory.cc	Implements CUDA plugin queries.
onnxruntime/​core/​providers/​cuda/​cuda_provider_factory.cc	Implements shared-provider CUDA queries.
include/​onnxruntime/​core/​session/​onnxruntime_ep_c_api.h	Appends and documents the optional callback.
include/​onnxruntime/​core/​session/​onnxruntime_cxx_inline.h	Implements the C++ wrapper.
include/​onnxruntime/​core/​session/​onnxruntime_cxx_api.h	Declares the C++ wrapper.
include/​onnxruntime/​core/​session/​onnxruntime_c_api.h	Appends and documents the public query API.

_{Review effort: Balanced}