Implement GetAvaiableResource() callback

[Dmitri Smirnov (yuslepukhin)]

This pull request introduces a new resource accounting mechanism to the ONNX Runtime Execution Provider (EP) plugin API, enabling execution providers to report available device resources (such as free GPU memory) for use in budget enforcement during graph partitioning. The main changes include the definition of an ABI-stable OrtResourceCount struct and enum, a new optional API for querying available resources, and a CUDA EP implementation that reports free device memory. Host-side logic is updated to use this information when explicit session memory limits are not set.

API and Struct Additions:

• Introduced the OrtResourceCountKind enum and ABI-stable OrtResourceCount struct to represent device resource counts in a forward-compatible way, including C++ helpers and ABI stability checks. (include/onnxruntime/core/session/onnxruntime_ep_c_api.h)
• Added the optional GetAvailableResource function pointer to the OrtEp struct, allowing EPs to report available device resources for partitioning budget. (include/onnxruntime/core/session/onnxruntime_ep_c_api.h)

CUDA EP Implementation:

• Implemented GetAvailableResourceImpl in CudaEp, which queries the current device for free memory using cudaMemGetInfo and returns it as an OrtResourceCount. (onnxruntime/core/providers/cuda/plugin/cuda_ep.cc, onnxruntime/core/providers/cuda/plugin/cuda_ep.h) [1] [2]
• Registered the new API in the CUDA EP constructor. (onnxruntime/core/providers/cuda/plugin/cuda_ep.cc)

Host-Side Integration:

• Updated the host-side plugin EP logic to call GetAvailableResource if no explicit threshold is set, using the returned value as the resource budget for partitioning (mirroring in-tree CUDA EP behavior). (onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc)
• Added necessary includes for type narrowing. (onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc)

[Copilot]

Pull request overview

This PR extends the Execution Provider (EP) plugin API with an optional resource-query callback so ORT can derive a partitioning budget (e.g., free GPU memory) when no explicit session threshold is configured, and wires up a CUDA plugin EP implementation plus host-side integration.

Changes:

• Added ABI-stable OrtResourceCount / OrtResourceCountKind and a new optional OrtEp::GetAvailableResource callback to the EP C API.
• Implemented CUDA plugin EP reporting of available device memory via cudaMemGetInfo.
• Updated host-side plugin EP capability flow to query GetAvailableResource and use it as a resource-accounting threshold when none is configured.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.

File	Description
include/onnxruntime/core/session/onnxruntime_ep_c_api.h	Defines new resource-count ABI type + adds OrtEp::GetAvailableResource to the plugin EP API.
onnxruntime/core/providers/cuda/plugin/cuda_ep.h	Declares CUDA plugin EP callback implementation for resource querying.
onnxruntime/core/providers/cuda/plugin/cuda_ep.cc	Implements and registers CUDA plugin EP resource query using cudaMemGetInfo.
onnxruntime/core/session/plugin_ep/ep_plugin_provider_interfaces.cc	Host-side: uses GetAvailableResource to set a default resource threshold for budget enforcement.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Tianlei Wu (tianleiwu)]

Overall: Clean ABI extension following established patterns (version-gated function pointer, C-safe tagged union, EXCEPTION_TO_STATUS wrapping). Previous automated review feedback has been addressed in this head (null-check on available, clamping instead of narrow<>, warning log for unsupported kinds). Comprehensive test coverage with both unit-level and integration-level tests.

Remaining items are minor quality-of-life improvements, none blocking.