Skip to content

Commit 860d085

Browse files
[Plugin EP] Allow EP to provide additional virtual devices (#26234)
### Description Adds APIs to allow a plugin EP to create a virtual `OrtHardwareDevice` that can be used for model cross-compilation. For example, this allows an EP to create a compiled model for NPU on a device that does not have an NPU. #### Application code An application must explicitly allow registered plugin EPs to create virtual devices. This is currently done by using a registration name that ends in the `".virtual"` suffix. Ex: ```c++ #include "onnxruntime_cxx_api.h" #include "onnxruntime_ep_device_ep_metadata_keys.h" const char* ep_registration_name = "my_ep_lib.virtual"; // IMPORTANT: ".virtual" suffix is a signal to EP library ort_env->RegisterExecutionProviderLibrary(ep_registration_name, "my_ep.dll"); std::vector<Ort::ConstEpDevice> ep_devices = ort_env->GetEpDevices(); // ep_devices includes an OrtEpDevice from "my_ep.dll" that uses a virtual OrtHardwareDevice. Ort::ConstEpDevice virtual_ep_device = std::find_if(ep_devices.begin(), ep_devices.end(), [](Ort::ConstEpDevice& device) { return device.EpName() == std::string("MyEpName"); }); // App can look in HW metadata to check if is virtual Ort::ConstHardwareDevice virtual_hw_device = virtual_ep_device.Device(); std::unordered_map<std::string, std::string> metadata = virtual_hw_device.Metadata().GetKeyValuePairs(); assert(metadata[kOrtHardwareDevice_MetadataKey_IsVirtual] == "1"); // App can use the virtual OrtEpDevice in a session to, for example, compile a model // ... ``` #### Plugin EP code This PR introduces a new _optional_ C API function in the `OrtEpFactory` struct called `SetEnvironmentOptions` that allows ORT to pass options (as key/value pairs) to an EP factory. Currently, the only key supported is `"allow_virtual_devices"`, which indicates to the EP factory that creating virtual devices is allowed. When the application registers a plugin EP library, ORT creates the library's EP factories and checks if they implement the `SetEnvironmentOptions` API function. If so, ORT calls `ep_factory.SetEnvironmentOptions` with `"allow_virtual_devices"` set to `"1"` if the EP registration name set by the application ends in the `".virtual"` suffix (or `"0"` otherwise). Here's an example implementation of `OrtEpFactory::SetEnvironmentOptions` taken from a [test plugin EP that supports a virtual GPU](https://github.com/microsoft/onnxruntime/tree/adrianl/plugin-ep-specify-ort-hw-device/onnxruntime/test/autoep/library/example_plugin_ep_virt_gpu): ```c++ /*static*/ OrtStatus* ORT_API_CALL EpFactoryVirtualGpu::SetEnvironmentOptionsImpl(OrtEpFactory* this_ptr, const OrtKeyValuePairs* options) noexcept { auto* factory = static_cast<EpFactoryVirtualGpu*>(this_ptr); const char* value = factory->ort_api_.GetKeyValue(options, "allow_virtual_devices"); if (value != nullptr) { factory->allow_virtual_devices_ = strcmp(value, "1") == 0; } return nullptr; } ``` An EP factory can create a virtual hardware device within `OrtEpFactory::GetSupportedDevices` by using a new API function called `CreateHardwareDevice`. The EP factory is expected to own the hardware device instance, which should be released when the factory is destroyed via `ReleaseHardwareDevice`. The [test plugin EP shows an implementation](https://github.com/microsoft/onnxruntime/blob/d87f8b86406525f5801a7a9933b1ced1eb40940c/onnxruntime/test/autoep/library/example_plugin_ep_virt_gpu/ep_factory.cc#L86) of `OrtEpFactory::GetSupportedDevices` that creates a virtual GPU device. ```c++ /*static*/ OrtStatus* ORT_API_CALL EpFactoryVirtualGpu::GetSupportedDevicesImpl(OrtEpFactory* this_ptr, const OrtHardwareDevice* const* /*devices*/, size_t /*num_devices*/, OrtEpDevice** ep_devices, size_t max_ep_devices, size_t* p_num_ep_devices) noexcept { size_t& num_ep_devices = *p_num_ep_devices; auto* factory = static_cast<EpFactoryVirtualGpu*>(this_ptr); num_ep_devices = 0; // Create a virtual OrtHardwareDevice if application indicated it is allowed (e.g., for cross-compiling). // This example EP creates a virtual GPU OrtHardwareDevice and adds a new OrtEpDevice that uses the virtual GPU. if (factory->allow_virtual_devices_ && num_ep_devices < max_ep_devices) { OrtKeyValuePairs* hw_metadata = nullptr; factory->ort_api_.CreateKeyValuePairs(&hw_metadata); factory->ort_api_.AddKeyValuePair(hw_metadata, kOrtHardwareDevice_MetadataKey_IsVirtual, "1"); auto* status = factory->ep_api_.CreateHardwareDevice(OrtHardwareDeviceType::OrtHardwareDeviceType_GPU, factory->vendor_id_, /*device_id*/ 0, factory->vendor_.c_str(), hw_metadata, &factory->virtual_hw_device_); // ... OrtEpDevice* virtual_ep_device = nullptr; status = factory->ort_api_.GetEpApi()->CreateEpDevice(factory, factory->virtual_hw_device_, ep_metadata, ep_options, &virtual_ep_device); // ... ep_devices[num_ep_devices++] = virtual_ep_device; ``` ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. -->
1 parent a958614 commit 860d085

40 files changed

+1152
-100
lines changed

cmake/onnxruntime_unittests.cmake

Lines changed: 44 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1982,9 +1982,13 @@ endif()
19821982
if (WIN32 AND onnxruntime_BUILD_SHARED_LIB AND
19831983
NOT CMAKE_SYSTEM_NAME STREQUAL "Emscripten" AND
19841984
NOT onnxruntime_MINIMAL_BUILD)
1985+
1986+
#
19851987
# example_plugin_ep
1986-
file(GLOB onnxruntime_autoep_test_library_src "${TEST_SRC_DIR}/autoep/library/*.h"
1987-
"${TEST_SRC_DIR}/autoep/library/*.cc")
1988+
#
1989+
file(GLOB onnxruntime_autoep_test_library_src "${TEST_SRC_DIR}/autoep/library/example_plugin_ep/*.h"
1990+
"${TEST_SRC_DIR}/autoep/library/example_plugin_ep/*.cc"
1991+
"${TEST_SRC_DIR}/autoep/library/plugin_ep_utils.h")
19881992
onnxruntime_add_shared_library_module(example_plugin_ep ${onnxruntime_autoep_test_library_src})
19891993
target_include_directories(example_plugin_ep PRIVATE ${REPO_ROOT}/include/onnxruntime/core/session)
19901994
target_link_libraries(example_plugin_ep PRIVATE onnxruntime)
@@ -1994,12 +1998,12 @@ if (WIN32 AND onnxruntime_BUILD_SHARED_LIB AND
19941998
set(ONNXRUNTIME_AUTOEP_LIB_LINK_FLAG "-Xlinker -dead_strip")
19951999
elseif (NOT CMAKE_SYSTEM_NAME MATCHES "AIX")
19962000
string(CONCAT ONNXRUNTIME_AUTOEP_LIB_LINK_FLAG
1997-
"-Xlinker --version-script=${TEST_SRC_DIR}/autoep/library/example_plugin_ep_library.lds "
2001+
"-Xlinker --version-script=${TEST_SRC_DIR}/autoep/library/example_plugin_ep/example_plugin_ep_library.lds "
19982002
"-Xlinker --no-undefined -Xlinker --gc-sections -z noexecstack")
19992003
endif()
20002004
else()
20012005
set(ONNXRUNTIME_AUTOEP_LIB_LINK_FLAG
2002-
"-DEF:${TEST_SRC_DIR}/autoep/library/example_plugin_ep_library.def")
2006+
"-DEF:${TEST_SRC_DIR}/autoep/library/example_plugin_ep/example_plugin_ep_library.def")
20032007
endif()
20042008

20052009
set_property(TARGET example_plugin_ep APPEND_STRING PROPERTY LINK_FLAGS
@@ -2008,7 +2012,42 @@ if (WIN32 AND onnxruntime_BUILD_SHARED_LIB AND
20082012
set_target_properties(example_plugin_ep PROPERTIES FOLDER "ONNXRuntimeTest")
20092013
source_group(TREE ${TEST_SRC_DIR} FILES ${onnxruntime_autoep_test_library_src})
20102014

2015+
#
2016+
# example_plugin_ep_virt_gpu
2017+
#
2018+
set(onnxruntime_autoep_test_example_plugin_ep_virt_gpu_src
2019+
"${TEST_SRC_DIR}/autoep/library/plugin_ep_utils.h"
2020+
"${TEST_SRC_DIR}/autoep/library/example_plugin_ep_virt_gpu/ep_lib_entry.cc"
2021+
"${TEST_SRC_DIR}/autoep/library/example_plugin_ep_virt_gpu/ep_factory.h"
2022+
"${TEST_SRC_DIR}/autoep/library/example_plugin_ep_virt_gpu/ep_factory.cc"
2023+
"${TEST_SRC_DIR}/autoep/library/example_plugin_ep_virt_gpu/ep.h"
2024+
"${TEST_SRC_DIR}/autoep/library/example_plugin_ep_virt_gpu/ep.cc")
2025+
onnxruntime_add_shared_library_module(example_plugin_ep_virt_gpu ${onnxruntime_autoep_test_example_plugin_ep_virt_gpu_src})
2026+
target_include_directories(example_plugin_ep_virt_gpu PRIVATE ${REPO_ROOT}/include/onnxruntime/core/session)
2027+
target_link_libraries(example_plugin_ep_virt_gpu PRIVATE onnxruntime)
2028+
2029+
if(UNIX)
2030+
if (APPLE)
2031+
set(ONNXRUNTIME_AUTOEP_EP_LIB_VIRT_GPU_LINK_FLAG "-Xlinker -dead_strip")
2032+
elseif (NOT CMAKE_SYSTEM_NAME MATCHES "AIX")
2033+
string(CONCAT ONNXRUNTIME_AUTOEP_EP_LIB_VIRT_GPU_LINK_FLAG
2034+
"-Xlinker --version-script=${TEST_SRC_DIR}/autoep/library/example_plugin_ep_virt_gpu/ep_lib.lds "
2035+
"-Xlinker --no-undefined -Xlinker --gc-sections -z noexecstack")
2036+
endif()
2037+
else()
2038+
set(ONNXRUNTIME_AUTOEP_EP_LIB_VIRT_GPU_LINK_FLAG
2039+
"-DEF:${TEST_SRC_DIR}/autoep/library/example_plugin_ep_virt_gpu/ep_lib.def")
2040+
endif()
2041+
2042+
set_property(TARGET example_plugin_ep_virt_gpu APPEND_STRING PROPERTY LINK_FLAGS
2043+
${ONNXRUNTIME_AUTOEP_EP_LIB_VIRT_GPU_LINK_FLAG})
2044+
2045+
set_target_properties(example_plugin_ep_virt_gpu PROPERTIES FOLDER "ONNXRuntimeTest")
2046+
source_group(TREE ${TEST_SRC_DIR} FILES ${onnxruntime_autoep_test_example_plugin_ep_virt_gpu_src})
2047+
2048+
#
20112049
# test library
2050+
#
20122051
file(GLOB onnxruntime_autoep_test_SRC "${ONNXRUNTIME_AUTOEP_TEST_SRC_DIR}/*.h"
20132052
"${ONNXRUNTIME_AUTOEP_TEST_SRC_DIR}/*.cc")
20142053

@@ -2041,7 +2080,7 @@ if (WIN32 AND onnxruntime_BUILD_SHARED_LIB AND
20412080
TARGET onnxruntime_autoep_test
20422081
SOURCES ${onnxruntime_autoep_test_SRC} ${onnxruntime_unittest_main_src}
20432082
LIBS ${onnxruntime_autoep_test_LIBS}
2044-
DEPENDS ${all_dependencies} example_plugin_ep
2083+
DEPENDS ${all_dependencies} example_plugin_ep example_plugin_ep_virt_gpu
20452084
)
20462085
endif()
20472086

include/onnxruntime/core/session/onnxruntime_ep_c_api.h

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -465,6 +465,33 @@ struct OrtEpApi {
465465
*/
466466
ORT_API_T(uint64_t, GetSyncIdForLastWaitOnSyncStream,
467467
_In_ const OrtSyncStream* producer_stream, _In_ const OrtSyncStream* consumer_stream);
468+
469+
/** \brief Create an OrtHardwareDevice.
470+
*
471+
* \note Called within OrtEpFactory::GetSupportedDevices to create a new hardware device (e.g., virtual).
472+
*
473+
* \param[in] type The hardware device type.
474+
* \param[in] vendor_id The hardware device's vendor identifier.
475+
* \param[in] device_id The hardware device's identifier.
476+
* \param[in] vendor_name The hardware device's vendor name as a null-terminated string. Copied by ORT.
477+
* \param[in] metadata Optional OrtKeyValuePairs instance for hardware device metadata that may be queried by
478+
* applications via OrtApi::GetEpDevices().
479+
* Refer to onnxruntime_ep_device_ep_metadata_keys.h for common OrtHardwareDevice metadata keys.
480+
* \param[out] hardware_device Output parameter set to the new OrtHardwareDevice instance that is created.
481+
* Must be release with ReleaseHardwareDevice().
482+
*
483+
* \snippet{doc} snippets.dox OrtStatus Return Value
484+
*
485+
* \since Version 1.24.
486+
*/
487+
ORT_API2_STATUS(CreateHardwareDevice, _In_ OrtHardwareDeviceType type,
488+
_In_ uint32_t vendor_id,
489+
_In_ uint32_t device_id,
490+
_In_ const char* vendor_name,
491+
_In_opt_ const OrtKeyValuePairs* metadata,
492+
_Out_ OrtHardwareDevice** hardware_device);
493+
494+
ORT_CLASS_RELEASE(HardwareDevice);
468495
};
469496

470497
/**
@@ -981,6 +1008,35 @@ struct OrtEpFactory {
9811008
_In_ const OrtMemoryDevice* memory_device,
9821009
_In_opt_ const OrtKeyValuePairs* stream_options,
9831010
_Outptr_ OrtSyncStreamImpl** stream);
1011+
1012+
/** \brief Set environment options on this EP factory.
1013+
*
1014+
* Environment options can be set by ORT after calling the library's 'CreateEpFactories' function to
1015+
* create EP factories.
1016+
*
1017+
* Supported options:
1018+
* "allow_virtual_devices": Allows EP factory to specify OrtEpDevice instances that use custom
1019+
* virtual OrtHardwareDevices, which can be created via OrtEpApi::CreateHardwareDevice().
1020+
*
1021+
* A virtual OrtHardwareDevice does not represent actual hardware on the device, and is identified
1022+
* via the metadata entry "is_virtual" with a value of "1".
1023+
* Refer to onnxruntime_ep_device_ep_metadata_keys.h for well-known OrtHardwareDevice metadata keys.
1024+
*
1025+
* Allowed values:
1026+
* -# "0": Default. Creation of virtual devices is not allowed.
1027+
* -# "1": Creation of virtual devices is allowed.
1028+
*
1029+
* \param[in] this_ptr The OrtEpFactory instance.
1030+
* \param[in] options The configuration options.
1031+
*
1032+
* \note Implementation of this function is optional.
1033+
* An EP factory should only implement this if it needs to handle any environment options.
1034+
*
1035+
* \snippet{doc} snippets.dox OrtStatus Return Value
1036+
*
1037+
* \since Version 1.24.
1038+
*/
1039+
ORT_API2_STATUS(SetEnvironmentOptions, _In_ OrtEpFactory* this_ptr, _In_ const OrtKeyValuePairs* options);
9841040
};
9851041

9861042
#ifdef __cplusplus

include/onnxruntime/core/session/onnxruntime_ep_device_ep_metadata_keys.h

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33

44
#pragma once
55

6-
// This file contains well-known keys for OrtEpDevice EP metadata entries.
6+
// This file contains well-known keys for OrtEpDevice and OrtHardwareDevice metadata entries.
77
// It does NOT specify all available metadata keys.
88

99
// Key for the execution provider version string. This should be available for all plugin EPs.
@@ -16,3 +16,10 @@ static const char* const kOrtModelMetadata_EpCompatibilityInfoPrefix = "ep_compa
1616

1717
// Key for the execution provider library path (for dynamically loaded EPs)
1818
static const char* const kOrtEpDevice_EpMetadataKey_LibraryPath = "library_path";
19+
20+
// Optional metadata key to determine if a OrtHardwareDevice represents a virtual (non-hardware) device.
21+
// Possible values:
22+
// - "0": OrtHardwareDevice is not virtual (i.e., actual hardware device). This is the assumed default value
23+
// if this metadata key is not present.
24+
// - "1": OrtHardwareDevice is virtual.
25+
static const char* const kOrtHardwareDevice_MetadataKey_IsVirtual = "is_virtual";

onnxruntime/core/session/environment.cc

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -746,6 +746,29 @@ std::vector<const OrtHardwareDevice*> SortDevicesByType() {
746746

747747
return sorted_devices;
748748
}
749+
750+
bool AreVirtualDevicesAllowed(std::string_view lib_registration_name) {
751+
constexpr std::string_view suffix{".virtual"};
752+
753+
return lib_registration_name.size() >= suffix.size() &&
754+
lib_registration_name.compare(lib_registration_name.size() - suffix.size(),
755+
suffix.size(), suffix) == 0;
756+
}
757+
758+
Status SetEpFactoryEnvironmentOptions(OrtEpFactory& factory, std::string_view lib_registration_name) {
759+
// OrtEpFactory::SetEnvironmentOptions was added in ORT 1.24
760+
if (factory.ort_version_supported < 24 || factory.SetEnvironmentOptions == nullptr) {
761+
return Status::OK();
762+
}
763+
764+
// We only set one option now but this can be generalized if necessary.
765+
OrtKeyValuePairs options;
766+
options.Add("allow_virtual_devices", AreVirtualDevicesAllowed(lib_registration_name) ? "1" : "0");
767+
768+
ORT_RETURN_IF_ERROR(ToStatusAndRelease(factory.SetEnvironmentOptions(&factory, &options)));
769+
770+
return Status::OK();
771+
}
749772
} // namespace
750773

751774
Status Environment::EpInfo::Create(std::unique_ptr<EpLibrary> library_in, std::unique_ptr<EpInfo>& out,
@@ -772,6 +795,8 @@ Status Environment::EpInfo::Create(std::unique_ptr<EpLibrary> library_in, std::u
772795

773796
auto& factory = *factory_ptr;
774797

798+
ORT_RETURN_IF_ERROR(SetEpFactoryEnvironmentOptions(factory, instance.library->RegistrationName()));
799+
775800
std::array<OrtEpDevice*, 8> ep_devices{nullptr};
776801
size_t num_ep_devices = 0;
777802
ORT_RETURN_IF_ERROR(ToStatusAndRelease(

onnxruntime/core/session/plugin_ep/ep_api.cc

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44
#include "core/session/plugin_ep/ep_api.h"
55

66
#include <algorithm>
7+
#include <memory>
8+
#include <string>
79
#include <vector>
810

911
#include "core/common/semver.h"
@@ -205,6 +207,32 @@ ORT_API(uint64_t, GetSyncIdForLastWaitOnSyncStream, _In_ const OrtSyncStream* pr
205207
return id;
206208
}
207209

210+
ORT_API_STATUS_IMPL(CreateHardwareDevice, _In_ OrtHardwareDeviceType type,
211+
_In_ uint32_t vendor_id,
212+
_In_ uint32_t device_id,
213+
_In_ const char* vendor_name,
214+
_In_opt_ const OrtKeyValuePairs* metadata,
215+
_Out_ OrtHardwareDevice** hardware_device) {
216+
API_IMPL_BEGIN
217+
auto device = std::make_unique<OrtHardwareDevice>();
218+
device->type = type;
219+
device->vendor_id = vendor_id;
220+
device->device_id = device_id;
221+
device->vendor = std::string(vendor_name);
222+
223+
if (metadata) {
224+
device->metadata = *metadata;
225+
}
226+
227+
*hardware_device = device.release();
228+
return nullptr;
229+
API_IMPL_END
230+
}
231+
232+
ORT_API(void, ReleaseHardwareDevice, _Frees_ptr_opt_ OrtHardwareDevice* device) {
233+
delete device;
234+
}
235+
208236
static constexpr OrtEpApi ort_ep_api = {
209237
// NOTE: ABI compatibility depends on the order within this struct so all additions must be at the end,
210238
// and no functions can be removed (the implementation needs to change to return an error).
@@ -231,6 +259,9 @@ static constexpr OrtEpApi ort_ep_api = {
231259
&OrtExecutionProviderApi::SyncStream_GetSyncId,
232260
&OrtExecutionProviderApi::GetSyncIdForLastWaitOnSyncStream,
233261
// End of Version 23 - DO NOT MODIFY ABOVE
262+
263+
&OrtExecutionProviderApi::CreateHardwareDevice,
264+
&OrtExecutionProviderApi::ReleaseHardwareDevice,
234265
};
235266

236267
// checks that we don't violate the rule that the functions must remain in the slots they were originally assigned

onnxruntime/core/session/plugin_ep/ep_api.h

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,4 +40,11 @@ ORT_API(const OrtSyncStreamImpl*, SyncStream_GetImpl, _In_ const OrtSyncStream*
4040
ORT_API(uint64_t, SyncStream_GetSyncId, _In_ const OrtSyncStream* stream);
4141
ORT_API(uint64_t, GetSyncIdForLastWaitOnSyncStream, _In_ const OrtSyncStream* producer_stream,
4242
_In_ const OrtSyncStream* consumer_stream);
43+
ORT_API_STATUS_IMPL(CreateHardwareDevice, _In_ OrtHardwareDeviceType type,
44+
_In_ uint32_t vendor_id,
45+
_In_ uint32_t device_id,
46+
_In_ const char* vendor_name,
47+
_In_opt_ const OrtKeyValuePairs* metadata,
48+
_Out_ OrtHardwareDevice** hardware_device);
49+
ORT_API(void, ReleaseHardwareDevice, _Frees_ptr_opt_ OrtHardwareDevice* device);
4350
} // namespace OrtExecutionProviderApi

onnxruntime/core/session/plugin_ep/ep_factory_internal.cc

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,8 @@
33

44
#include "core/session/plugin_ep/ep_factory_internal.h"
55

6+
#include <utility>
7+
68
#include "core/framework/error_code_helper.h"
79
#include "core/session/abi_devices.h"
810
#include "core/session/abi_session_options_impl.h"
@@ -13,7 +15,8 @@ namespace onnxruntime {
1315
using Forward = ForwardToFactoryImpl<EpFactoryInternal>;
1416

1517
EpFactoryInternal::EpFactoryInternal(std::unique_ptr<EpFactoryInternalImpl> impl)
16-
: impl_{std::move(impl)} {
18+
: OrtEpFactory{}, // Ensure optional functions are default initialized to nullptr
19+
impl_{std::move(impl)} {
1720
ort_version_supported = ORT_API_VERSION;
1821

1922
OrtEpFactory::GetName = Forward::GetFactoryName;
@@ -29,6 +32,7 @@ EpFactoryInternal::EpFactoryInternal(std::unique_ptr<EpFactoryInternalImpl> impl
2932
OrtEpFactory::CreateDataTransfer = Forward::CreateDataTransfer;
3033
OrtEpFactory::IsStreamAware = Forward::IsStreamAware;
3134
OrtEpFactory::CreateSyncStreamForDevice = Forward::CreateSyncStreamForDevice;
35+
OrtEpFactory::SetEnvironmentOptions = Forward::SetEnvironmentOptions;
3236
}
3337

3438
InternalExecutionProviderFactory::InternalExecutionProviderFactory(EpFactoryInternal& ep_factory,

onnxruntime/core/session/plugin_ep/ep_factory_internal.h

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,10 @@ class EpFactoryInternal : public OrtEpFactory {
8787
return impl_->ValidateCompiledModelCompatibilityInfo(devices, num_devices, compatibility_info, model_compatibility);
8888
}
8989

90+
OrtStatus* SetEnvironmentOptions(_In_ const OrtKeyValuePairs* options) noexcept {
91+
return impl_->SetEnvironmentOptions(options);
92+
}
93+
9094
// Function ORT calls to release an EP instance.
9195
void ReleaseEp(OrtEp* /*ep*/) noexcept {
9296
// we never create an OrtEp so we should never be trying to release one

onnxruntime/core/session/plugin_ep/ep_factory_internal_impl.h

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,11 @@ class EpFactoryInternalImpl {
8383
"CreateSyncStreamForDevice is not implemented for this EP factory.");
8484
}
8585

86+
virtual OrtStatus* SetEnvironmentOptions(const OrtKeyValuePairs* /*options*/) noexcept {
87+
// Default implementation does not handle any options.
88+
return nullptr;
89+
}
90+
8691
// Function ORT calls to release an EP instance.
8792
void ReleaseEp(OrtEp* ep);
8893

onnxruntime/core/session/plugin_ep/forward_to_factory_impl.h

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,11 @@ struct ForwardToFactoryImpl {
8282
return static_cast<TFactory*>(this_ptr)->CreateSyncStreamForDevice(memory_device, stream_options, stream);
8383
}
8484

85+
static OrtStatus* ORT_API_CALL SetEnvironmentOptions(_In_ OrtEpFactory* this_ptr,
86+
_In_ const OrtKeyValuePairs* options) noexcept {
87+
return static_cast<TFactory*>(this_ptr)->SetEnvironmentOptions(options);
88+
}
89+
8590
static void ORT_API_CALL ReleaseEp(OrtEpFactory* this_ptr, OrtEp* ep) noexcept {
8691
static_cast<TFactory*>(this_ptr)->ReleaseEp(ep);
8792
}

0 commit comments

Comments
 (0)