Add file descriptor support for weightless model loading #32572
Conversation
github-actions
bot
added
category: inference
|
NPUW part looks good |
|
build_jenkins |
| * This is useful for scenarios where file access needs to be controlled externally. | ||
| * @ingroup ov_runtime_cpp_prop_api | ||
| */ | ||
| using FdGetterType = std::function<int(const std::string&)>; |
There was a problem hiding this comment.
We should not add as public property as this some internal NPU handling
There was a problem hiding this comment.
It could be dev_api nested trait, if needed elsewhere.
There was a problem hiding this comment.
I have added a new commit(ffb6519) which moves the property under npuw_private_properties.hpp.
But, one challenge here is that the type definition FdGetterType won't be visible to the caller.
I verified that it works when the type is defined and used by the caller :
using FdGetterType = std::function<int(const std::string&)>;
akodanka
force-pushed
the
fd_getter
branch
2 times, most recently
from
7d8abfb to
414d179
Compare
github-actions
bot
removed
category: inference
| * Function that takes a file path string and returns a file descriptor as int. | ||
| * This is useful for scenarios where file access needs to be controlled externally. | ||
| */ | ||
| using FdGetterType = std::function<int(const std::string&)>; |
There was a problem hiding this comment.
Please move it below under npuw namespace
|
|
||
| namespace npuw { | ||
| /** | ||
| * @brief This key identifies callback function to get file descriptor for a given file path. |
There was a problem hiding this comment.
Please align the comment with other npuw properties' comments below
There was a problem hiding this comment.
Also you can mention that this property is used during weightless import only
github-actions
bot
added
category: inference
Changes: - Add ov::FdGetterType and ov::hint::fd_getter property - Extend load_mmap_object() with fd-based overload (Linux only) - Integrate fd_getter through NPUW deserialization flow - Windows implementation throws unsupported exception Signed-off-by: Anoob Anto Kodankandath <[email protected]>
Signed-off-by: Anoob Anto Kodankandath <[email protected]>
8 participants
Changes:
Details:
Tickets: