-
Notifications
You must be signed in to change notification settings - Fork 231
Description
Refactoring pinned host memory resources in RMM
RMM started with a pinned_memory_resource in mr/host/. Later, a pinned_host_memory_resource was added in mr/ in #1392 / #618. The new pinned_host_memory_resource is designed around the CCCL MR concepts that we want to adopt throughout RMM (xref: #2011). I propose we adopt pinned_host_memory_resource and drop rmm::mr::pinned_memory_resource (which will also allow us to drop rmm::mr::host_memory_resource).
Below, we compare rmm::mr::pinned_memory_resource and rmm::mr::pinned_host_memory_resource and provide guidance for migrating to the newer API.
Summary
pinned_memory_resourceis a polymorphic RMM host MR deriving fromrmm::mr::host_memory_resourceand usescudaMallocHost/cudaFreeHost.pinned_host_memory_resourceis a stateless CCCL-style resource that exposes static allocate/deallocate functions and usescudaHostAlloc/cudaFreeHost(default flags). It explicitly modelscuda::mr::{memory_resource, device_memory_resource}concepts and declares both host/device accessibility properties.- Both allocate pinned/page-locked host memory; there are subtle (but likely inconsequential) differences in API surface, inheritance/equality semantics, and backend API choice.
Detailed comparison
-
API paradigm:
pinned_memory_resource(PMR): object-oriented, polymorphic basehost_memory_resourcewith virtualdo_allocate/do_deallocate. Integrates with existing RMM host MR design and any code expecting ahost_memory_resource*.pinned_host_memory_resource(PHMR): stateless, concept-based CCCL MR with static methods. No base-class polymorphism; equality is type-based (all instances equivalent).
-
Inheritance and equality
- PMR inherits
host_memory_resource; equality is identity-based (two different instances are not equal) viahost_memory_resourcedefault. - PHMR is standalone; compares all instances equal (stateless type equality).
- PMR inherits
-
Backend CUDA API: slightly better for PHMR
- PMR:
cudaMallocHost/cudaFreeHost. - PHMR:
cudaHostAlloc(cudaHostAllocDefault)/cudaFreeHost. - Practical difference is negligible when using default flags;
cudaHostAllocsupports flags if we later extend. PHMR design withcudaHostAllocis more desirable because of its flexibility.
- PMR:
-
Stream/async semantics: slightly better for PHMR
- Both expose overloads that accept a stream but ignore it; pinned host alloc/free are synchronous. Names may be
allocate_async/deallocate_asyncfor interface compatibility but behavior is synchronous. - We will be refactoring everything in RMM towards new names for all these methods anyway as we progress on CCCL MR adoption ([FEA] Support memory resources from CCCL 3.1.0+ #2011).
- Both expose overloads that accept a stream but ignore it; pinned host alloc/free are synchronous. Names may be
-
Accessibility properties: the same for both
- PMR:
device_accessibleviaget_property(pinned_memory_resource, cuda::mr::device_accessible)and host accessible viahost_memory_resource. - PHMR: explicitly declares both
device_accessibleandhost_accessible.
- PMR:
-
Header locations
- PMR:
rmm/mr/host/pinned_memory_resource.hpp(host MR group). - PHMR:
rmm/mr/pinned_host_memory_resource.hpp(general MR group).
- PMR:
Migration plan
I looked at usage of pinned_memory_resource across RAPIDS and I did not see significant usage of host_memory_resource polymorphism. That means we can do the following:
- Update RMM docs, examples, benchmarks to use PHMR (eliminate internal usage of PMR)
- Directly replace PMR with PHMR in downstream RAPIDS repositories
- Deprecate PMR and
host_memory_resource(25.12) - Remove PMR and
host_memory_resource(26.02) - Simplify the structure of RMM MRs -- we no longer need to distinguish "host" and "device" MRs when everything adheres to the CCCL MR design. That means we can move all the headers to
rmm/mr/*instead ofrmm/mr/device/*(with a deprecation period)
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Status