Skip to content

[P2P] Multi-backend Support and Auto-detection #570

@derekwin

Description

@derekwin
  • Bundle the native shared libraries into the UCCL Python package, eliminating the need for users to manually set library paths (e.g., when using Nix or similar environments).

  • Enable automatic backend selection based on hardware capabilities:

    • Use the RDMA engine on machines with RDMA support,
    • Fall back to EFA backend on AWS instances with EFA NICs,
    • Use NCCL-TCPX on systems without specialized networking hardware.
  • Add pybind11 support for all communication backends (not just the RDMA engine), so they can be used via the uccl.p2p Python API.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions