Skip to content

Benchmark NVSHMEM 3.5.0 with EFA optimal patch #518

@HermesCui

Description

@HermesCui

Hi, I ran the UCCl baseline benchmark using the latest NVSHMEM 3.5.0 version, which includes the EFA optimization patch. Here are the results from the baseline benchmark.
EP=16 DP=16

E	E/tok	tok	dim	CPU+Torch_total	CPU+Torch_lat	CPU+Torch_bw	CPU+Torch_comm	CPU_pack	CPU+NVSHMEM_total	CPU+NVSHMEM_lat	CPU+NVSHMEM_bw	CPU+NVSHMEM_comm	Dense_lat	Dense_bw	Sparse_bytes	Dense_bytes	CUDA+Torch_total	CUDA+Torch_lat	CUDA+Torch_bw	CUDA+Torch_comm	CUDA_pack	CUDA+NVSHMEM_total	CUDA+NVSHMEM_lat	CUDA+NVSHMEM_bw	CUDA+NVSHMEM_comm	Speedup_Torch	Speedup_NVSHMEM
256	8	128	7168	105417.6μs	30053.0μs	0.25GB/s	1659.9μs	75364.6μs	131503.7μs	56139.1μs	0.15GB/s	27780.5μs	1405.4μs	5.23GB/s	7,348,224	7,348,224	3956.4μs	1405.4μs	5.23GB/s	405.2μs	2551.0μs	28608.4μs	26057.4μs	0.82GB/s	25058.1μs	26.65x	4.60x
256	8	256	7168	199167.0μs	55347.9μs	0.27GB/s	525.5μs	143819.0μs	208128.5μs	64309.5μs	0.23GB/s	9547.2μs	2361.1μs	6.22GB/s	14,696,448	14,696,448	4421.7μs	2361.1μs	6.22GB/s	508.4μs	2060.5μs	5422.3μs	3361.8μs	4.37GB/s	1538.0μs	45.04x	38.38x
256	8	512	7168	396906.4μs	108975.4μs	0.27GB/s	797.7μs	287931.0μs	411028.2μs	123097.2μs	0.24GB/s	15033.0μs	4349.6μs	6.76GB/s	29,392,896	29,392,896	6412.1μs	4349.6μs	6.76GB/s	802.5μs	2062.5μs	8781.8μs	6719.3μs	4.37GB/s	3190.0μs	61.90x	46.80x
256	8	768	7168	592873.8μs	162597.7μs	0.27GB/s	1262.0μs	430276.1μs	615808.6μs	185532.6μs	0.24GB/s	24424.2μs	6164.4μs	7.15GB/s	44,089,344	44,089,344	8314.2μs	6164.4μs	7.15GB/s	1259.3μs	2149.8μs	11410.4μs	9260.6μs	4.76GB/s	4383.2μs	71.31x	53.97x
256	8	1024	7168	790688.1μs	216602.8μs	0.27GB/s	1530.3μs	574085.3μs	817837.3μs	243752.0μs	0.24GB/s	29121.8μs	8083.8μs	7.27GB/s	58,785,792	58,785,792	10276.3μs	8083.8μs	7.27GB/s	1532.1μs	2192.4μs	14813.5μs	12621.0μs	4.66GB/s	6088.6μs	76.94x	55.21x
256	8	2048	7168	1574841.9μs	429761.8μs	0.27GB/s	3006.6μs	1145080.0μs	1631731.0μs	486651.0μs	0.24GB/s	60564.4μs	15267.4μs	7.70GB/s	117,571,584	117,571,584	17592.7μs	15267.4μs	7.70GB/s	2899.0μs	2325.3μs	33208.4μs	30883.1μs	3.81GB/s	18545.3μs	89.52x	49.14x
256	8	4096	7168	3162618.5μs	863716.8μs	0.27GB/s	5975.8μs	2298901.8μs	3265733.0μs	966831.4μs	0.24GB/s	108283.3μs	30284.3μs	7.76GB/s	235,143,168	235,143,168	33907.1μs	30284.3μs	7.76GB/s	5710.9μs	3622.8μs	63082.3μs	59459.5μs	3.96GB/s	34931.3μs	93.27x	51.77x

Image

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions