No GPU-GPU communication #40
Description
A rookie here.. I tried to use JUHPC to install Julia on the Norwegian supercomputer Olivia (https://documentation.sigma2.no/hpc_machines/olivia.html) which is the same architecture as Alps used in some examples here. The installation seems to go fine, and my application (https://clima.github.io/ClimaOceanDocumentation/dev/) seems to run fine, but when I do scaling tests, and use Nsight profiler to profile, it looks like the communication between GPUs goes via the CPU, with the GPU idling a lot. rather than direct GPU-GPU communication. Could it have been that something went wrong in choosing the correct CUDA/MPI versions during installation? Or what else could be the problem?
In the Alps example, there is a file cudaaware.jl mentioned to test CUDA-aware MPI. Is that available somewhere here? Or is there another good test I could do to figure out if CUDA and MPI are working together nicely?
Cheers, Andreas