Usage

For Linux 64, Open MPI is built with CUDA awareness but this support is disabled by default. To enable it, please set the environment variable OMPI_MCA_opal_cuda_support=true before launching your MPI processes. Equivalently, you can set the MCA parameter in the command line:

mpiexec --mca opal_cuda_support 1 ...

In addition, the UCX support is also built but disabled by default. To enable it, set the environment variables OMPI_MCA_pml="ucx" OMPI_MCA_osc="ucx" before launching your MPI processes. Equivalently, you can set the MCA parameters in the command line:

mpiexec --mca pml ucx --mca osc ucx ... 

Note that you might also need to set UCX_MEMTYPE_CACHE=n for CUDA awareness via UCX. Please consult the UCX Documentation Page for detailed information.