As you all know, we’ve been working on an MPI stream triggering API that relies on persistent operations and proposes relatively modest changes to the MPI standard to enable high-performance one-sided data movement using persistent operations. We have completed an open source, high-performance CPU-free implementation of the point-to-point portion of this API for HPE Slingshot 11 systems that leverages the triggering operations includes in OFI and the GPU interfaces to them in the HPE CXI libfabric provider.
The arxiv draft paper below describes the API, its implementation on Slingshot 11, and its evaluation up to 8,192 GPUs of ORNL Frontier and 4,06 GPUs of LLNL Tuolumne. The submission of this paper for peer reviewed publication, with some additional planned work no included in this paper, is imminent. This implementation, which is certainly still research-ware, is available as part of MPI Advance, and both our benchmarks, run scripts, and raw data are also linked from the paper.
<https://urldefense.us/v3/__https://arxiv.org/abs/2602.15356__;!!G_uCfscf7eW… >
[arxiv-logo-fb.png]
Co-Design and Evaluation of a CPU-Free MPI GPU Communication Abstraction and Implementation<https://urldefense.us/v3/__https://arxiv.org/abs/2602.15356__;!!G_uCfscf7eW… >
arxiv.org<https://urldefense.us/v3/__https://arxiv.org/abs/2602.15356__;!!G_uCfscf7eW… >
Comments and suggestions welcome; we would certainly be interested in discussing the potential options and pitfalls for its implementation on other networking systems or its integration into full MPI implementations.
Finally, we have scheduled working meetings begnning next week to start drafting potential language for this API for inclusion in the MPI standard.
Thanks,
Patrick
