Hi all,
My notes from the meeting today (more detail and/or explanation available on request, or watch the recording):
Notified-RMA feedback 2024_10_10
Jeff: notification for fetch_and_op and compare_and_op?
Overlapping windows.
Fine-grained completion of fetch-and-op without requirement to progress other things
Target or origin side? Origin s/be request-based?
Joseph: is the extra flexibility of changing the number of notifications valuable enough?
Does it compromise the performance of the implementation?
Add MPI_Win_allocate_with_notifications initially; add others later, if needed and proven to work?
Dmitry - allocation and window creation might be in different levels of the application; needs separate APIs
Jeff: MPI_Win_allocate_shared should only be allowed for getting shared memory, not for RMA operations!
This API exists as a wrapper over POSIX shmem; app should do language atomics, not RMA
Dmitry: making the user implement notifications in shared memory is a bad idea
Efficiency is secondary to portability, correctness, etc.
Jeff: Fortran has no memory model, MPI can help by defining notification memory semantics
Personal feeling - Forum will demand all four for orthogonality
Maybe an advice - one of these is better than the others
Joseph: implement "during constructor" and see if it is faster, then decide
Burden of proof - need to prove a negative; hmm
Jeff: no law preventing pre-allocation of N counters during constructor
If user asks for less, great! Otherwise, allocate more.
Alexander: INFO key for constructor to specify expected max number of counters
Also, window memory on device and notifications on host
Joseph: agree that notifications should be at message boundaries
Maybe we can have both kinds [ED: yes, that's orthogonal]
Jeff: can you use IB "send immediate" for put-notify?
Needs recent implementor experience -- maybe there's a size limit?
Ken - size is not consistent across providers, also not exposed in some low-level APIs
Jeff - if network is in-order then two writes is ordered for free
Ken - if out-of-order network, we'll look at it
Jeff: add request-based notified-RMA, restrict from device-initiated when that happens
Alexander - we proposed what we implemented, other things are possible
Jeff - implementation of request-based not necessary for evaluation of this proposal
Best wishes,
Dan.
From: mpiwg-rma <mpiwg-rma-bounces(a)lists.mpi-forum.org> On Behalf Of Joseph Schuchart via mpiwg-rma
Sent: Tuesday, October 8, 2024 9:34 PM
To: mpiwg-rma <mpiwg-rma(a)lists.mpi-forum.org>
Cc: Joseph Schuchart <schuchart(a)icl.utk.edu>
Subject: [mpiwg-rma] Meeting Oct 10: Notified communication discussion
Dear all, The RMA WG will meet again this Thursday (October 10) at 10am US Central / 11am US Eastern / 5pm Central European time to discuss the notified communication proposal put forward by Intel [1]. Dan will lead the discussion. The connection
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd
Dear all,
The RMA WG will meet again this Thursday (October 10) at 10am US Central
/ 11am US Eastern / 5pm Central European time to discuss the notified
communication proposal put forward by Intel [1]. Dan will lead the
discussion.
The connection information can be found at
https://urldefense.us/v3/__https://github.com/mpiwg-rma/mpi-standard/wiki__…<https://urldefense.us/v3/__https:/github.com/mpiwg-rma/mpi-standard/wiki__;…>. We started using a new
zoom room last time so please make sure you use the link on the github wiki.
Cheers
Joseph
[1] https://urldefense.us/v3/__https://github.com/mpiwg-rma/mpi-standard/pull/1…<https://urldefense.us/v3/__https:/github.com/mpiwg-rma/mpi-standard/pull/11…>
