Hi Alain,
I could not find anything explicitly stating that MPI_Iprobe would
initiate a receive the way MPI_Irecv does.
This is because the probe functions do not initiate a receive the way a receive does; they only probe the matching list and report back to the user but do not change what they find there.
Your modified example is guaranteed to deadlock using any standard-conforming MPI library. The MPI_SSEND must not complete before a matching receive operation has been started and has matched. The matching receive operation (the MPI_IRECV in rank 1) cannot be reached because the MPI_RECV must not complete before a matching send operation has been started. The matching send operation (the MPI_SEND in rank 0) cannot be reached because the MPI_SSEND is blocked. The MPI_IPROBE is not a receive operation and does not cause a match to happen - it is therefore not relevant to the deadlock situation described above.
Your modified example can be fixed by replacing MPI_IPROBE with MPI_MPROBE (note this a blocking function) and replacing MPI_IRECV with MPI_IMRECV, although a nonblocking function immediately followed by a call to MPI_WAIT is better expressed by a blocking function, i.e. MPI_MRECV in this case.
The example must use the blocking MPI_MPROBE rather than the nonblocking MPI_IMPROBE because the match for the MPI_SSEND must occur before the MPI_RECV can complete (because that requires the MPI_SEND to be reached in rank 0, which requires that the MPI_SSEND completes, which requires that it must have been matched in rank 1, which requires that the mprobe must be complete not just started).
Cheers,
Dan.
—
Dr Daniel Holmes PhD
Applications Consultant in HPC Research
d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]>
Phone: +44 (0) 131 651 3465
Mobile: +44 (0) 7940 524 088
Address: Room 2.09, Bayes Centre, 47 Potterrow, Central Area, Edinburgh, EH8 9BT
—
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
—
On 13 Sep 2018, at 15:08, mpi-comments-request(a)lists.mpi-forum.org<mailto:[email protected]> wrote:
----- Original Message -----
Hi,
(text location provided w.r.t MPI 3.1 https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report.pdf)
a) In 3.2.4, page 29, there is a discussion regarding the size of the
buffer and size of the actual message with an "advice to users" section
stating that :
"The MPI_PROBE function described in Section 3.8 can be used to receive
messages of unknown length."
b) There is no such discussion in 3.7.2 so one could assume (maybe too
optimistically) that the same advice applies w.r.t. MPI_Iprobe.
c) In 3.8.1. page 66 line 9 "The MPI implementation of MPI_PROBE and
MPI_IPROBE needs to guarantee progress: [calling those functions, msg
should arrive eventually if send and not intercepted].
d) The "Progress" note in 3.7.4, illustrated by example 3.14, seems to
indicate that a synchronous send can not be blocked if a recv has been
posted regardless of the completion of that receive.
But I could not find anything explicitly stating that MPI_Iprobe would
initiate a receive the way MPI_Irecv does.
That is, a slightly modified 3.14 example:
CALL MPI_COMM_RANK(comm, rank, ierr)
IF (RANK.EQ.0) THEN
CALL MPI_SSEND(a, n, MPI_REAL, 1, 0, comm, ierr)
CALL MPI_SEND(b, 1, MPI_REAL, 1, 1, comm, ierr)
ELSE IF (rank.EQ.1) THEN
CALL MPI_IPROBE(0, 0, comm, flag, status1, ierr)
CALL MPI_RECV(b, 1, MPI_REAL, 0, 1, comm, status2, ierr)
... extract size info from status1 once available ....
CALL MPI_IRECV(a, n, MPI_REAL, 0, 0, comm, r, ierr)
CALL MPI_WAIT(r, status, ierr)
END IF
could deadlock in a correct MPI implementation.
If I am correct in assuming that the MPI standard ask for the MPI_Iprobe
to progress, it is not required to initiate. If true, it means the the
point a) above does not extend to the non blocking communications and
that the MPI does not provide a way to receive messages of unknown size
whithout adding more opportunities for deadlocks.
Am I correct ? and if yes, does that indicate that the discussion and
advice of point a could be adapted and explicitly added for the non
blocking case (point b).
Thanks
Alain
End of mpi-comments Digest, Vol 8, Issue 1
******************************************
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
Gilles,
Not so sure, that argument could be made for MPI_Irecv (used in the
original code), which is not allowed to deadlook. There is a special
send mode called "ready" that explicitly assume that the recv call has
been made, but that's just an optimization.
From 3.4: "Thus, a send in standard mode can be started whether or not
a matching receive has been posted."
I kept the MPI_Ssend because it was in the original example, but the
question still hold if replaced with MPI_Send which leaves more choice
to the implementation.
If MPI_Iprobe initiate a reception (which is not explicitly stated, and
which is basically my question) the first MPI_Send could just wait for
that event and send the data (maybe using the buffered mode).
An argument could be made that since the receiver buffer is not
provided, and the implementation could choose a communication mode
requiring that a user provided buffer to be ready ready. But then the
same argument could be made for MPI_PROBE, and state 3.8 that:
"The MPI_PROBE, MPI_IPROBE, MPI_MPROBE, and MPI_IMPROBE operations allow
incoming messages to be checked for,� without actually receiving them."
and later on (p 65):
"it is not necessary to receive a message immediately after it has been
probed for, and the same message may be probed for several times before
it is received."
So it seems that the implementation is expected to send the size first,
for MPI_[IM]probe to look it up, but that the message itself can be kept
pending and that the matching send can be kept blocked until the message
is actually copied. Put another way, the message size must be made
available to the receiver no matter what communication protocol is chosen.
Cheers
Alain
On 13/09/2018 15:24, gilles--- via mpi-comments wrote:
>
> �Alain,
>
> Regardless of what the standard says or how it is interpreted, your
> example can simply deadlock if rank 1 returns from MPI_Iprobe()
>
> before rank 0 calls MPI_Ssend().
>
> Cheers,
>
> Gilles
>
> ----- Original Message -----
>
> Hi,
>
> (text location provided w.r.t MPI 3.1
> https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report.pdf)
>
> �a) In 3.2.4, page 29, there is a discussion regarding the size of
> the buffer and size of the actual message with an "advice to
> users" section stating that :
> "The MPI_PROBE function described in Section 3.8 can be used to
> receive messages of unknown length."
>
> �b) There is no such discussion in 3.7.2 so one could assume
> (maybe too optimistically) that the same advice applies w.r.t.
> MPI_Iprobe.
>
> �c) In 3.8.1. page 66 line 9 "The MPI implementation of MPI_PROBE
> and MPI_IPROBE needs to guarantee progress: [calling those
> functions, msg should arrive eventually if send and not intercepted].
>
> �d) The "Progress" note in 3.7.4, illustrated by example 3.14,
> seems to indicate that a synchronous send can not be blocked if a
> recv has been posted regardless of the completion of that receive.
>
> But I could not find anything explicitly stating that MPI_Iprobe
> would initiate a receive the way MPI_Irecv does.
>
> That is, a slightly modified 3.14 example:
>
> CALL MPI_COMM_RANK(comm, rank, ierr)
> IF (RANK.EQ.0) THEN
> ��� CALL MPI_SSEND(a, n, MPI_REAL, 1, 0, comm, ierr)
> ��� CALL MPI_SEND(b, 1, MPI_REAL, 1, 1, comm, ierr)
> ELSE IF (rank.EQ.1) THEN
> ��� CALL MPI_IPROBE(0, 0, comm, flag, status1, ierr)
> ��� CALL MPI_RECV(b, 1, MPI_REAL, 0, 1, comm, status2, ierr)
> ��� ... extract size info from status1� once available ....
> ��� CALL MPI_IRECV(a, n, MPI_REAL, 0, 0, comm, r, ierr)
> ��� CALL MPI_WAIT(r, status, ierr)
> END IF
>
> could deadlock in a correct MPI implementation.
>
> If I am correct in assuming that the MPI standard ask for the
> MPI_Iprobe to progress, it is not required to initiate. If true,
> it means the the point a) above does not extend to the non
> blocking communications and that the MPI does not provide a way to
> receive messages of unknown size whithout adding more
> opportunities for deadlocks.
>
> Am I correct ? and if yes, does that indicate that the discussion
> and advice of point a could be adapted and explicitly added for
> the non blocking case (point b).
>
> Thanks
>
> Alain
>
>
>
> _______________________________________________
> mpi-comments mailing list
> mpi-comments(a)lists.mpi-forum.org
> https://lists.mpi-forum.org/mailman/listinfo/mpi-comments
Alain,
Regardless of what the standard says or how it is interpreted, your
example can simply deadlock if rank 1 returns from MPI_Iprobe()
before rank 0 calls MPI_Ssend().
Cheers,
Gilles
----- Original Message -----
Hi,
(text location provided w.r.t MPI 3.1 https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report.pdf)
a) In 3.2.4, page 29, there is a discussion regarding the size of the
buffer and size of the actual message with an "advice to users" section
stating that :
"The MPI_PROBE function described in Section 3.8 can be used to receive
messages of unknown length."
b) There is no such discussion in 3.7.2 so one could assume (maybe too
optimistically) that the same advice applies w.r.t. MPI_Iprobe.
c) In 3.8.1. page 66 line 9 "The MPI implementation of MPI_PROBE and
MPI_IPROBE needs to guarantee progress: [calling those functions, msg
should arrive eventually if send and not intercepted].
d) The "Progress" note in 3.7.4, illustrated by example 3.14, seems to
indicate that a synchronous send can not be blocked if a recv has been
posted regardless of the completion of that receive.
But I could not find anything explicitly stating that MPI_Iprobe would
initiate a receive the way MPI_Irecv does.
That is, a slightly modified 3.14 example:
CALL MPI_COMM_RANK(comm, rank, ierr)
IF (RANK.EQ.0) THEN
CALL MPI_SSEND(a, n, MPI_REAL, 1, 0, comm, ierr)
CALL MPI_SEND(b, 1, MPI_REAL, 1, 1, comm, ierr)
ELSE IF (rank.EQ.1) THEN
CALL MPI_IPROBE(0, 0, comm, flag, status1, ierr)
CALL MPI_RECV(b, 1, MPI_REAL, 0, 1, comm, status2, ierr)
... extract size info from status1 once available ....
CALL MPI_IRECV(a, n, MPI_REAL, 0, 0, comm, r, ierr)
CALL MPI_WAIT(r, status, ierr)
END IF
could deadlock in a correct MPI implementation.
If I am correct in assuming that the MPI standard ask for the MPI_Iprobe
to progress, it is not required to initiate. If true, it means the the
point a) above does not extend to the non blocking communications and
that the MPI does not provide a way to receive messages of unknown size
whithout adding more opportunities for deadlocks.
Am I correct ? and if yes, does that indicate that the discussion and
advice of point a could be adapted and explicitly added for the non
blocking case (point b).
Thanks
Alain
Hi,
(text location provided w.r.t MPI 3.1
https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report.pdf)
a) In 3.2.4, page 29, there is a discussion regarding the size of the
buffer and size of the actual message with an "advice to users" section
stating that :
"The MPI_PROBE function described in Section 3.8 can be used to receive
messages of unknown length."
b) There is no such discussion in 3.7.2 so one could assume (maybe too
optimistically) that the same advice applies w.r.t. MPI_Iprobe.
c) In 3.8.1. page 66 line 9 "The MPI implementation of MPI_PROBE and
MPI_IPROBE needs to guarantee progress: [calling those functions, msg
should arrive eventually if send and not intercepted].
d) The "Progress" note in 3.7.4, illustrated by example 3.14, seems to
indicate that a synchronous send can not be blocked if a recv has been
posted regardless of the completion of that receive.
But I could not find anything explicitly stating that MPI_Iprobe would
initiate a receive the way MPI_Irecv does.
That is, a slightly modified 3.14 example:
CALL MPI_COMM_RANK(comm, rank, ierr)
IF (RANK.EQ.0) THEN
CALL MPI_SSEND(a, n, MPI_REAL, 1, 0, comm, ierr)
CALL MPI_SEND(b, 1, MPI_REAL, 1, 1, comm, ierr)
ELSE IF (rank.EQ.1) THEN
CALL MPI_IPROBE(0, 0, comm, flag, status1, ierr)
CALL MPI_RECV(b, 1, MPI_REAL, 0, 1, comm, status2, ierr)
... extract size info from status1 once available ....
CALL MPI_IRECV(a, n, MPI_REAL, 0, 0, comm, r, ierr)
CALL MPI_WAIT(r, status, ierr)
END IF
could deadlock in a correct MPI implementation.
If I am correct in assuming that the MPI standard ask for the MPI_Iprobe
to progress, it is not required to initiate. If true, it means the the
point a) above does not extend to the non blocking communications and
that the MPI does not provide a way to receive messages of unknown size
whithout adding more opportunities for deadlocks.
Am I correct ? and if yes, does that indicate that the discussion and
advice of point a could be adapted and explicitly added for the non
blocking case (point b).
Thanks
Alain
