February 2014 - mpiwg-rma - mailman3.cels.anl.gov

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Balaji, Pavan 25 Feb '14

25 Feb '14

Then the following sentence in the standard is weird: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier” Why this special treatment for NOPRECEDE as no WIN_FENCE has barrier semantics? — Pavan On Feb 25, 2014, at 10:54 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > Agreed that the handshake can't be done in the RMA operation. > > If you choose to implement fence eagerly, i.e., perform RMA ops as they are called, the first fence will have to be a barrier. > > But if you choose to implement the deferred fence, the first fence and all RMA ops can be deferred until the fence that completes the epoch. > > > On Feb 25, 2014, at 10:46 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> > wrote: > >> I agree with this. If the handshake is done in the first RMA operation after the fence, then that operation becomes blocking. In Standard P418 it says all communication calls are non-blocking. >> >> Xin >> ________________________________________ >> From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] >> Sent: Tuesday, February 25, 2014 9:38 PM >> To: MPI WG Remote Memory Access working group >> Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic >> >> In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. >> >> — Pavan >> >> On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >> >>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>> >>> It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." >>> >>> Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. >>> >>> Rajeev >>> >>> >>> On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: >>> >>>> Hi all, >>>> >>>> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >>>> >>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>> >>>> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >>>> >>>> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >>>> >>>> Thanks, >>>> >>>> Xin >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Rajeev Thakur 25 Feb '14

25 Feb '14

Then that implementation chooses to do a barrier. Another implementation that defers will choose not to do a barrier. So the standard says the user cannot assume a barrier. On Feb 25, 2014, at 10:55 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: > > What if one has an implementation where not all operations are deferred to the end, e.g., MPICH (in the shared memory case). > > — Pavan > > On Feb 25, 2014, at 10:49 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > >> Not sure why the Get will give incorrect results. In a deferred implementation, the Get can be implemented in the last fence. >> >> On Feb 25, 2014, at 10:30 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: >> >>> >>> Whoops, sorry, I meant to add a LOCK/UNLOCK in there. Here’s the updated example: >>> >>> Fence >>> if (rank == 0) { LOCK; PUT(to rank 1); UNLOCK; } >>> Fence >>> Get(from rank 1) >>> Fence >>> >>> — Pavan >>> >>> On Feb 25, 2014, at 10:28 PM, Zhao, Xin <xinzhao3(a)illinois.edu> wrote: >>> >>>> Hi Pavan, >>>> >>>> Is Get posted on rank 0 or rank 1? Also Flush is only used for passive target. >>>> >>>> Xin >>>> >>>> ________________________________________ >>>> From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] >>>> Sent: Tuesday, February 25, 2014 10:22 PM >>>> To: MPI WG Remote Memory Access working group >>>> Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic >>>> >>>> Your example is too simplistic. Here’s another example — >>>> >>>> Fence >>>> if (rank == 0) { PUT(to rank 1); Flush } >>>> Fence >>>> Get >>>> Fence >>>> >>>> Without barrier semantics, the above program will lead to incorrect results if GET is expecting to see the value deposited by the PUT. Are you saying this is an incorrect program? I’m not sure I agree with that. >>>> >>>> The standard states that FENCE does not have barrier semantics when NOPRECEDE is given. I think the intention here was to say that it does have barrier semantics when that assertion is not given. >>>> >>>> — Pavan >>>> >>>> On Feb 25, 2014, at 10:10 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>>> >>>>> The second sentence refers to case where there are a sequence of fences >>>>> >>>>> Fence >>>>> Fence >>>>> Put >>>>> Fence >>>>> >>>>> The second fence above need not be a barrier particularly if it had a NOPRECEDE. >>>>> >>>>> >>>>> Note that in a fence-put-fence you are allowed to do a barrier in the first fence, but you are not required to. You can do a deferred mode where the first fence is a no-op, all RMA ops are queued, and everything is done in the second fence and determining that all processes have reached there. >>>>> >>>>> Rajeev >>>>> >>>>> >>>>> >>>>> On Feb 25, 2014, at 9:48 PM, "Balaji, Pavan" <balaji(a)anl.gov> >>>>> wrote: >>>>> >>>>>> >>>>>> Besides, the second sentence says, if MPI_MODE_NOPRECEDE is given, it’ll not act as a barrier. >>>>>> >>>>>> What’s the necessity for that if WIN_FENCE does not act as a barrier even when MPI_MODE_NOPRECEDE is not given. >>>>>> >>>>>> — Pavan >>>>>> >>>>>> On Feb 25, 2014, at 9:38 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>>> >>>>>>> >>>>>>> In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. >>>>>>> >>>>>>> — Pavan >>>>>>> >>>>>>> On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>>>>>> >>>>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>>>> >>>>>>>> It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." >>>>>>>> >>>>>>>> Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. >>>>>>>> >>>>>>>> Rajeev >>>>>>>> >>>>>>>> >>>>>>>> On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >>>>>>>>> >>>>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>>>>> >>>>>>>>> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >>>>>>>>> >>>>>>>>> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Xin >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> mpiwg-rma mailing list >>>>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> mpiwg-rma mailing list >>>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpiwg-rma mailing list >>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Balaji, Pavan 25 Feb '14

25 Feb '14

What if one has an implementation where not all operations are deferred to the end, e.g., MPICH (in the shared memory case). — Pavan On Feb 25, 2014, at 10:49 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > Not sure why the Get will give incorrect results. In a deferred implementation, the Get can be implemented in the last fence. > > On Feb 25, 2014, at 10:30 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: > >> >> Whoops, sorry, I meant to add a LOCK/UNLOCK in there. Here’s the updated example: >> >> Fence >> if (rank == 0) { LOCK; PUT(to rank 1); UNLOCK; } >> Fence >> Get(from rank 1) >> Fence >> >> — Pavan >> >> On Feb 25, 2014, at 10:28 PM, Zhao, Xin <xinzhao3(a)illinois.edu> wrote: >> >>> Hi Pavan, >>> >>> Is Get posted on rank 0 or rank 1? Also Flush is only used for passive target. >>> >>> Xin >>> >>> ________________________________________ >>> From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] >>> Sent: Tuesday, February 25, 2014 10:22 PM >>> To: MPI WG Remote Memory Access working group >>> Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic >>> >>> Your example is too simplistic. Here’s another example — >>> >>> Fence >>> if (rank == 0) { PUT(to rank 1); Flush } >>> Fence >>> Get >>> Fence >>> >>> Without barrier semantics, the above program will lead to incorrect results if GET is expecting to see the value deposited by the PUT. Are you saying this is an incorrect program? I’m not sure I agree with that. >>> >>> The standard states that FENCE does not have barrier semantics when NOPRECEDE is given. I think the intention here was to say that it does have barrier semantics when that assertion is not given. >>> >>> — Pavan >>> >>> On Feb 25, 2014, at 10:10 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>> >>>> The second sentence refers to case where there are a sequence of fences >>>> >>>> Fence >>>> Fence >>>> Put >>>> Fence >>>> >>>> The second fence above need not be a barrier particularly if it had a NOPRECEDE. >>>> >>>> >>>> Note that in a fence-put-fence you are allowed to do a barrier in the first fence, but you are not required to. You can do a deferred mode where the first fence is a no-op, all RMA ops are queued, and everything is done in the second fence and determining that all processes have reached there. >>>> >>>> Rajeev >>>> >>>> >>>> >>>> On Feb 25, 2014, at 9:48 PM, "Balaji, Pavan" <balaji(a)anl.gov> >>>> wrote: >>>> >>>>> >>>>> Besides, the second sentence says, if MPI_MODE_NOPRECEDE is given, it’ll not act as a barrier. >>>>> >>>>> What’s the necessity for that if WIN_FENCE does not act as a barrier even when MPI_MODE_NOPRECEDE is not given. >>>>> >>>>> — Pavan >>>>> >>>>> On Feb 25, 2014, at 9:38 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>>> >>>>>> >>>>>> In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. >>>>>> >>>>>> — Pavan >>>>>> >>>>>> On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>>>>> >>>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>>> >>>>>>> It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." >>>>>>> >>>>>>> Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. >>>>>>> >>>>>>> Rajeev >>>>>>> >>>>>>> >>>>>>> On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >>>>>>>> >>>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>>>> >>>>>>>> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >>>>>>>> >>>>>>>> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Xin >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> mpiwg-rma mailing list >>>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpiwg-rma mailing list >>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Rajeev Thakur 25 Feb '14

25 Feb '14

Agreed that the handshake can't be done in the RMA operation. If you choose to implement fence eagerly, i.e., perform RMA ops as they are called, the first fence will have to be a barrier. But if you choose to implement the deferred fence, the first fence and all RMA ops can be deferred until the fence that completes the epoch. On Feb 25, 2014, at 10:46 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: > I agree with this. If the handshake is done in the first RMA operation after the fence, then that operation becomes blocking. In Standard P418 it says all communication calls are non-blocking. > > Xin > ________________________________________ > From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] > Sent: Tuesday, February 25, 2014 9:38 PM > To: MPI WG Remote Memory Access working group > Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic > > In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. > > — Pavan > > On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > >>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >> >> It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." >> >> Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. >> >> Rajeev >> >> >> On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: >> >>> Hi all, >>> >>> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >>> >>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>> >>> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >>> >>> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >>> >>> Thanks, >>> >>> Xin >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Rajeev Thakur 25 Feb '14

25 Feb '14

Not sure why the Get will give incorrect results. In a deferred implementation, the Get can be implemented in the last fence. On Feb 25, 2014, at 10:30 PM, "Balaji, Pavan" <balaji(a)anl.gov> wrote: > > Whoops, sorry, I meant to add a LOCK/UNLOCK in there. Here’s the updated example: > > Fence > if (rank == 0) { LOCK; PUT(to rank 1); UNLOCK; } > Fence > Get(from rank 1) > Fence > > — Pavan > > On Feb 25, 2014, at 10:28 PM, Zhao, Xin <xinzhao3(a)illinois.edu> wrote: > >> Hi Pavan, >> >> Is Get posted on rank 0 or rank 1? Also Flush is only used for passive target. >> >> Xin >> >> ________________________________________ >> From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] >> Sent: Tuesday, February 25, 2014 10:22 PM >> To: MPI WG Remote Memory Access working group >> Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic >> >> Your example is too simplistic. Here’s another example — >> >> Fence >> if (rank == 0) { PUT(to rank 1); Flush } >> Fence >> Get >> Fence >> >> Without barrier semantics, the above program will lead to incorrect results if GET is expecting to see the value deposited by the PUT. Are you saying this is an incorrect program? I’m not sure I agree with that. >> >> The standard states that FENCE does not have barrier semantics when NOPRECEDE is given. I think the intention here was to say that it does have barrier semantics when that assertion is not given. >> >> — Pavan >> >> On Feb 25, 2014, at 10:10 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >> >>> The second sentence refers to case where there are a sequence of fences >>> >>> Fence >>> Fence >>> Put >>> Fence >>> >>> The second fence above need not be a barrier particularly if it had a NOPRECEDE. >>> >>> >>> Note that in a fence-put-fence you are allowed to do a barrier in the first fence, but you are not required to. You can do a deferred mode where the first fence is a no-op, all RMA ops are queued, and everything is done in the second fence and determining that all processes have reached there. >>> >>> Rajeev >>> >>> >>> >>> On Feb 25, 2014, at 9:48 PM, "Balaji, Pavan" <balaji(a)anl.gov> >>> wrote: >>> >>>> >>>> Besides, the second sentence says, if MPI_MODE_NOPRECEDE is given, it’ll not act as a barrier. >>>> >>>> What’s the necessity for that if WIN_FENCE does not act as a barrier even when MPI_MODE_NOPRECEDE is not given. >>>> >>>> — Pavan >>>> >>>> On Feb 25, 2014, at 9:38 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>> >>>>> >>>>> In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. >>>>> >>>>> — Pavan >>>>> >>>>> On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>>>> >>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>> >>>>>> It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." >>>>>> >>>>>> Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. >>>>>> >>>>>> Rajeev >>>>>> >>>>>> >>>>>> On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >>>>>>> >>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>>> >>>>>>> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >>>>>>> >>>>>>> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Xin >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpiwg-rma mailing list >>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Zhao, Xin 25 Feb '14

25 Feb '14

I agree with this. If the handshake is done in the first RMA operation after the fence, then that operation becomes blocking. In Standard P418 it says all communication calls are non-blocking. Xin ________________________________________ From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] Sent: Tuesday, February 25, 2014 9:38 PM To: MPI WG Remote Memory Access working group Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. — Pavan On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. > > It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." > > Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. > > Rajeev > > > On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: > >> Hi all, >> >> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >> >> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >> >> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >> >> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >> >> Thanks, >> >> Xin >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma _______________________________________________ mpiwg-rma mailing list mpiwg-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Zhao, Xin 25 Feb '14

25 Feb '14

I mean, why not the example to be just like: Fence if (rank == 0) {PUT(to rank 1);} Fence Get (from rank 1); Fence Xin ________________________________________ From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] Sent: Tuesday, February 25, 2014 10:38 PM To: MPI WG Remote Memory Access working group Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic What do you mean? Passive target epoch. — Pavan On Feb 25, 2014, at 10:36 PM, Zhao, Xin <xinzhao3(a)illinois.edu> wrote: > What's the purpose of LOCK/UNLOCK? > ________________________________________ > From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] > Sent: Tuesday, February 25, 2014 10:30 PM > To: MPI WG Remote Memory Access working group > Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic > > Whoops, sorry, I meant to add a LOCK/UNLOCK in there. Here’s the updated example: > > Fence > if (rank == 0) { LOCK; PUT(to rank 1); UNLOCK; } > Fence > Get(from rank 1) > Fence > > — Pavan > > On Feb 25, 2014, at 10:28 PM, Zhao, Xin <xinzhao3(a)illinois.edu> wrote: > >> Hi Pavan, >> >> Is Get posted on rank 0 or rank 1? Also Flush is only used for passive target. >> >> Xin >> >> ________________________________________ >> From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] >> Sent: Tuesday, February 25, 2014 10:22 PM >> To: MPI WG Remote Memory Access working group >> Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic >> >> Your example is too simplistic. Here’s another example — >> >> Fence >> if (rank == 0) { PUT(to rank 1); Flush } >> Fence >> Get >> Fence >> >> Without barrier semantics, the above program will lead to incorrect results if GET is expecting to see the value deposited by the PUT. Are you saying this is an incorrect program? I’m not sure I agree with that. >> >> The standard states that FENCE does not have barrier semantics when NOPRECEDE is given. I think the intention here was to say that it does have barrier semantics when that assertion is not given. >> >> — Pavan >> >> On Feb 25, 2014, at 10:10 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >> >>> The second sentence refers to case where there are a sequence of fences >>> >>> Fence >>> Fence >>> Put >>> Fence >>> >>> The second fence above need not be a barrier particularly if it had a NOPRECEDE. >>> >>> >>> Note that in a fence-put-fence you are allowed to do a barrier in the first fence, but you are not required to. You can do a deferred mode where the first fence is a no-op, all RMA ops are queued, and everything is done in the second fence and determining that all processes have reached there. >>> >>> Rajeev >>> >>> >>> >>> On Feb 25, 2014, at 9:48 PM, "Balaji, Pavan" <balaji(a)anl.gov> >>> wrote: >>> >>>> >>>> Besides, the second sentence says, if MPI_MODE_NOPRECEDE is given, it’ll not act as a barrier. >>>> >>>> What’s the necessity for that if WIN_FENCE does not act as a barrier even when MPI_MODE_NOPRECEDE is not given. >>>> >>>> — Pavan >>>> >>>> On Feb 25, 2014, at 9:38 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>> >>>>> >>>>> In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. >>>>> >>>>> — Pavan >>>>> >>>>> On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>>>> >>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>> >>>>>> It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." >>>>>> >>>>>> Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. >>>>>> >>>>>> Rajeev >>>>>> >>>>>> >>>>>> On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >>>>>>> >>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>>> >>>>>>> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >>>>>>> >>>>>>> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Xin >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpiwg-rma mailing list >>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma _______________________________________________ mpiwg-rma mailing list mpiwg-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Balaji, Pavan 25 Feb '14

25 Feb '14

What do you mean? Passive target epoch. — Pavan On Feb 25, 2014, at 10:36 PM, Zhao, Xin <xinzhao3(a)illinois.edu> wrote: > What's the purpose of LOCK/UNLOCK? > ________________________________________ > From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] > Sent: Tuesday, February 25, 2014 10:30 PM > To: MPI WG Remote Memory Access working group > Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic > > Whoops, sorry, I meant to add a LOCK/UNLOCK in there. Here’s the updated example: > > Fence > if (rank == 0) { LOCK; PUT(to rank 1); UNLOCK; } > Fence > Get(from rank 1) > Fence > > — Pavan > > On Feb 25, 2014, at 10:28 PM, Zhao, Xin <xinzhao3(a)illinois.edu> wrote: > >> Hi Pavan, >> >> Is Get posted on rank 0 or rank 1? Also Flush is only used for passive target. >> >> Xin >> >> ________________________________________ >> From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] >> Sent: Tuesday, February 25, 2014 10:22 PM >> To: MPI WG Remote Memory Access working group >> Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic >> >> Your example is too simplistic. Here’s another example — >> >> Fence >> if (rank == 0) { PUT(to rank 1); Flush } >> Fence >> Get >> Fence >> >> Without barrier semantics, the above program will lead to incorrect results if GET is expecting to see the value deposited by the PUT. Are you saying this is an incorrect program? I’m not sure I agree with that. >> >> The standard states that FENCE does not have barrier semantics when NOPRECEDE is given. I think the intention here was to say that it does have barrier semantics when that assertion is not given. >> >> — Pavan >> >> On Feb 25, 2014, at 10:10 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >> >>> The second sentence refers to case where there are a sequence of fences >>> >>> Fence >>> Fence >>> Put >>> Fence >>> >>> The second fence above need not be a barrier particularly if it had a NOPRECEDE. >>> >>> >>> Note that in a fence-put-fence you are allowed to do a barrier in the first fence, but you are not required to. You can do a deferred mode where the first fence is a no-op, all RMA ops are queued, and everything is done in the second fence and determining that all processes have reached there. >>> >>> Rajeev >>> >>> >>> >>> On Feb 25, 2014, at 9:48 PM, "Balaji, Pavan" <balaji(a)anl.gov> >>> wrote: >>> >>>> >>>> Besides, the second sentence says, if MPI_MODE_NOPRECEDE is given, it’ll not act as a barrier. >>>> >>>> What’s the necessity for that if WIN_FENCE does not act as a barrier even when MPI_MODE_NOPRECEDE is not given. >>>> >>>> — Pavan >>>> >>>> On Feb 25, 2014, at 9:38 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>>> >>>>> >>>>> In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. >>>>> >>>>> — Pavan >>>>> >>>>> On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>>>> >>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>> >>>>>> It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." >>>>>> >>>>>> Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. >>>>>> >>>>>> Rajeev >>>>>> >>>>>> >>>>>> On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >>>>>>> >>>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>>> >>>>>>> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >>>>>>> >>>>>>> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Xin >>>>>>> >>>>>>> _______________________________________________ >>>>>>> mpiwg-rma mailing list >>>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Zhao, Xin 25 Feb '14

25 Feb '14

What's the purpose of LOCK/UNLOCK? ________________________________________ From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] Sent: Tuesday, February 25, 2014 10:30 PM To: MPI WG Remote Memory Access working group Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic Whoops, sorry, I meant to add a LOCK/UNLOCK in there. Here’s the updated example: Fence if (rank == 0) { LOCK; PUT(to rank 1); UNLOCK; } Fence Get(from rank 1) Fence — Pavan On Feb 25, 2014, at 10:28 PM, Zhao, Xin <xinzhao3(a)illinois.edu> wrote: > Hi Pavan, > > Is Get posted on rank 0 or rank 1? Also Flush is only used for passive target. > > Xin > > ________________________________________ > From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] > Sent: Tuesday, February 25, 2014 10:22 PM > To: MPI WG Remote Memory Access working group > Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic > > Your example is too simplistic. Here’s another example — > > Fence > if (rank == 0) { PUT(to rank 1); Flush } > Fence > Get > Fence > > Without barrier semantics, the above program will lead to incorrect results if GET is expecting to see the value deposited by the PUT. Are you saying this is an incorrect program? I’m not sure I agree with that. > > The standard states that FENCE does not have barrier semantics when NOPRECEDE is given. I think the intention here was to say that it does have barrier semantics when that assertion is not given. > > — Pavan > > On Feb 25, 2014, at 10:10 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > >> The second sentence refers to case where there are a sequence of fences >> >> Fence >> Fence >> Put >> Fence >> >> The second fence above need not be a barrier particularly if it had a NOPRECEDE. >> >> >> Note that in a fence-put-fence you are allowed to do a barrier in the first fence, but you are not required to. You can do a deferred mode where the first fence is a no-op, all RMA ops are queued, and everything is done in the second fence and determining that all processes have reached there. >> >> Rajeev >> >> >> >> On Feb 25, 2014, at 9:48 PM, "Balaji, Pavan" <balaji(a)anl.gov> >> wrote: >> >>> >>> Besides, the second sentence says, if MPI_MODE_NOPRECEDE is given, it’ll not act as a barrier. >>> >>> What’s the necessity for that if WIN_FENCE does not act as a barrier even when MPI_MODE_NOPRECEDE is not given. >>> >>> — Pavan >>> >>> On Feb 25, 2014, at 9:38 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>> >>>> >>>> In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. >>>> >>>> — Pavan >>>> >>>> On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>>> >>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>> >>>>> It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." >>>>> >>>>> Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. >>>>> >>>>> Rajeev >>>>> >>>>> >>>>> On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >>>>>> >>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>> >>>>>> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >>>>>> >>>>>> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Xin >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma _______________________________________________ mpiwg-rma mailing list mpiwg-rma(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic
by Balaji, Pavan 25 Feb '14

25 Feb '14

Whoops, sorry, I meant to add a LOCK/UNLOCK in there. Here’s the updated example: Fence if (rank == 0) { LOCK; PUT(to rank 1); UNLOCK; } Fence Get(from rank 1) Fence — Pavan On Feb 25, 2014, at 10:28 PM, Zhao, Xin <xinzhao3(a)illinois.edu> wrote: > Hi Pavan, > > Is Get posted on rank 0 or rank 1? Also Flush is only used for passive target. > > Xin > > ________________________________________ > From: mpiwg-rma [mpiwg-rma-bounces(a)lists.mpi-forum.org] on behalf of Balaji, Pavan [balaji(a)anl.gov] > Sent: Tuesday, February 25, 2014 10:22 PM > To: MPI WG Remote Memory Access working group > Subject: Re: [mpiwg-rma] Inconsistency of MPI_WIN_FENCE semantic > > Your example is too simplistic. Here’s another example — > > Fence > if (rank == 0) { PUT(to rank 1); Flush } > Fence > Get > Fence > > Without barrier semantics, the above program will lead to incorrect results if GET is expecting to see the value deposited by the PUT. Are you saying this is an incorrect program? I’m not sure I agree with that. > > The standard states that FENCE does not have barrier semantics when NOPRECEDE is given. I think the intention here was to say that it does have barrier semantics when that assertion is not given. > > — Pavan > > On Feb 25, 2014, at 10:10 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: > >> The second sentence refers to case where there are a sequence of fences >> >> Fence >> Fence >> Put >> Fence >> >> The second fence above need not be a barrier particularly if it had a NOPRECEDE. >> >> >> Note that in a fence-put-fence you are allowed to do a barrier in the first fence, but you are not required to. You can do a deferred mode where the first fence is a no-op, all RMA ops are queued, and everything is done in the second fence and determining that all processes have reached there. >> >> Rajeev >> >> >> >> On Feb 25, 2014, at 9:48 PM, "Balaji, Pavan" <balaji(a)anl.gov> >> wrote: >> >>> >>> Besides, the second sentence says, if MPI_MODE_NOPRECEDE is given, it’ll not act as a barrier. >>> >>> What’s the necessity for that if WIN_FENCE does not act as a barrier even when MPI_MODE_NOPRECEDE is not given. >>> >>> — Pavan >>> >>> On Feb 25, 2014, at 9:38 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >>> >>>> >>>> In practice, it’ll need to have barrier semantics. Otherwise, PUT will need to be a two-sided operation to ensure that it’s not issued before the other process calls MPI_WIN_FENCE. >>>> >>>> — Pavan >>>> >>>> On Feb 25, 2014, at 9:21 PM, Rajeev Thakur <thakur(a)mcs.anl.gov> wrote: >>>> >>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>> >>>>> It only says "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". NOT "This requires MPI_WIN_FENCE that starts an epoch to act as an barrier." >>>>> >>>>> Why does the fence have to act as a barrier. The handshake could be done when first RMA operation is called after the fence. >>>>> >>>>> Rajeev >>>>> >>>>> >>>>> On Feb 25, 2014, at 8:41 PM, "Zhao, Xin" <xinzhao3(a)illinois.edu> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> There is an inconsistency of MPI_WIN_FENCE semantic in MPI 3.0 Standard that makes me confused: >>>>>> >>>>>> (1) On P440-P441 it say that "RMA operations on win started by a process after the fence call returns will access their target window only after MPI_WIN_FENCE has been called by the target process". This requires MPI_WIN_FENCE that starts an epoch to act as an barrier. >>>>>> >>>>>> (2) However, (1) contradict with the word at end of P441: "a call to MPI_WIN_FENCE that is known not to end any epoch (in particular a call with assert equal to MPI_MODE_NOPRECEDE) does not necessarily act as a barrier". >>>>>> >>>>>> Should the word of (1) add: "when MPI_MODE_NOPRECEDE is not given"? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Xin >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-rma mailing list >>>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>>> >>>>> _______________________________________________ >>>>> mpiwg-rma mailing list >>>>> mpiwg-rma(a)lists.mpi-forum.org >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>>> >>>> _______________________________________________ >>>> mpiwg-rma mailing list >>>> mpiwg-rma(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >>> >>> _______________________________________________ >>> mpiwg-rma mailing list >>> mpiwg-rma(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0