December 2010 - mpiwg-rma - mailman3.cels.anl.gov

Re: [Mpi3-rma] Conflicting accesses
by Underwood, Keith D 16 Dec '10

16 Dec '10

> > Ewww... yeah, I think that should work. Would this also work? It > > might be cheaper and easier to program than the barrier... > I would suspect that barrier is the easiest for average programmers. Yes, but an asynchronous code might have trouble figuring out when it was appropriate to barrier... > > Rank 0: > > (addr 42 starts as 0) > > MPI_Put(rank=1, addr=0, size=1912) > > MPI_Flush(rank=1) > > MPI_Accumulate(rank=1, addr=42, MPI_REPLACE, 1) > > MPI_Flush(rank=1) > > > > Rank 1: > > while (temp == 0) {MPI_GetAccumulate(rank=1, addr=42, &temp); > MPI_Flush(rank=1);} > > MPI_Get(rank=1, addr=12) > > Yes, that should work but would poll on address 42 (many messages). > (You > are aware that the accumulate address overlaps with the Put buffer, > which is ok here but might be unexpected). Brain glitch - making MPI_Put long was a last minute change ;-) Yeah, I didn't intend to make those overlap. Keith

1 0

Re: [Mpi3-rma] Conflicting accesses
by James Dinan 16 Dec '10

16 Dec '10

On 12/16/2010 12:21 PM, Torsten Hoefler wrote: >> Ewww... yeah, I think that should work. Would this also work? It >> might be cheaper and easier to program than the barrier... > > I would suspect that barrier is the easiest for average programmers. This idiom of update shared followed by barrier is actually pretty common in PGAS code - especially in non-performance critical parts. >> Rank 0: >> (addr 42 starts as 0) >> MPI_Put(rank=1, addr=0, size=1912) >> MPI_Flush(rank=1) >> MPI_Accumulate(rank=1, addr=42, MPI_REPLACE, 1) >> MPI_Flush(rank=1) >> >> Rank 1: >> while (temp == 0) {MPI_GetAccumulate(rank=1, addr=42,&temp); MPI_Flush(rank=1);} >> MPI_Get(rank=1, addr=12) > > Yes, that should work but would poll on address 42 (many messages). (You > are aware that the accumulate address overlaps with the Put buffer, > which is ok here but might be unexpected). Looks okay to me too (minus the overlap), this is definitely something we need to support. We can also use two-sided messaging to notify to a waiting process: Rank 0: MPI_Put(rank=1, addr=0, size=1912) MPI_Flush(rank=1) MPI_Send(rank=1,tag=NOTIFY_TAG,...) Rank 1: MPI_Recv(rank=MPI_SOURCE_ANY,tag=NOTIFY_TAG,...) MPI_Get(rank=1, addr=12) ~Jim.

1 0

Re: [Mpi3-rma] Conflicting accesses
by Torsten Hoefler 16 Dec '10

16 Dec '10

On Thu, Dec 16, 2010 at 10:13:04AM -0700, Underwood, Keith D wrote: > > Yes, the example was exclusively locked (which is what Jim did in his > > code). However, I would say that it's important to also allow this > > within shared lock epochs. But this would then force us to define > > something like a pairwise consistency, i.e., conflicting accesses are > > only guaranteed to produce useful outcome if they are between the same > > source-destination pair and if they are separated by flush calls. This > > is what is implied right now (if we remove the "for example" sentence). > > > > However, we have another problem in lockall/shared. I'd say the > > following should be valid too: > > > > Rank 0: > > MPI_Put(rank=1, addr=0) > > MPI_Flush(rank=1) > > MPI_Barrier() > > > > Rank 1: > > MPI_Barrier() > > > > Rank 2: > > MPI_Barrier() > > MPI_Get(rank=1, addr=0) > > > > > > > > I think this is implied with the current semantics too, do we all > > agree? > > > > Put and Get are conflicting, however, they are separated by a Flush and > > the according dependencies. We should make this more explicit (could be > > tricky to define). > > Ewww... yeah, I think that should work. Would this also work? It > might be cheaper and easier to program than the barrier... I would suspect that barrier is the easiest for average programmers. > Rank 0: > (addr 42 starts as 0) > MPI_Put(rank=1, addr=0, size=1912) > MPI_Flush(rank=1) > MPI_Accumulate(rank=1, addr=42, MPI_REPLACE, 1) > MPI_Flush(rank=1) > > Rank 1: > while (temp == 0) {MPI_GetAccumulate(rank=1, addr=42, &temp); MPI_Flush(rank=1);} > MPI_Get(rank=1, addr=12) Yes, that should work but would poll on address 42 (many messages). (You are aware that the accumulate address overlaps with the Put buffer, which is ok here but might be unexpected). All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] Conflicting accesses
by Underwood, Keith D 16 Dec '10

16 Dec '10

> Yes, the example was exclusively locked (which is what Jim did in his > code). However, I would say that it's important to also allow this > within shared lock epochs. But this would then force us to define > something like a pairwise consistency, i.e., conflicting accesses are > only guaranteed to produce useful outcome if they are between the same > source-destination pair and if they are separated by flush calls. This > is what is implied right now (if we remove the "for example" sentence). > > However, we have another problem in lockall/shared. I'd say the > following should be valid too: > > Rank 0: > MPI_Put(rank=1, addr=0) > MPI_Flush(rank=1) > MPI_Barrier() > > Rank 1: > MPI_Barrier() > > Rank 2: > MPI_Barrier() > MPI_Get(rank=1, addr=0) > > > > I think this is implied with the current semantics too, do we all > agree? > > Put and Get are conflicting, however, they are separated by a Flush and > the according dependencies. We should make this more explicit (could be > tricky to define). Ewww... yeah, I think that should work. Would this also work? It might be cheaper and easier to program than the barrier... Rank 0: (addr 42 starts as 0) MPI_Put(rank=1, addr=0, size=1912) MPI_Flush(rank=1) MPI_Accumulate(rank=1, addr=42, MPI_REPLACE, 1) MPI_Flush(rank=1) Rank 1: while (temp == 0) {MPI_GetAccumulate(rank=1, addr=42, &temp); MPI_Flush(rank=1);} MPI_Get(rank=1, addr=12)

1 0

Re: [Mpi3-rma] Conflicting accesses
by Barrett, Brian W 14 Dec '10

14 Dec '10

On Dec 13, 2010, at 2:54 PM, Torsten Hoefler wrote: > On Sun, Dec 12, 2010 at 08:14:05PM -0700, Underwood, Keith D wrote: >> In theory, this example should work in the new proposal, but what does >> that have to do with your text above the example? You wrote two >> paragraphs about lockall/shared and your example is lock(exclusive). >> If it is not an exclusive lock, then this should work - as long as you >> could guarantee that there would be no conflicting accesses from other >> nodes. I'm not sure how useful this example would be without an >> exclusive lock... We shouldn't give this example without an exclusive >> lock without a LOT of big BOLD warning text ;-) > Yes, the example was exclusively locked (which is what Jim did in his > code). However, I would say that it's important to also allow this > within shared lock epochs. But this would then force us to define > something like a pairwise consistency, i.e., conflicting accesses are > only guaranteed to produce useful outcome if they are between the same > source-destination pair and if they are separated by flush calls. This > is what is implied right now (if we remove the "for example" sentence). > > However, we have another problem in lockall/shared. I'd say the > following should be valid too: > > Rank 0: > MPI_Put(rank=1, addr=0) > MPI_Flush(rank=1) > MPI_Barrier() > > Rank 1: > MPI_Barrier() > > Rank 2: > MPI_Barrier() > MPI_Get(rank=1, addr=0) > > > > I think this is implied with the current semantics too, do we all agree? > > Put and Get are conflicting, however, they are separated by a Flush and > the according dependencies. We should make this more explicit (could be > tricky to define). I believe that the current rules would allow it (at least, I couldn't find anything that would disallow it, since we discuss conflicting accesses in terms of "becoming visible in the public window", which the flush would guarantee occurred. Is there any chance the non-unified model could have issues with this? Brian -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories

1 0

Re: [Mpi3-rma] Conflicting accesses
by Torsten Hoefler 13 Dec '10

13 Dec '10

On Sun, Dec 12, 2010 at 08:14:05PM -0700, Underwood, Keith D wrote: > In theory, this example should work in the new proposal, but what does > that have to do with your text above the example? You wrote two > paragraphs about lockall/shared and your example is lock(exclusive). > If it is not an exclusive lock, then this should work - as long as you > could guarantee that there would be no conflicting accesses from other > nodes. I'm not sure how useful this example would be without an > exclusive lock... We shouldn't give this example without an exclusive > lock without a LOT of big BOLD warning text ;-) Yes, the example was exclusively locked (which is what Jim did in his code). However, I would say that it's important to also allow this within shared lock epochs. But this would then force us to define something like a pairwise consistency, i.e., conflicting accesses are only guaranteed to produce useful outcome if they are between the same source-destination pair and if they are separated by flush calls. This is what is implied right now (if we remove the "for example" sentence). However, we have another problem in lockall/shared. I'd say the following should be valid too: Rank 0: MPI_Put(rank=1, addr=0) MPI_Flush(rank=1) MPI_Barrier() Rank 1: MPI_Barrier() Rank 2: MPI_Barrier() MPI_Get(rank=1, addr=0) I think this is implied with the current semantics too, do we all agree? Put and Get are conflicting, however, they are separated by a Flush and the according dependencies. We should make this more explicit (could be tricky to define). All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] Conflicting accesses
by Torsten Hoefler 13 Dec '10

13 Dec '10

On Sun, Dec 12, 2010 at 08:36:17PM -0600, Rajeev Thakur wrote: > Which sentence beginning with "For example" are you referring to? I > don't see it on pg 42 of the current proposal 1. Sorry, I meant page 40 line 39-40 :). > If the example below doesn't work, then what is the use of flush? Not much. I just wanted to point out that we might have misunderstood (or wrongly implied) the rules for conflicting accesses within the same epoch. All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi3-rma] Conflicting accesses
by Barrett, Brian W 13 Dec '10

13 Dec '10

Like Rajeev, I can't find the "For example" text Torsten's talking about. But I agree that the text on page 10 does seem to make it sound like the example Torsten gives should be valid. Further, I would think the following example would also be valid: lockall(shared) get(tail) flush() do { compare_and_swap(new_tail, tail, tmp) } while (tail != tmp) unlockall(shared) So maybe that's good enough... Brian On Dec 12, 2010, at 8:14 PM, Underwood, Keith D wrote: > In theory, this example should work in the new proposal, but what does that have to do with your text above the example? You wrote two paragraphs about lockall/shared and your example is lock(exclusive). If it is not an exclusive lock, then this should work - as long as you could guarantee that there would be no conflicting accesses from other nodes. I'm not sure how useful this example would be without an exclusive lock... We shouldn't give this example without an exclusive lock without a LOT of big BOLD warning text ;-) > > Keith > >> -----Original Message----- >> From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- >> bounces(a)lists.mpi-forum.org] On Behalf Of Torsten Hoefler >> Sent: Saturday, December 11, 2010 5:51 PM >> To: mpi3-rma(a)lists.mpi-forum.org >> Subject: [Mpi3-rma] Conflicting accesses >> >> Hi all, >> >> We discussed the issue with the lockall/shared mode that the outcome of >> overlapping puts or put/get will be undefined. Our discussion ended at >> a >> point where we believed that such accesses are not valid in >> lockall/shared because there is only one access/exposure epoch. >> >> The statement (not in MPI-2) at the end of page 42 seems to imply that. >> However, I don't think that anything on page 10 (the rules for >> conflicting accesses) mandate this. Jim created this interesting >> example >> and both of us think it should be legal to do (without unlocking as was >> required in MPI-2): >> >> lock(exclusive) >> get(tail) >> flush() >> put(tail) >> unlock() >> >> It should be true because flush() implies completion at the target >> which >> means that the "mini-epoch" is now finished (yes, in MPI-2, we needed >> to >> close an epoch to get completion, however, in MPI-3 we don't). >> >> Do we all agree? I'd like to remove the sentence starting with "For >> example" on page 42. >> >> All the Best, >> Torsten >> >> -- >> bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- >> Torsten Hoefler | Performance Modeling and Simulation Lead >> Blue Waters Directorate | University of Illinois (UIUC) >> 1205 W Clark Street | Urbana, IL, 61801 >> NCSA Building | +01 (217) 244-7736 >> _______________________________________________ >> mpi3-rma mailing list >> mpi3-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > -- Brian W. Barrett Dept. 1423: Scalable System Software Sandia National Laboratories

1 0

Re: [Mpi3-rma] Conflicting accesses
by William Gropp 12 Dec '10

12 Dec '10

I agree that this example should work. Bill On Dec 11, 2010, at 6:50 PM, Torsten Hoefler wrote: > Hi all, > > We discussed the issue with the lockall/shared mode that the outcome > of > overlapping puts or put/get will be undefined. Our discussion ended > at a > point where we believed that such accesses are not valid in > lockall/shared because there is only one access/exposure epoch. > > The statement (not in MPI-2) at the end of page 42 seems to imply > that. > However, I don't think that anything on page 10 (the rules for > conflicting accesses) mandate this. Jim created this interesting > example > and both of us think it should be legal to do (without unlocking as > was > required in MPI-2): > > lock(exclusive) > get(tail) > flush() > put(tail) > unlock() > > It should be true because flush() implies completion at the target > which > means that the "mini-epoch" is now finished (yes, in MPI-2, we > needed to > close an epoch to get completion, however, in MPI-3 we don't). > > Do we all agree? I'd like to remove the sentence starting with "For > example" on page 42. > > All the Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Re: [Mpi3-rma] Conflicting accesses
by Underwood, Keith D 12 Dec '10

12 Dec '10

In theory, this example should work in the new proposal, but what does that have to do with your text above the example? You wrote two paragraphs about lockall/shared and your example is lock(exclusive). If it is not an exclusive lock, then this should work - as long as you could guarantee that there would be no conflicting accesses from other nodes. I'm not sure how useful this example would be without an exclusive lock... We shouldn't give this example without an exclusive lock without a LOT of big BOLD warning text ;-) Keith > -----Original Message----- > From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- > bounces(a)lists.mpi-forum.org] On Behalf Of Torsten Hoefler > Sent: Saturday, December 11, 2010 5:51 PM > To: mpi3-rma(a)lists.mpi-forum.org > Subject: [Mpi3-rma] Conflicting accesses > > Hi all, > > We discussed the issue with the lockall/shared mode that the outcome of > overlapping puts or put/get will be undefined. Our discussion ended at > a > point where we believed that such accesses are not valid in > lockall/shared because there is only one access/exposure epoch. > > The statement (not in MPI-2) at the end of page 42 seems to imply that. > However, I don't think that anything on page 10 (the rules for > conflicting accesses) mandate this. Jim created this interesting > example > and both of us think it should be legal to do (without unlocking as was > required in MPI-2): > > lock(exclusive) > get(tail) > flush() > put(tail) > unlock() > > It should be true because flush() implies completion at the target > which > means that the "mini-epoch" is now finished (yes, in MPI-2, we needed > to > close an epoch to get completion, however, in MPI-3 we don't). > > Do we all agree? I'd like to remove the sentence starting with "For > example" on page 42. > > All the Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Torsten Hoefler | Performance Modeling and Simulation Lead > Blue Waters Directorate | University of Illinois (UIUC) > 1205 W Clark Street | Urbana, IL, 61801 > NCSA Building | +01 (217) 244-7736 > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0