April 2011 - mpiwg-rma - mailman3.cels.anl.gov

[Mpi3-rma] next RMA telecon, Mon April 18, 3pm Central
by Rajeev Thakur 11 Apr '11

11 Apr '11

The next RMA working group telecon will be at the same time next Monday. Please mark your calendars. Monday, April 18, 3.00pm Central Dial-in Number: 866-654-6744 International Dial-in Number: 1-517-308-8226 Participant Access code: 6086006 Rajeev

1 0

[Mpi3-rma] Reminder: next RMA telecon, Mon April 11, 3pm Central
by Rajeev Thakur 09 Apr '11

09 Apr '11

Reminder. RMA telecon on Monday. Begin forwarded message: > From: Rajeev Thakur <thakur(a)mcs.anl.gov> > Date: April 2, 2011 1:11:28 PM CDT > To: "MPI 3.0 Remote Memory Access working group" <mpi3-rma(a)lists.mpi-forum.org> > Subject: next RMA telecon, Mon April 11, 3pm Central > > The next RMA working group telecon will be on Monday, April 11, at 3.00pm Central. Please mark your calendars. > > Dial-in Number: 866-654-6744 > International Dial-in Number: 1-517-308-8226 > Participant Access code: 6086006 > > Rajeev >

1 0

[Mpi3-rma] First reading notes
by James Dinan 04 Apr '11

04 Apr '11

Hi All, I've attached my notes and edits from the first reading. Please let me know if anything needs discussion or more explanation. Thanks, ~Jim.

1 0

Re: [Mpi3-rma] GetAccumulate restriction needed?
by Underwood, Keith D 03 Apr '11

03 Apr '11

I suppose I could live with MPI_Get_accumulate_replace, if we had a use case... BTW, Pavan, for "use case", our typical goal has been a use case in some program using any API/language that could not be done in a reasonable, high performance way using exiting semantics. We've had that for most of our semantics so far... Keith > -----Original Message----- > From: mpi3-rma-bounces(a)lists.mpi-forum.org [mailto:mpi3-rma- > bounces(a)lists.mpi-forum.org] On Behalf Of Rajeev Thakur > Sent: Sunday, April 03, 2011 8:51 PM > To: MPI 3.0 Remote Memory Access working group > Subject: Re: [Mpi3-rma] GetAccumulate restriction needed? > > There is also MPI_Sendrecv_replace :-) > > > On Apr 3, 2011, at 9:32 PM, Underwood, Keith D wrote: > > > > >> On 04/03/2011 06:14 PM, Underwood, Keith D wrote: > >>> Could you perhaps provide a use case? Otherwise, it looks like an > >>> implementation nightmare with potentially unexpected performance > >>> characteristics associated with any working implementation choice. > >> > >> This would be required if the application does not want to create > two > >> buffers, one for the input and one for the output. The > implementation > >> can internally use pipelining to use a smaller buffer space. > > > > That is not a use case :-) My understanding of fetch-and-add style > operations is that you want both your increment and the old value at > the target. So, I am back to: what is the use case? > > > >> Also, we need this for symmetry. Two-sided calls support it. > > > > MPI_Sendrecv requires disjoint buffers, and that is the closest thing > in two-sided. > > > >> I don't follow the implementation nightmare with this? Why is this a > >> problem? > > > > Would you think the user would expect that having the same source and > target buffer would cause the implementation to: > > > > -allocate a buffer > > -perform a get into that buffer > > -wait for get completion > > -perform an atomic > > -wait for atomic completion > > -copy the data from the internal buffer to the user buffer > > > > Now, the part that sucks is that I believe the IB implementation > looks like this, but I believe that even a network that provided native > fetch-and-add semantics will wind up looking like this if it provides > an end-to-end reliability protocol. Heck, even if it doesn't, you may > do it rather than guesstimating the amount of network buffer between > the source and destination. That means that having the same source and > destination buffer will have radically different performance than > having different buffers. So, I count that as an implementation that > sucks ;-) > > > > Keith > > > > > > _______________________________________________ > > mpi3-rma mailing list > > mpi3-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] GetAccumulate restriction needed?
by Rajeev Thakur 03 Apr '11

03 Apr '11

There is also MPI_Sendrecv_replace :-) On Apr 3, 2011, at 9:32 PM, Underwood, Keith D wrote: > >> On 04/03/2011 06:14 PM, Underwood, Keith D wrote: >>> Could you perhaps provide a use case? Otherwise, it looks like an >>> implementation nightmare with potentially unexpected performance >>> characteristics associated with any working implementation choice. >> >> This would be required if the application does not want to create two >> buffers, one for the input and one for the output. The implementation >> can internally use pipelining to use a smaller buffer space. > > That is not a use case :-) My understanding of fetch-and-add style operations is that you want both your increment and the old value at the target. So, I am back to: what is the use case? > >> Also, we need this for symmetry. Two-sided calls support it. > > MPI_Sendrecv requires disjoint buffers, and that is the closest thing in two-sided. > >> I don't follow the implementation nightmare with this? Why is this a >> problem? > > Would you think the user would expect that having the same source and target buffer would cause the implementation to: > > -allocate a buffer > -perform a get into that buffer > -wait for get completion > -perform an atomic > -wait for atomic completion > -copy the data from the internal buffer to the user buffer > > Now, the part that sucks is that I believe the IB implementation looks like this, but I believe that even a network that provided native fetch-and-add semantics will wind up looking like this if it provides an end-to-end reliability protocol. Heck, even if it doesn't, you may do it rather than guesstimating the amount of network buffer between the source and destination. That means that having the same source and destination buffer will have radically different performance than having different buffers. So, I count that as an implementation that sucks ;-) > > Keith > > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] GetAccumulate restriction needed?
by Underwood, Keith D 03 Apr '11

03 Apr '11

> > Would you think the user would expect that having the same source and > > target buffer would cause the implementation to: > > > > -allocate a buffer > > -perform a get into that buffer > > -wait for get completion > > -perform an atomic > > -wait for atomic completion > > -copy the data from the internal buffer to the user buffer > > Why are you trying to it atomically? Get_accumulate is only element- > wise > atomic. Er, atomic only in the sense that Get_accumulate implies... So, replace "atomic" with "put" above and you still have something ugly. Point is: you have to wait for the get of a segment to complete before you can do the put. You have to wait for the put of a segment to complete before you can copy the result of the get (otherwise your retransmits are going to cause you great fun). Keith

1 0

Re: [Mpi3-rma] GetAccumulate restriction needed?
by Pavan Balaji 03 Apr '11

03 Apr '11

On 04/03/2011 09:32 PM, Underwood, Keith D wrote: >> This would be required if the application does not want to create two >> buffers, one for the input and one for the output. The implementation >> can internally use pipelining to use a smaller buffer space. > > That is not a use case :-) My understanding of fetch-and-add style > operations is that you want both your increment and the old value at > the target. So, I am back to: what is the use case? I don't have a use case as in an application that uses it currently, as I don't know of any applications that do large-message fetch-and-op today. >> Also, we need this for symmetry. Two-sided calls support it. > > MPI_Sendrecv requires disjoint buffers, and that is the closest thing in two-sided. Ah, yes. This is only for collectives. I misspoke. >> I don't follow the implementation nightmare with this? Why is this a >> problem? > > Would you think the user would expect that having the same source and > target buffer would cause the implementation to: > > -allocate a buffer > -perform a get into that buffer > -wait for get completion > -perform an atomic > -wait for atomic completion > -copy the data from the internal buffer to the user buffer Why are you trying to it atomically? Get_accumulate is only element-wise atomic. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi3-rma] GetAccumulate restriction needed?
by Underwood, Keith D 03 Apr '11

03 Apr '11

> On 04/03/2011 06:14 PM, Underwood, Keith D wrote: > > Could you perhaps provide a use case? Otherwise, it looks like an > > implementation nightmare with potentially unexpected performance > > characteristics associated with any working implementation choice. > > This would be required if the application does not want to create two > buffers, one for the input and one for the output. The implementation > can internally use pipelining to use a smaller buffer space. That is not a use case :-) My understanding of fetch-and-add style operations is that you want both your increment and the old value at the target. So, I am back to: what is the use case? > Also, we need this for symmetry. Two-sided calls support it. MPI_Sendrecv requires disjoint buffers, and that is the closest thing in two-sided. > I don't follow the implementation nightmare with this? Why is this a > problem? Would you think the user would expect that having the same source and target buffer would cause the implementation to: -allocate a buffer -perform a get into that buffer -wait for get completion -perform an atomic -wait for atomic completion -copy the data from the internal buffer to the user buffer Now, the part that sucks is that I believe the IB implementation looks like this, but I believe that even a network that provided native fetch-and-add semantics will wind up looking like this if it provides an end-to-end reliability protocol. Heck, even if it doesn't, you may do it rather than guesstimating the amount of network buffer between the source and destination. That means that having the same source and destination buffer will have radically different performance than having different buffers. So, I count that as an implementation that sucks ;-) Keith

1 0

Re: [Mpi3-rma] GetAccumulate restriction needed?
by Rajeev Thakur 03 Apr '11

03 Apr '11

No call tomorrow. It's a week from tomorrow. On Apr 3, 2011, at 6:14 PM, Underwood, Keith D wrote: >>> On 03/31/2011 10:20 PM, Rajeev Thakur wrote: >>>> Good catch. I think we should add that restriction. MPI_Sendrecv, >> for >>>> example, explicitly says the send and recv buffers must be disjoint. >>> >>> No MPI_IN_PLACE for Get_accumulate? >> Not yet, I agree that it could make sense though -- we should discuss >> this tomorrow. > > Could you perhaps provide a use case? Otherwise, it looks like an implementation nightmare with potentially unexpected performance characteristics associated with any working implementation choice. > > Do we have a call tomorrow? I thought Rajeev's note said 4/11... > > _______________________________________________ > mpi3-rma mailing list > mpi3-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-rma

1 0

Re: [Mpi3-rma] GetAccumulate restriction needed?
by Pavan Balaji 03 Apr '11

03 Apr '11

Hi Keith, On 04/03/2011 06:14 PM, Underwood, Keith D wrote: > Could you perhaps provide a use case? Otherwise, it looks like an > implementation nightmare with potentially unexpected performance > characteristics associated with any working implementation choice. This would be required if the application does not want to create two buffers, one for the input and one for the output. The implementation can internally use pipelining to use a smaller buffer space. Also, we need this for symmetry. Two-sided calls support it. I don't follow the implementation nightmare with this? Why is this a problem? -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0