April 2009 - mpi-forum - mailman3.cels.anl.gov

Re: [Mpi-forum] MPI One-Sided Communication
by Greg Lindahl 23 Apr '09

23 Apr '09

On Thu, Apr 23, 2009 at 01:28:22PM -0600, Simone Atzeni wrote: > Hi, I'm an italian student and I'm doing a thesis about Formal > Verification. Now, I'm looking for information about mpi one-sided > communication to see who uses one-sided and for what (user applications > or implementing library). As far as I can tell, usage of the MPI-2 one-sided library is about zero. People tried it on various interconnects, and when it turned out to be slower than 2-sided, stopped trying. The only significant public one-sided app that I've run into is NWCHEM, which uses a non-MPI one-sided library. And in that case, PathScale did an implementation on top of 2-sided MPI-1 send/recv, and it scaled better than competing real one-sided hardware. (Check out my old whitepaper for numbers.) I've gored enough oxen for one post, so I won't rain on your model-checking parade ;-) Now that I'm not an HPC guy anymore, I'll note that the distributed database that I'm doing is completely implemeted in terms of active messages. They're easier to use corrrectly than real one-sided communications, which are hard because programmers screw up synchronization & double-buffering. -- greg p.s. I'm agreeing with Tony?!

1 0

Re: [Mpi-forum] MPI-3 One-Sided Communications
by Anthony Skjellum 23 Apr '09

23 Apr '09

Can you point me at the proposed solutions? :-) On Apr 23, 2009, at 8:07 PM, Vinod tipparaju wrote: > > First, as far as my understanding goes, no, that is not why we have > the active message working group. > > I will not comment on if the MPI-2 missed its mark. You are more > familiar with what went on that I am. In my opinion, it was a design > that approached the problem of defining one-sided data access in a > ubiquitous way on both CC and non-CC machines -- I wasn't > participating in the forum when these choices where made. > > Lower latency than two sided means many things. One of them is tag > matching ( as you know very well, this can have good and bad effects > based on the scenario). There are may others such as lower latency > single element updates and lesser than the currently necessary > synchronization for remote data access and update. > > All of these have been addressed with a proposed potential solution > in MPI forum RMA working group in _two_ different ways with-in the > last one year. If you think neither of these two approaches have > satisfied the "starting over" criterion, I would like to know, why? > > Bandwidth is an entirely different story. > > Vinod Tipparaju. > > > To: mpi-forum(a)lists.mpi-forum.org > > From: tony(a)cis.uab.edu > > Date: Fri, 24 Apr 2009 00:32:54 +0000 > > Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications > > > > Hi, I was giving my opinion... > > > > The main requirements for one sided was achieve lower latency than > two sided (as well as one sidedness), and allow for potentially > lower bandwidth on long transfers as a side effect... What one sided > achieved is higher latency and higher bandwidth in typical (quality) > implementations. The api is consequently a mismatch to its original > purpose... It diverged from its design center.. Missed the mark... > Way too complex compared to simple put and get. > > > > Starting again from first principles with the goal of very low > latency remote put and get without the legacy of existing one sided > is my suggestion. > > > > In other words : Start over. > > > > Perhaps that is why now we have an active messages group? > > Not sure. > > > > > > > > Tony > > Anthony Skjellum, PhD > > Professor and Chair > > Dept. of Computer and Information Sciences > > University of Alabama at Birmingham > > +1-205-807-4968 > > > > -----Original Message----- > > From: Vinod tipparaju <tipparajuv(a)hotmail.com> > > > > Date: Thu, 23 Apr 2009 20:09:38 > > To: <mpi-forum(a)lists.mpi-forum.org> > > Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > > > > _______________________________________________ > > mpi-forum mailing list > > mpi-forum(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] MPI-3 One-Sided Communications
by Anthony Skjellum 23 Apr '09

23 Apr '09

Hi, I am not familiar with you project. Tell us more. In order to achieve lowest latency (or overhead, depending on your optimization, or a blend thereof), a protocol designer might not want to pay for fixed costs that once amortized over long transfers yields more long transfer performance. Classic Fixed vs variable cost situation. The classic trades in two-sided are zero copy for long and two copy for short. If you have evidence to the contrary, great... Lots of people reported in the past the effect of a tradeoff as I described coming from the complex semantics of the mpi one sided api. One also has to ask if we can get even lower latency with less complex protocol too imho. Do you get both lower latency than two sided short msgs and higher bandwidth than two sided? What is the baseline of performance for two sided? Is it optimized? :-) Tony Anthony Skjellum, PhD Verari Systems, Inc +1-205-807-4968 -----Original Message----- From: Jeff Hammond <jeff.science(a)gmail.com> Date: Thu, 23 Apr 2009 20:03:07 To: Main MPI Forum mailing list<mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications What does "allow for potentially lower bandwidth on long transfers as a side effect" mean? That one-sided should have lower latency at the expense of lower bandwidth? That's an unnecessary and intolerable compromise. ARMCI proves that one-sided bandwidth can equal if not beat MPI (http://www.emsl.pnl.gov/docs/parsoft/armci/performance.htm) Jeff On Thu, Apr 23, 2009 at 7:32 PM, Anthony Skjellum <tony(a)cis.uab.edu> wrote: > Hi, I was giving my opinion... > > The main requirements for one sided was achieve lower latency than two sided (as well as one sidedness), and allow for potentially lower bandwidth on long transfers as a side effect... What one sided achieved is higher latency and higher bandwidth in typical (quality) implementations. ï¿½The api is consequently a mismatch to its original purpose... It diverged from its design center.. Missed the mark... Way too complex compared to simple put and get. > > Starting again from first principles with the goal of very low latency remote put and get without the legacy of existing one sided is my suggestion. > > In other words : Start over. > > Perhaps that is why now we have an active messages group? > Not sure. > > > > Tony > Anthony Skjellum, PhD > Professor and Chair > Dept. of Computer and Information Sciences > University of Alabama at Birmingham > +1-205-807-4968 > > -----Original Message----- > From: Vinod tipparaju <tipparajuv(a)hotmail.com> > > Date: Thu, 23 Apr 2009 20:09:38 > To: <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications > > >_______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > >_______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- Jeff Hammond The University of Chicago http://home.uchicago.edu/~jhammond/ _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] MPI-3 One-Sided Communications
by Vinod tipparaju 23 Apr '09

23 Apr '09

First, as far as my understanding goes, no, that is not why we have the active message working group. I will not comment on if the MPI-2 missed its mark. You are more familiar with what went on that I am. In my opinion, it was a design that approached the problem of defining one-sided data access in a ubiquitous way on both CC and non-CC machines -- I wasn't participating in the forum when these choices where made. Lower latency than two sided means many things. One of them is tag matching ( as you know very well, this can have good and bad effects based on the scenario). There are may others such as lower latency single element updates and lesser than the currently necessary synchronization for remote data access and update. All of these have been addressed with a proposed potential solution in MPI forum RMA working group in _two_ different ways with-in the last one year. If you think neither of these two approaches have satisfied the "starting over" criterion, I would like to know, why? Bandwidth is an entirely different story. Vinod Tipparaju. > To: mpi-forum(a)lists.mpi-forum.org > From: tony(a)cis.uab.edu > Date: Fri, 24 Apr 2009 00:32:54 +0000 > Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications > > Hi, I was giving my opinion... > > The main requirements for one sided was achieve lower latency than two sided (as well as one sidedness), and allow for potentially lower bandwidth on long transfers as a side effect... What one sided achieved is higher latency and higher bandwidth in typical (quality) implementations. The api is consequently a mismatch to its original purpose... It diverged from its design center.. Missed the mark... Way too complex compared to simple put and get. > > Starting again from first principles with the goal of very low latency remote put and get without the legacy of existing one sided is my suggestion. > > In other words : Start over. > > Perhaps that is why now we have an active messages group? > Not sure. > > > > Tony > Anthony Skjellum, PhD > Professor and Chair > Dept. of Computer and Information Sciences > University of Alabama at Birmingham > +1-205-807-4968 > > -----Original Message----- > From: Vinod tipparaju <tipparajuv(a)hotmail.com> > > Date: Thu, 23 Apr 2009 20:09:38 > To: <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] MPI-3 One-Sided Communications
by Jeff Hammond 23 Apr '09

23 Apr '09

What does "allow for potentially lower bandwidth on long transfers as a side effect" mean? That one-sided should have lower latency at the expense of lower bandwidth? That's an unnecessary and intolerable compromise. ARMCI proves that one-sided bandwidth can equal if not beat MPI (http://www.emsl.pnl.gov/docs/parsoft/armci/performance.htm) Jeff On Thu, Apr 23, 2009 at 7:32 PM, Anthony Skjellum <tony(a)cis.uab.edu> wrote: > Hi, I was giving my opinion... > > The main requirements for one sided was achieve lower latency than two sided (as well as one sidedness), and allow for potentially lower bandwidth on long transfers as a side effect... What one sided achieved is higher latency and higher bandwidth in typical (quality) implementations. ï¿½The api is consequently a mismatch to its original purpose... It diverged from its design center.. Missed the mark... Way too complex compared to simple put and get. > > Starting again from first principles with the goal of very low latency remote put and get without the legacy of existing one sided is my suggestion. > > In other words : Start over. > > Perhaps that is why now we have an active messages group? > Not sure. > > > > Tony > Anthony Skjellum, PhD > Professor and Chair > Dept. of Computer and Information Sciences > University of Alabama at Birmingham > +1-205-807-4968 > > -----Original Message----- > From: Vinod tipparaju <tipparajuv(a)hotmail.com> > > Date: Thu, 23 Apr 2009 20:09:38 > To: <mpi-forum(a)lists.mpi-forum.org> > Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- Jeff Hammond The University of Chicago http://home.uchicago.edu/~jhammond/

1 0

Re: [Mpi-forum] MPI-3 One-Sided Communications
by Anthony Skjellum 23 Apr '09

23 Apr '09

Hi, I was giving my opinion... The main requirements for one sided was achieve lower latency than two sided (as well as one sidedness), and allow for potentially lower bandwidth on long transfers as a side effect... What one sided achieved is higher latency and higher bandwidth in typical (quality) implementations. The api is consequently a mismatch to its original purpose... It diverged from its design center.. Missed the mark... Way too complex compared to simple put and get. Starting again from first principles with the goal of very low latency remote put and get without the legacy of existing one sided is my suggestion. In other words : Start over. Perhaps that is why now we have an active messages group? Not sure. Tony Anthony Skjellum, PhD Professor and Chair Dept. of Computer and Information Sciences University of Alabama at Birmingham +1-205-807-4968 -----Original Message----- From: Vinod tipparaju <tipparajuv(a)hotmail.com> Date: Thu, 23 Apr 2009 20:09:38 To: <mpi-forum(a)lists.mpi-forum.org> Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] MPI-3 One-Sided Communications
by Vinod tipparaju 23 Apr '09

23 Apr '09

I am not sure if every one is listening... if you are not, you should be. Many people have been talking and discussing in the Forum about RMA interfaces. I am not sure what you mean by starting from scratch? > Date: Thu, 23 Apr 2009 18:57:12 -0500 > From: tony(a)cis.uab.edu > To: mpi-forum(a)lists.mpi-forum.org > Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications > > I suggest we start from scratch. > > ----- Original Message ----- > From: "Jeff Squyres" <jsquyres(a)cisco.com> > To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> > Sent: Thursday, April 23, 2009 3:59:35 PM GMT -06:00 US/Canada Central > Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications > > MPI-3 one-sided actually hasn't been defined yet. > > > On Apr 23, 2009, at 4:46 PM, Simone Atzeni wrote: > >> Hi, >> >> When will an implementation of MPI-3 one-sided be ready? >> >> Thanks >> Simone >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > > -- > Jeff Squyres > Cisco Systems > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > > -- > > Anthony Skjellum, PhD > Professor and Chair > Dept. of Computer and Information Sciences > University of Alabama at Birmingham > +1-(205)934-8657; FAX: +1- (205)934-5473 > > It has been said that MPI is the worst form of parallel programming except all the others that have been tried. > ___________________________________________ > CONFIDENTIALITY: This e-mail and any attachments are confidential and > may be privileged. If you are not a named recipient, please notify the > sender immediately and do not disclose the contents to another person, > use it for any purpose or store or copy the information in any medium. > > Please consider the environment before printing this e-mail > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] MPI-3 One-Sided Communications
by Anthony Skjellum 23 Apr '09

23 Apr '09

I suggest we start from scratch. ----- Original Message ----- From: "Jeff Squyres" <jsquyres(a)cisco.com> To: "Main MPI Forum mailing list" <mpi-forum(a)lists.mpi-forum.org> Sent: Thursday, April 23, 2009 3:59:35 PM GMT -06:00 US/Canada Central Subject: Re: [Mpi-forum] MPI-3 One-Sided Communications MPI-3 one-sided actually hasn't been defined yet. On Apr 23, 2009, at 4:46 PM, Simone Atzeni wrote: > Hi, > > When will an implementation of MPI-3 one-sided be ready? > > Thanks > Simone > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Squyres Cisco Systems _______________________________________________ mpi-forum mailing list mpi-forum(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Anthony Skjellum, PhD Professor and Chair Dept. of Computer and Information Sciences University of Alabama at Birmingham +1-(205)934-8657; FAX: +1- (205)934-5473 It has been said that MPI is the worst form of parallel programming except all the others that have been tried. ___________________________________________ CONFIDENTIALITY: This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium. Please consider the environment before printing this e-mail

1 0

Re: [Mpi-forum] MPI-3 One-Sided Communications
by Jeff Squyres 23 Apr '09

23 Apr '09

MPI-3 one-sided actually hasn't been defined yet. On Apr 23, 2009, at 4:46 PM, Simone Atzeni wrote: > Hi, > > When will an implementation of MPI-3 one-sided be ready? > > Thanks > Simone > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum -- Jeff Squyres Cisco Systems

1 0

MPI-3 One-Sided Communications
by Simone Atzeni 23 Apr '09

23 Apr '09

Hi, When will an implementation of MPI-3 one-sided be ready? Thanks Simone

1 0