February 2016 - mpi-forum - mailman3.cels.anl.gov

Re: [Mpi-forum] another request for iaccept
by Jeff Hammond 26 Feb '16

26 Feb '16

On Fri, Feb 26, 2016 at 12:25 AM, George Bosilca <bosilca(a)icl.utk.edu> wrote: > JeffH, > > You might have been misled by the date on the ticket. This is an old > (stale ticket) that was imported from TRAC when we moved to github. Thanks > for pointing it out. > > So the current release of Open-MPI supports MPI_THREAD_MULTIPLE for all BTL/MTL without exception? That is fantastic, because it means that the application community can finally move forward with interesting MPI+X designs. Please close that Github ticket now, because I frequently cite it as evidence that the MPI community does not take threads seriously and users unfortunately must rely upon MPICH-based implementations rather than implementations of the MPI standard in order to use MPI_THREAD_MULTIPLE. Jeff > That being said, your point remains valid, we should not force the users > to use complicated solution based on threads doing blocking MPI calls just > to cope with our inability to timely converge toward a reasonable > non-blocking solution. > > George. > > > On Fri, Feb 26, 2016 at 12:55 AM, Jeff Hammond <jeff.science(a)gmail.com> > wrote: > >> >> >> On Thu, Feb 25, 2016 at 3:40 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: >> >>> >>> > On Feb 25, 2016, at 2:48 PM, Jeff Squyres (jsquyres) < >>> jsquyres(a)cisco.com> wrote: >>> >> 1) Low overhead (justifies Isend/Irecv etc.) >>> >> 2) Scarcity of threads (e.g., the BlueGene/L rationale) >>> > >>> > Agreed -- neither of these are likely important for an >>> iconnect/iaccept scenario. >>> >>> I would disagree. This is *always* a problem since adding threads hurts >>> other operations in the application. For example, if I need to use a >>> nonblocking iconnect/iaccept in one small part of the application, it now >>> means that every fine-grained PUT/GET/ISEND operation in the rest of the >>> application would be more expensive. >>> >>> >> Does making a blocking operation nonblocking via threads not require >> MPI_THREAD_MULTIPLE, which Open-MPI does not support ( >> https://github.com/open-mpi/ompi/issues/157) more than 12 years after >> its standardization? It seems that a significant fraction of the MPI Forum >> does not believe that thread-safe MPI calls are important, so how can >> anyone argue that threads are a solution to this problem? >> >> Jeff >> >> >>> > But I do think the progression overlap with application threads can be >>> quite useful. >>> >>> Right. Having a nonblocking operation is not about performance >>> improvements, but that I can now stick in a request into an existing >>> Waitall or Testany in my application. FWIW, at least one of our >>> applications uses NBC I/O in exactly this way. Before MPI-3.1, they had to >>> do an event-based model (with Testany) for everything else and a blocking >>> call for I/O, which was inconvenient and hurts performance. >>> >>> >> There are some interactions with multiple-competion routines and >>> limitations in the generalized requests, but fixing generalized requests >>> would be a more general solution. >>> > >>> > Agreed -- fixing generalized requests has been a white whale for quite >>> a while now. >>> >>> There are technical reasons for why this was not easily fixed, unlike >>> iconnect/iaccept where people are bandwidth limited to put together a >>> proposal. >>> >>> -- Pavan >>> >>> _______________________________________________ >>> mpi-forum mailing list >>> mpi-forum(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >>> >> >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> http://jeffhammond.github.io/ >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] another request for iaccept
by George Bosilca 26 Feb '16

26 Feb '16

JeffH, You might have been misled by the date on the ticket. This is an old (stale ticket) that was imported from TRAC when we moved to github. Thanks for pointing it out. That being said, your point remains valid, we should not force the users to use complicated solution based on threads doing blocking MPI calls just to cope with our inability to timely converge toward a reasonable non-blocking solution. George. On Fri, Feb 26, 2016 at 12:55 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > > > On Thu, Feb 25, 2016 at 3:40 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > >> >> > On Feb 25, 2016, at 2:48 PM, Jeff Squyres (jsquyres) < >> jsquyres(a)cisco.com> wrote: >> >> 1) Low overhead (justifies Isend/Irecv etc.) >> >> 2) Scarcity of threads (e.g., the BlueGene/L rationale) >> > >> > Agreed -- neither of these are likely important for an iconnect/iaccept >> scenario. >> >> I would disagree. This is *always* a problem since adding threads hurts >> other operations in the application. For example, if I need to use a >> nonblocking iconnect/iaccept in one small part of the application, it now >> means that every fine-grained PUT/GET/ISEND operation in the rest of the >> application would be more expensive. >> >> > Does making a blocking operation nonblocking via threads not require > MPI_THREAD_MULTIPLE, which Open-MPI does not support ( > https://github.com/open-mpi/ompi/issues/157) more than 12 years after its > standardization? It seems that a significant fraction of the MPI Forum > does not believe that thread-safe MPI calls are important, so how can > anyone argue that threads are a solution to this problem? > > Jeff > > >> > But I do think the progression overlap with application threads can be >> quite useful. >> >> Right. Having a nonblocking operation is not about performance >> improvements, but that I can now stick in a request into an existing >> Waitall or Testany in my application. FWIW, at least one of our >> applications uses NBC I/O in exactly this way. Before MPI-3.1, they had to >> do an event-based model (with Testany) for everything else and a blocking >> call for I/O, which was inconvenient and hurts performance. >> >> >> There are some interactions with multiple-competion routines and >> limitations in the generalized requests, but fixing generalized requests >> would be a more general solution. >> > >> > Agreed -- fixing generalized requests has been a white whale for quite >> a while now. >> >> There are technical reasons for why this was not easily fixed, unlike >> iconnect/iaccept where people are bandwidth limited to put together a >> proposal. >> >> -- Pavan >> >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >> > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum >

1 0

Re: [Mpi-forum] another request for iaccept
by Jeff Hammond 25 Feb '16

25 Feb '16

On Thu, Feb 25, 2016 at 3:40 PM, Balaji, Pavan <balaji(a)anl.gov> wrote: > > > On Feb 25, 2016, at 2:48 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> > wrote: > >> 1) Low overhead (justifies Isend/Irecv etc.) > >> 2) Scarcity of threads (e.g., the BlueGene/L rationale) > > > > Agreed -- neither of these are likely important for an iconnect/iaccept > scenario. > > I would disagree. This is *always* a problem since adding threads hurts > other operations in the application. For example, if I need to use a > nonblocking iconnect/iaccept in one small part of the application, it now > means that every fine-grained PUT/GET/ISEND operation in the rest of the > application would be more expensive. > > Does making a blocking operation nonblocking via threads not require MPI_THREAD_MULTIPLE, which Open-MPI does not support ( https://github.com/open-mpi/ompi/issues/157) more than 12 years after its standardization? It seems that a significant fraction of the MPI Forum does not believe that thread-safe MPI calls are important, so how can anyone argue that threads are a solution to this problem? Jeff > > But I do think the progression overlap with application threads can be > quite useful. > > Right. Having a nonblocking operation is not about performance > improvements, but that I can now stick in a request into an existing > Waitall or Testany in my application. FWIW, at least one of our > applications uses NBC I/O in exactly this way. Before MPI-3.1, they had to > do an event-based model (with Testany) for everything else and a blocking > call for I/O, which was inconvenient and hurts performance. > > >> There are some interactions with multiple-competion routines and > limitations in the generalized requests, but fixing generalized requests > would be a more general solution. > > > > Agreed -- fixing generalized requests has been a white whale for quite a > while now. > > There are technical reasons for why this was not easily fixed, unlike > iconnect/iaccept where people are bandwidth limited to put together a > proposal. > > -- Pavan > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [Mpi-forum] another request for iaccept
by Balaji, Pavan 25 Feb '16

25 Feb '16

> On Feb 25, 2016, at 2:48 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: >> 1) Low overhead (justifies Isend/Irecv etc.) >> 2) Scarcity of threads (e.g., the BlueGene/L rationale) > > Agreed -- neither of these are likely important for an iconnect/iaccept scenario. I would disagree. This is *always* a problem since adding threads hurts other operations in the application. For example, if I need to use a nonblocking iconnect/iaccept in one small part of the application, it now means that every fine-grained PUT/GET/ISEND operation in the rest of the application would be more expensive. > But I do think the progression overlap with application threads can be quite useful. Right. Having a nonblocking operation is not about performance improvements, but that I can now stick in a request into an existing Waitall or Testany in my application. FWIW, at least one of our applications uses NBC I/O in exactly this way. Before MPI-3.1, they had to do an event-based model (with Testany) for everything else and a blocking call for I/O, which was inconvenient and hurts performance. >> There are some interactions with multiple-competion routines and limitations in the generalized requests, but fixing generalized requests would be a more general solution. > > Agreed -- fixing generalized requests has been a white whale for quite a while now. There are technical reasons for why this was not easily fixed, unlike iconnect/iaccept where people are bandwidth limited to put together a proposal. -- Pavan

1 0

Re: [Mpi-forum] another request for iaccept
by Jeff Squyres (jsquyres) 25 Feb '16

25 Feb '16

On Feb 25, 2016, at 2:44 PM, William Gropp <wgropp(a)illinois.edu> wrote: > > Some of the rationales do not apply. In particular, I don’t believe these are important for Accept: > > 1) Low overhead (justifies Isend/Irecv etc.) > 2) Scarcity of threads (e.g., the BlueGene/L rationale) Agreed -- neither of these are likely important for an iconnect/iaccept scenario. But I do think the progression overlap with application threads can be quite useful. > There are some interactions with multiple-competion routines and limitations in the generalized requests, but fixing generalized requests would be a more general solution. Agreed -- fixing generalized requests has been a white whale for quite a while now. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] another request for iaccept
by William Gropp 25 Feb '16

25 Feb '16

Some of the rationales do not apply. In particular, I don’t believe these are important for Accept: 1) Low overhead (justifies Isend/Irecv etc.) 2) Scarcity of threads (e.g., the BlueGene/L rationale) There are some interactions with multiple-competion routines and limitations in the generalized requests, but fixing generalized requests would be a more general solution. Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Feb 25, 2016, at 2:40 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: >> On Feb 25, 2016, at 2:01 PM, William Gropp <wgropp(a)illinois.edu> wrote: >> >> Part of that proposal must explain why using Accept in a separate thread (as suggested by this very post) isn’t sufficient. > > Is it different than the rationale for other non-blocking MPI operations? > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] another request for iaccept
by Jeff Squyres (jsquyres) 25 Feb '16

25 Feb '16

> On Feb 25, 2016, at 2:01 PM, William Gropp <wgropp(a)illinois.edu> wrote: > > Part of that proposal must explain why using Accept in a separate thread (as suggested by this very post) isn’t sufficient. Is it different than the rationale for other non-blocking MPI operations? -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi-forum] another request for iaccept
by William Gropp 25 Feb '16

25 Feb '16

Part of that proposal must explain why using Accept in a separate thread (as suggested by this very post) isn’t sufficient. Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Feb 25, 2016, at 1:56 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: > FWIW, I don't think the issue has been a lack of use cases. I think that the issue has been no one has been willing to create a proposal and shepherd it through. > > Sent from my phone. No type good. > >> On Feb 25, 2016, at 9:33 AM, Rob Latham <robl(a)mcs.anl.gov> wrote: >> >> Every 3-4 years I come across someone who wants to use a non-blocking accept, and the response is "the forum considered it but did not find a use case". Here's another end-user wishing for an iaccept: >> >> https://groups.google.com/d/msg/mpi4py/nSy17SavJnI/8EyfKT4-IAAJ >> >> ==rob >> _______________________________________________ >> mpi-forum mailing list >> mpi-forum(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

Re: [Mpi-forum] another request for iaccept
by Jeff Squyres (jsquyres) 25 Feb '16

25 Feb '16

FWIW, I don't think the issue has been a lack of use cases. I think that the issue has been no one has been willing to create a proposal and shepherd it through. Sent from my phone. No type good. > On Feb 25, 2016, at 9:33 AM, Rob Latham <robl(a)mcs.anl.gov> wrote: > > Every 3-4 years I come across someone who wants to use a non-blocking accept, and the response is "the forum considered it but did not find a use case". Here's another end-user wishing for an iaccept: > > https://groups.google.com/d/msg/mpi4py/nSy17SavJnI/8EyfKT4-IAAJ > > ==rob > _______________________________________________ > mpi-forum mailing list > mpi-forum(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

1 0

another request for iaccept
by Rob Latham 25 Feb '16

25 Feb '16

Every 3-4 years I come across someone who wants to use a non-blocking accept, and the response is "the forum considered it but did not find a use case". Here's another end-user wishing for an iaccept: https://groups.google.com/d/msg/mpi4py/nSy17SavJnI/8EyfKT4-IAAJ ==rob

1 0