February 2008 - Mpi3-subsetting - mailman3.cels.anl.gov

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

Hi, Thanks. I understand your motivation. When you say "most real applications" - what applications do you mean? At least, in what area? For the NIC part, the stress was on "here". In my opinion, subsetting is not about making things more complicated, more challenging to the implementors, or to the underlying hardware. It's about making things simple, easy to use, and easy to implement - including implementation of only those features your users actually need. That the implementation may be faster due to this is an added bonus, not the primary goal. Still, regarding user side copying. Yes, when people do this one wonders why. There's a reason, apart from them: 1) not caring about datatypes and their complexity and 2) not trusting their performance. A modern compiler can rather well optimize a loop with a constant stride, and may have difficulty with an unknown stride. This is why explicit loops are sometimes indeed faster (much faster) in the resulting code than any generic implementation. Best regards. Alexander -----Original Message----- From: Bronis R. de Supinski [mailto:bronis_at_[hidden]] Sent: Friday, February 29, 2008 6:20 AM To: Supalov, Alexander Cc: mpi3-subsetting_at_[hidden] Subject: RE: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09 Alexander: Most real applications need to send non-contiguous data. If they do not use datatypes then they are doing the equivalent of either the packing/unpacking or smaller messages at the user level. This s hould be discouraged, not encouraged. A small savings in library object size is not ample reason to go against that. And, yes, we are after encouraging hardware vendors to provide the right hardware. Bronis On Fri, 29 Feb 2008, Supalov, Alexander wrote: > Hi, > > Thanks. I think the main thrust here is the library footprint (no > pack/unpack, etc.) and complexity of the user side of the datatype > interface, rather than performance. Many applications just don't need > any of this, and never will. Why not translating this application > non-requirement into a minimum MPI subset? Same with communicator/group > management, etc. > > Moreover, homogeneous installations that dominate HPC now don't actually > need any datatype support at all. They send chunks of bytes. This may > change in the future, though. > > A minor performance implication is that without holes that are only > possible with derived datatypes, one does not need to track this, split > the critical path, and make special provisions inside the MPI device > layer to handle iov or such. > > The NIC capability argument is interesting, but it turns the discussion > on its head: we're not after motivating network vendors to provide > scatter/gather in hardware here, are we? Please clarify. > > Best regards. > > Alexander > > -----Original Message----- > From: mpi3-subsetting-bounces_at_[hidden] > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Bronis > R. de Supinski > Sent: Friday, February 29, 2008 5:53 AM > To: mpi3-subsetting_at_[hidden] > Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon > ww09 > > > All: > > OK, I have to respond to the notion that derived datatypes > limit performance. It is just not a reasonable position. > > Sure, if you can send contiguous locations, you will get > higher performance. The problem is that codes do not only > need to send contiguous data so that is not an adequate > reason to say derived datatypes limit performance. > > So, what is left? That there is some more efficient way > to send non-contiguous data? How? As multiple messages, > each of which send contiguous data? If so, then the > implementation could do this under the covers and the > datatypes are just a convenience for the user not to > have to specify the individual sends. OK, suppose that's > not the reason. Perhaps the user can do the copying into > a contiguous buffer and get better performance? While > I have seen this hold with some implementations, it is > absurd. There is no reason that I can discern as to why > the user should be able to deduce a better copying > mechanism than the MPI implementer. So, again, at worst, > the datatypes should be a convenience. Do you have an > alternative reason or a refutation of these opinions? > > What is more important, it is certainly possible to build > scatter/gather support into a NIC and achieve better > performance with datatypes than without. While there are > issues to be resolved for that (primarily the issue of > pinning memory), they are solvable with the right hardware > mechanism. Just because it does not yet exist is not > an adequate reason to say "Get rid of datatypes". OK, > you are not saying that but you are saying to deprecate > them in a sense. And saying you could send contiguous > sends more efficiently is a bad argument here. How do > datatypes cause inefficiency for that? How much is > that cost really? At what point do you hit where the > answer is "It would be faster not to compute anything"? > > Bronis > > > On Fri, 29 Feb 2008, Supalov, Alexander wrote: > > > Hi, > > > > Thanks. What subsets inside the current standard would you propose? > What > > interfaces between them would you envision? > > > > Good idea about the optimization opportunities. Here's an initial > > combined list, with the main benefits as I see them. Please > > comment/extend. > > > > - Dynamic process support: less overhead in the progress engine, > easier > > global rank handling. > > - Heterogeneity: better memory footprint, easier data handling. > > - Derived datatypes (especially those with holes): better memory > > footprint. > > - MPI_ANY_SOURCE: faster, more simple multifabric progress. > > - File I/O: smaller requests, easier wait/test functions. > > - One-sided ops: no passive target w/o MPI calls - no extra progress > > thread. > > - Communicator & group management: better memory footprint. > > - Message tagging: better support for stable dataflow exchanges, > smaller > > packets. > > - Non-blocking communication: easier ordering, simplified request > > handling. > > > > Best regards. > > > > Alexander > > > > -----Original Message----- > > From: mpi3-subsetting-bounces_at_[hidden] > > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of > > Torsten Hoefler > > Sent: Friday, February 29, 2008 5:08 AM > > To: mpi3-subsetting_at_[hidden] > > Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon > > ww09 > > > > Hi, > > > Present: Leonid Meyerguz (Microsoft), Rich Graham (ORNL), Richard > > Barrett > > > (ORNL), Torsten Hoefler (ISU), Alexander Supalov (Intel) > > just for the record, it's "IU" not "ISU" :-) > > > > > - Scope of the effort > > > - Rich > > > - Minimum subset consistent with the rest of MPI, for > > > performance/memory footprint optimization > > > - Danger of splitting MPI, hence against optional features in > > the > > > standard > > I back that (danger of optional features for portability). I'd propose > > to split the current standard into mostly self-contained subsets that > > have clearly defined interfaces to the rest of the standard. Note: > this > > only defines logical interfaces, that does *not* define how those > things > > are to be implemented. This makes it easier to understand the standard > > and have separate (portable) libraries for the subsets, it does not > > influence optimization possibilities by implementing everything in a > > monolithic block (i.e., central progress). > > > > > - Both blocking & nonblocking belong to the core > > > - Torsten > > > - Some collectives may go into selectable subsets > > I see three subsets: blocking colls, non-blocking colls and > topological > > colls (maybe also blocking / non-blocking). > > > > > - MPI_ANY_SOURCE considered harmful > > I'd like to add datatypes and heterogeneity to this list (with regards > > to performance). Alexander mentioned the dynamics. I think we should > > have a lit of items ready that could influence optimization > > possibilities significanty if they were to be announced by the user > > before he can use them. That would give another strong argument for > the > > subsetting. > > > > Best, > > Torsten > > > > -- > > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > > Indiana University | http://www.indiana.edu > > Open Systems Lab | http://osl.iu.edu/ > > 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA > > Lindley Hall Room 135 | +01 (812) 855-3608 > > _______________________________________________ > > Mpi3-subsetting mailing list > > Mpi3-subsetting_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > > > > > _______________________________________________ > > Mpi3-subsetting mailing list > > Mpi3-subsetting_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > > > _______________________________________________ > Mpi3-subsetting mailing list > Mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

Alexander: Most real applications need to send non-contiguous data. If they do not use datatypes then they are doing the equivalent of either the packing/unpacking or smaller messages at the user level. This s hould be discouraged, not encouraged. A small savings in library object size is not ample reason to go against that. And, yes, we are after encouraging hardware vendors to provide the right hardware. Bronis On Fri, 29 Feb 2008, Supalov, Alexander wrote: > Hi, > > Thanks. I think the main thrust here is the library footprint (no > pack/unpack, etc.) and complexity of the user side of the datatype > interface, rather than performance. Many applications just don't need > any of this, and never will. Why not translating this application > non-requirement into a minimum MPI subset? Same with communicator/group > management, etc. > > Moreover, homogeneous installations that dominate HPC now don't actually > need any datatype support at all. They send chunks of bytes. This may > change in the future, though. > > A minor performance implication is that without holes that are only > possible with derived datatypes, one does not need to track this, split > the critical path, and make special provisions inside the MPI device > layer to handle iov or such. > > The NIC capability argument is interesting, but it turns the discussion > on its head: we're not after motivating network vendors to provide > scatter/gather in hardware here, are we? Please clarify. > > Best regards. > > Alexander > > -----Original Message----- > From: mpi3-subsetting-bounces_at_[hidden] > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Bronis > R. de Supinski > Sent: Friday, February 29, 2008 5:53 AM > To: mpi3-subsetting_at_[hidden] > Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon > ww09 > > > All: > > OK, I have to respond to the notion that derived datatypes > limit performance. It is just not a reasonable position. > > Sure, if you can send contiguous locations, you will get > higher performance. The problem is that codes do not only > need to send contiguous data so that is not an adequate > reason to say derived datatypes limit performance. > > So, what is left? That there is some more efficient way > to send non-contiguous data? How? As multiple messages, > each of which send contiguous data? If so, then the > implementation could do this under the covers and the > datatypes are just a convenience for the user not to > have to specify the individual sends. OK, suppose that's > not the reason. Perhaps the user can do the copying into > a contiguous buffer and get better performance? While > I have seen this hold with some implementations, it is > absurd. There is no reason that I can discern as to why > the user should be able to deduce a better copying > mechanism than the MPI implementer. So, again, at worst, > the datatypes should be a convenience. Do you have an > alternative reason or a refutation of these opinions? > > What is more important, it is certainly possible to build > scatter/gather support into a NIC and achieve better > performance with datatypes than without. While there are > issues to be resolved for that (primarily the issue of > pinning memory), they are solvable with the right hardware > mechanism. Just because it does not yet exist is not > an adequate reason to say "Get rid of datatypes". OK, > you are not saying that but you are saying to deprecate > them in a sense. And saying you could send contiguous > sends more efficiently is a bad argument here. How do > datatypes cause inefficiency for that? How much is > that cost really? At what point do you hit where the > answer is "It would be faster not to compute anything"? > > Bronis > > > On Fri, 29 Feb 2008, Supalov, Alexander wrote: > > > Hi, > > > > Thanks. What subsets inside the current standard would you propose? > What > > interfaces between them would you envision? > > > > Good idea about the optimization opportunities. Here's an initial > > combined list, with the main benefits as I see them. Please > > comment/extend. > > > > - Dynamic process support: less overhead in the progress engine, > easier > > global rank handling. > > - Heterogeneity: better memory footprint, easier data handling. > > - Derived datatypes (especially those with holes): better memory > > footprint. > > - MPI_ANY_SOURCE: faster, more simple multifabric progress. > > - File I/O: smaller requests, easier wait/test functions. > > - One-sided ops: no passive target w/o MPI calls - no extra progress > > thread. > > - Communicator & group management: better memory footprint. > > - Message tagging: better support for stable dataflow exchanges, > smaller > > packets. > > - Non-blocking communication: easier ordering, simplified request > > handling. > > > > Best regards. > > > > Alexander > > > > -----Original Message----- > > From: mpi3-subsetting-bounces_at_[hidden] > > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of > > Torsten Hoefler > > Sent: Friday, February 29, 2008 5:08 AM > > To: mpi3-subsetting_at_[hidden] > > Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon > > ww09 > > > > Hi, > > > Present: Leonid Meyerguz (Microsoft), Rich Graham (ORNL), Richard > > Barrett > > > (ORNL), Torsten Hoefler (ISU), Alexander Supalov (Intel) > > just for the record, it's "IU" not "ISU" :-) > > > > > - Scope of the effort > > > - Rich > > > - Minimum subset consistent with the rest of MPI, for > > > performance/memory footprint optimization > > > - Danger of splitting MPI, hence against optional features in > > the > > > standard > > I back that (danger of optional features for portability). I'd propose > > to split the current standard into mostly self-contained subsets that > > have clearly defined interfaces to the rest of the standard. Note: > this > > only defines logical interfaces, that does *not* define how those > things > > are to be implemented. This makes it easier to understand the standard > > and have separate (portable) libraries for the subsets, it does not > > influence optimization possibilities by implementing everything in a > > monolithic block (i.e., central progress). > > > > > - Both blocking & nonblocking belong to the core > > > - Torsten > > > - Some collectives may go into selectable subsets > > I see three subsets: blocking colls, non-blocking colls and > topological > > colls (maybe also blocking / non-blocking). > > > > > - MPI_ANY_SOURCE considered harmful > > I'd like to add datatypes and heterogeneity to this list (with regards > > to performance). Alexander mentioned the dynamics. I think we should > > have a lit of items ready that could influence optimization > > possibilities significanty if they were to be announced by the user > > before he can use them. That would give another strong argument for > the > > subsetting. > > > > Best, > > Torsten > > > > -- > > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > > Indiana University | http://www.indiana.edu > > Open Systems Lab | http://osl.iu.edu/ > > 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA > > Lindley Hall Room 135 | +01 (812) 855-3608 > > _______________________________________________ > > Mpi3-subsetting mailing list > > Mpi3-subsetting_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > > --------------------------------------------------------------------- > > Intel GmbH > > Dornacher Strasse 1 > > 85622 Feldkirchen/Muenchen Germany > > Sitz der Gesellschaft: Feldkirchen bei Muenchen > > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > > VAT Registration No.: DE129385895 > > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > > > This e-mail and any attachments may contain confidential material for > > the sole use of the intended recipient(s). Any review or distribution > > by others is strictly prohibited. If you are not the intended > > recipient, please contact the sender and delete all copies. > > > > > > _______________________________________________ > > Mpi3-subsetting mailing list > > Mpi3-subsetting_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > > > _______________________________________________ > Mpi3-subsetting mailing list > Mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > >

1 0

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

Woops... Chunks of bytes may have holes. Discard that argument. Homogeneous installations don't need data transformation, but this is a different matter. -----Original Message----- From: Supalov, Alexander Sent: Friday, February 29, 2008 6:11 AM To: 'Bronis R. de Supinski'; 'mpi3-subsetting_at_[hidden]' Subject: RE: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09 Hi, Thanks. I think the main thrust here is the library footprint (no pack/unpack, etc.) and complexity of the user side of the datatype interface, rather than performance. Many applications just don't need any of this, and never will. Why not translating this application non-requirement into a minimum MPI subset? Same with communicator/group management, etc. Moreover, homogeneous installations that dominate HPC now don't actually need any datatype support at all. They send chunks of bytes. This may change in the future, though. A minor performance implication is that without holes that are only possible with derived datatypes, one does not need to track this, split the critical path, and make special provisions inside the MPI device layer to handle iov or such. The NIC capability argument is interesting, but it turns the discussion on its head: we're not after motivating network vendors to provide scatter/gather in hardware here, are we? Please clarify. Best regards. Alexander -----Original Message----- From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Bronis R. de Supinski Sent: Friday, February 29, 2008 5:53 AM To: mpi3-subsetting_at_[hidden] Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09 All: OK, I have to respond to the notion that derived datatypes limit performance. It is just not a reasonable position. Sure, if you can send contiguous locations, you will get higher performance. The problem is that codes do not only need to send contiguous data so that is not an adequate reason to say derived datatypes limit performance. So, what is left? That there is some more efficient way to send non-contiguous data? How? As multiple messages, each of which send contiguous data? If so, then the implementation could do this under the covers and the datatypes are just a convenience for the user not to have to specify the individual sends. OK, suppose that's not the reason. Perhaps the user can do the copying into a contiguous buffer and get better performance? While I have seen this hold with some implementations, it is absurd. There is no reason that I can discern as to why the user should be able to deduce a better copying mechanism than the MPI implementer. So, again, at worst, the datatypes should be a convenience. Do you have an alternative reason or a refutation of these opinions? What is more important, it is certainly possible to build scatter/gather support into a NIC and achieve better performance with datatypes than without. While there are issues to be resolved for that (primarily the issue of pinning memory), they are solvable with the right hardware mechanism. Just because it does not yet exist is not an adequate reason to say "Get rid of datatypes". OK, you are not saying that but you are saying to deprecate them in a sense. And saying you could send contiguous sends more efficiently is a bad argument here. How do datatypes cause inefficiency for that? How much is that cost really? At what point do you hit where the answer is "It would be faster not to compute anything"? Bronis On Fri, 29 Feb 2008, Supalov, Alexander wrote: > Hi, > > Thanks. What subsets inside the current standard would you propose? What > interfaces between them would you envision? > > Good idea about the optimization opportunities. Here's an initial > combined list, with the main benefits as I see them. Please > comment/extend. > > - Dynamic process support: less overhead in the progress engine, easier > global rank handling. > - Heterogeneity: better memory footprint, easier data handling. > - Derived datatypes (especially those with holes): better memory > footprint. > - MPI_ANY_SOURCE: faster, more simple multifabric progress. > - File I/O: smaller requests, easier wait/test functions. > - One-sided ops: no passive target w/o MPI calls - no extra progress > thread. > - Communicator & group management: better memory footprint. > - Message tagging: better support for stable dataflow exchanges, smaller > packets. > - Non-blocking communication: easier ordering, simplified request > handling. > > Best regards. > > Alexander > > -----Original Message----- > From: mpi3-subsetting-bounces_at_[hidden] > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of > Torsten Hoefler > Sent: Friday, February 29, 2008 5:08 AM > To: mpi3-subsetting_at_[hidden] > Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon > ww09 > > Hi, > > Present: Leonid Meyerguz (Microsoft), Rich Graham (ORNL), Richard > Barrett > > (ORNL), Torsten Hoefler (ISU), Alexander Supalov (Intel) > just for the record, it's "IU" not "ISU" :-) > > > - Scope of the effort > > - Rich > > - Minimum subset consistent with the rest of MPI, for > > performance/memory footprint optimization > > - Danger of splitting MPI, hence against optional features in > the > > standard > I back that (danger of optional features for portability). I'd propose > to split the current standard into mostly self-contained subsets that > have clearly defined interfaces to the rest of the standard. Note: this > only defines logical interfaces, that does *not* define how those things > are to be implemented. This makes it easier to understand the standard > and have separate (portable) libraries for the subsets, it does not > influence optimization possibilities by implementing everything in a > monolithic block (i.e., central progress). > > > - Both blocking & nonblocking belong to the core > > - Torsten > > - Some collectives may go into selectable subsets > I see three subsets: blocking colls, non-blocking colls and topological > colls (maybe also blocking / non-blocking). > > > - MPI_ANY_SOURCE considered harmful > I'd like to add datatypes and heterogeneity to this list (with regards > to performance). Alexander mentioned the dynamics. I think we should > have a lit of items ready that could influence optimization > possibilities significanty if they were to be announced by the user > before he can use them. That would give another strong argument for the > subsetting. > > Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Indiana University | http://www.indiana.edu > Open Systems Lab | http://osl.iu.edu/ > 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA > Lindley Hall Room 135 | +01 (812) 855-3608 > _______________________________________________ > Mpi3-subsetting mailing list > Mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > Mpi3-subsetting mailing list > Mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > _______________________________________________ Mpi3-subsetting mailing list Mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

Hi, Thanks. I think the main thrust here is the library footprint (no pack/unpack, etc.) and complexity of the user side of the datatype interface, rather than performance. Many applications just don't need any of this, and never will. Why not translating this application non-requirement into a minimum MPI subset? Same with communicator/group management, etc. Moreover, homogeneous installations that dominate HPC now don't actually need any datatype support at all. They send chunks of bytes. This may change in the future, though. A minor performance implication is that without holes that are only possible with derived datatypes, one does not need to track this, split the critical path, and make special provisions inside the MPI device layer to handle iov or such. The NIC capability argument is interesting, but it turns the discussion on its head: we're not after motivating network vendors to provide scatter/gather in hardware here, are we? Please clarify. Best regards. Alexander -----Original Message----- From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Bronis R. de Supinski Sent: Friday, February 29, 2008 5:53 AM To: mpi3-subsetting_at_[hidden] Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09 All: OK, I have to respond to the notion that derived datatypes limit performance. It is just not a reasonable position. Sure, if you can send contiguous locations, you will get higher performance. The problem is that codes do not only need to send contiguous data so that is not an adequate reason to say derived datatypes limit performance. So, what is left? That there is some more efficient way to send non-contiguous data? How? As multiple messages, each of which send contiguous data? If so, then the implementation could do this under the covers and the datatypes are just a convenience for the user not to have to specify the individual sends. OK, suppose that's not the reason. Perhaps the user can do the copying into a contiguous buffer and get better performance? While I have seen this hold with some implementations, it is absurd. There is no reason that I can discern as to why the user should be able to deduce a better copying mechanism than the MPI implementer. So, again, at worst, the datatypes should be a convenience. Do you have an alternative reason or a refutation of these opinions? What is more important, it is certainly possible to build scatter/gather support into a NIC and achieve better performance with datatypes than without. While there are issues to be resolved for that (primarily the issue of pinning memory), they are solvable with the right hardware mechanism. Just because it does not yet exist is not an adequate reason to say "Get rid of datatypes". OK, you are not saying that but you are saying to deprecate them in a sense. And saying you could send contiguous sends more efficiently is a bad argument here. How do datatypes cause inefficiency for that? How much is that cost really? At what point do you hit where the answer is "It would be faster not to compute anything"? Bronis On Fri, 29 Feb 2008, Supalov, Alexander wrote: > Hi, > > Thanks. What subsets inside the current standard would you propose? What > interfaces between them would you envision? > > Good idea about the optimization opportunities. Here's an initial > combined list, with the main benefits as I see them. Please > comment/extend. > > - Dynamic process support: less overhead in the progress engine, easier > global rank handling. > - Heterogeneity: better memory footprint, easier data handling. > - Derived datatypes (especially those with holes): better memory > footprint. > - MPI_ANY_SOURCE: faster, more simple multifabric progress. > - File I/O: smaller requests, easier wait/test functions. > - One-sided ops: no passive target w/o MPI calls - no extra progress > thread. > - Communicator & group management: better memory footprint. > - Message tagging: better support for stable dataflow exchanges, smaller > packets. > - Non-blocking communication: easier ordering, simplified request > handling. > > Best regards. > > Alexander > > -----Original Message----- > From: mpi3-subsetting-bounces_at_[hidden] > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of > Torsten Hoefler > Sent: Friday, February 29, 2008 5:08 AM > To: mpi3-subsetting_at_[hidden] > Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon > ww09 > > Hi, > > Present: Leonid Meyerguz (Microsoft), Rich Graham (ORNL), Richard > Barrett > > (ORNL), Torsten Hoefler (ISU), Alexander Supalov (Intel) > just for the record, it's "IU" not "ISU" :-) > > > - Scope of the effort > > - Rich > > - Minimum subset consistent with the rest of MPI, for > > performance/memory footprint optimization > > - Danger of splitting MPI, hence against optional features in > the > > standard > I back that (danger of optional features for portability). I'd propose > to split the current standard into mostly self-contained subsets that > have clearly defined interfaces to the rest of the standard. Note: this > only defines logical interfaces, that does *not* define how those things > are to be implemented. This makes it easier to understand the standard > and have separate (portable) libraries for the subsets, it does not > influence optimization possibilities by implementing everything in a > monolithic block (i.e., central progress). > > > - Both blocking & nonblocking belong to the core > > - Torsten > > - Some collectives may go into selectable subsets > I see three subsets: blocking colls, non-blocking colls and topological > colls (maybe also blocking / non-blocking). > > > - MPI_ANY_SOURCE considered harmful > I'd like to add datatypes and heterogeneity to this list (with regards > to performance). Alexander mentioned the dynamics. I think we should > have a lit of items ready that could influence optimization > possibilities significanty if they were to be announced by the user > before he can use them. That would give another strong argument for the > subsetting. > > Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Indiana University | http://www.indiana.edu > Open Systems Lab | http://osl.iu.edu/ > 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA > Lindley Hall Room 135 | +01 (812) 855-3608 > _______________________________________________ > Mpi3-subsetting mailing list > Mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > Mpi3-subsetting mailing list > Mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > _______________________________________________ Mpi3-subsetting mailing list Mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

Hi, Thanks. As soon as there's a couple of non-blocking recvs out there, waiting for them in reverse order requires tracking of the moment when the receives were posted. In some cases this leads to extra fields and data exchanges. The footprint argument generally says that the library will be smaller. This may be a minor matter for general purpose computers, but as soon as you go to Petascale, you need every byte on the compute nodes for user data, especially if dynamic libraries are not supported. As for the collectives, many are implemented using SendRecv, and that blocking call in turn often uses non-blocking communication. Classic Alltoallv algorithm uses nonblocking calls, too. So, I'm not sure that even unoptimized blocking collectives will always use only blocking pt2pt. Best regards. Alexander -----Original Message----- From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Torsten Hoefler Sent: Friday, February 29, 2008 5:44 AM To: mpi3-subsetting_at_[hidden] Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09 Hi Alexander, > Thanks. What subsets inside the current standard would you propose? > What interfaces between them would you envision? that is a long discussion, I guess. So just to put something up for discussion: One subset could be collective communication and it would use Send/Recv from the MPI-core interface. Same for non-blockong colls (using nonblocking send/recv). Again, this is a logical design, it enables us to easily implement a portable library that only uses this interface and offers the standardized features. This library can be imported by vendors who do not want to optimize the substet that is supported by the lib. However, the MPI implementor is free to ignore the interface and do the collectives inside the library in a monolithic way (for performance). Other subsets could be: - topology functions - language bindings (certainly needs discussion) - data-type handling - groups/communicator handling (interface definition would be complex) - profiling interface (similar to language bindings) - parallel I/O - process management - one-sided (if this is not in core) - grequests > Good idea about the optimization opportunities. Here's an initial > combined list, with the main benefits as I see them. Please > comment/extend. > > - Dynamic process support: less overhead in the progress engine, easier > global rank handling. ack > - Heterogeneity: better memory footprint, easier data handling. easier equals faster in this case > - Derived datatypes (especially those with holes): better memory > footprint. hmm, I don't get the memory footprint argument? But I'd say that it simplifies the critical path (one if less) and many applications just don't need datatypes. This is necessary if we want to broaden our scope (cf. the sockets interface has no datatypes and works well) > - MPI_ANY_SOURCE: faster, more simple multifabric progress. ack + receiver-based protocols (I wrote about this in "Optimizing non-blocking Collective Operations for InfiniBand" will be presented at the CAC workshop at IPDPS'07. > - File I/O: smaller requests, easier wait/test functions. yes > - One-sided ops: no passive target w/o MPI calls - no extra progress > thread. > - Communicator & group management: better memory footprint. > - Message tagging: better support for stable dataflow exchanges, smaller > packets. ack > - Non-blocking communication: easier ordering, simplified request > handling. I am not sure about this since only the local matching differs (slightly) here, i.e., packets match a waiting recv (potentially dozens of them in different threads) vs. packets match a non-blocking request. This is pretty much the same overhead. How does that influence MPIs ordering constraints? Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Indiana University | http://www.indiana.edu Open Systems Lab | http://osl.iu.edu/ 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA Lindley Hall Room 135 | +01 (812) 855-3608 _______________________________________________ Mpi3-subsetting mailing list Mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

All: OK, I have to respond to the notion that derived datatypes limit performance. It is just not a reasonable position. Sure, if you can send contiguous locations, you will get higher performance. The problem is that codes do not only need to send contiguous data so that is not an adequate reason to say derived datatypes limit performance. So, what is left? That there is some more efficient way to send non-contiguous data? How? As multiple messages, each of which send contiguous data? If so, then the implementation could do this under the covers and the datatypes are just a convenience for the user not to have to specify the individual sends. OK, suppose that's not the reason. Perhaps the user can do the copying into a contiguous buffer and get better performance? While I have seen this hold with some implementations, it is absurd. There is no reason that I can discern as to why the user should be able to deduce a better copying mechanism than the MPI implementer. So, again, at worst, the datatypes should be a convenience. Do you have an alternative reason or a refutation of these opinions? What is more important, it is certainly possible to build scatter/gather support into a NIC and achieve better performance with datatypes than without. While there are issues to be resolved for that (primarily the issue of pinning memory), they are solvable with the right hardware mechanism. Just because it does not yet exist is not an adequate reason to say "Get rid of datatypes". OK, you are not saying that but you are saying to deprecate them in a sense. And saying you could send contiguous sends more efficiently is a bad argument here. How do datatypes cause inefficiency for that? How much is that cost really? At what point do you hit where the answer is "It would be faster not to compute anything"? Bronis On Fri, 29 Feb 2008, Supalov, Alexander wrote: > Hi, > > Thanks. What subsets inside the current standard would you propose? What > interfaces between them would you envision? > > Good idea about the optimization opportunities. Here's an initial > combined list, with the main benefits as I see them. Please > comment/extend. > > - Dynamic process support: less overhead in the progress engine, easier > global rank handling. > - Heterogeneity: better memory footprint, easier data handling. > - Derived datatypes (especially those with holes): better memory > footprint. > - MPI_ANY_SOURCE: faster, more simple multifabric progress. > - File I/O: smaller requests, easier wait/test functions. > - One-sided ops: no passive target w/o MPI calls - no extra progress > thread. > - Communicator & group management: better memory footprint. > - Message tagging: better support for stable dataflow exchanges, smaller > packets. > - Non-blocking communication: easier ordering, simplified request > handling. > > Best regards. > > Alexander > > -----Original Message----- > From: mpi3-subsetting-bounces_at_[hidden] > [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of > Torsten Hoefler > Sent: Friday, February 29, 2008 5:08 AM > To: mpi3-subsetting_at_[hidden] > Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon > ww09 > > Hi, > > Present: Leonid Meyerguz (Microsoft), Rich Graham (ORNL), Richard > Barrett > > (ORNL), Torsten Hoefler (ISU), Alexander Supalov (Intel) > just for the record, it's "IU" not "ISU" :-) > > > - Scope of the effort > > - Rich > > - Minimum subset consistent with the rest of MPI, for > > performance/memory footprint optimization > > - Danger of splitting MPI, hence against optional features in > the > > standard > I back that (danger of optional features for portability). I'd propose > to split the current standard into mostly self-contained subsets that > have clearly defined interfaces to the rest of the standard. Note: this > only defines logical interfaces, that does *not* define how those things > are to be implemented. This makes it easier to understand the standard > and have separate (portable) libraries for the subsets, it does not > influence optimization possibilities by implementing everything in a > monolithic block (i.e., central progress). > > > - Both blocking & nonblocking belong to the core > > - Torsten > > - Some collectives may go into selectable subsets > I see three subsets: blocking colls, non-blocking colls and topological > colls (maybe also blocking / non-blocking). > > > - MPI_ANY_SOURCE considered harmful > I'd like to add datatypes and heterogeneity to this list (with regards > to performance). Alexander mentioned the dynamics. I think we should > have a lit of items ready that could influence optimization > possibilities significanty if they were to be announced by the user > before he can use them. That would give another strong argument for the > subsetting. > > Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Indiana University | http://www.indiana.edu > Open Systems Lab | http://osl.iu.edu/ > 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA > Lindley Hall Room 135 | +01 (812) 855-3608 > _______________________________________________ > Mpi3-subsetting mailing list > Mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > Mpi3-subsetting mailing list > Mpi3-subsetting_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting >

1 0

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

Hi Alexander, > Thanks. What subsets inside the current standard would you propose? > What interfaces between them would you envision? that is a long discussion, I guess. So just to put something up for discussion: One subset could be collective communication and it would use Send/Recv from the MPI-core interface. Same for non-blockong colls (using nonblocking send/recv). Again, this is a logical design, it enables us to easily implement a portable library that only uses this interface and offers the standardized features. This library can be imported by vendors who do not want to optimize the substet that is supported by the lib. However, the MPI implementor is free to ignore the interface and do the collectives inside the library in a monolithic way (for performance). Other subsets could be: - topology functions - language bindings (certainly needs discussion) - data-type handling - groups/communicator handling (interface definition would be complex) - profiling interface (similar to language bindings) - parallel I/O - process management - one-sided (if this is not in core) - grequests > Good idea about the optimization opportunities. Here's an initial > combined list, with the main benefits as I see them. Please > comment/extend. > > - Dynamic process support: less overhead in the progress engine, easier > global rank handling. ack > - Heterogeneity: better memory footprint, easier data handling. easier equals faster in this case > - Derived datatypes (especially those with holes): better memory > footprint. hmm, I don't get the memory footprint argument? But I'd say that it simplifies the critical path (one if less) and many applications just don't need datatypes. This is necessary if we want to broaden our scope (cf. the sockets interface has no datatypes and works well) > - MPI_ANY_SOURCE: faster, more simple multifabric progress. ack + receiver-based protocols (I wrote about this in "Optimizing non-blocking Collective Operations for InfiniBand" will be presented at the CAC workshop at IPDPS'07. > - File I/O: smaller requests, easier wait/test functions. yes > - One-sided ops: no passive target w/o MPI calls - no extra progress > thread. > - Communicator & group management: better memory footprint. > - Message tagging: better support for stable dataflow exchanges, smaller > packets. ack > - Non-blocking communication: easier ordering, simplified request > handling. I am not sure about this since only the local matching differs (slightly) here, i.e., packets match a waiting recv (potentially dozens of them in different threads) vs. packets match a non-blocking request. This is pretty much the same overhead. How does that influence MPIs ordering constraints? Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Indiana University | http://www.indiana.edu Open Systems Lab | http://osl.iu.edu/ 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

Hi, Thanks. What subsets inside the current standard would you propose? What interfaces between them would you envision? Good idea about the optimization opportunities. Here's an initial combined list, with the main benefits as I see them. Please comment/extend. - Dynamic process support: less overhead in the progress engine, easier global rank handling. - Heterogeneity: better memory footprint, easier data handling. - Derived datatypes (especially those with holes): better memory footprint. - MPI_ANY_SOURCE: faster, more simple multifabric progress. - File I/O: smaller requests, easier wait/test functions. - One-sided ops: no passive target w/o MPI calls - no extra progress thread. - Communicator & group management: better memory footprint. - Message tagging: better support for stable dataflow exchanges, smaller packets. - Non-blocking communication: easier ordering, simplified request handling. Best regards. Alexander -----Original Message----- From: mpi3-subsetting-bounces_at_[hidden] [mailto:mpi3-subsetting-bounces_at_[hidden]] On Behalf Of Torsten Hoefler Sent: Friday, February 29, 2008 5:08 AM To: mpi3-subsetting_at_[hidden] Subject: Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09 Hi, > Present: Leonid Meyerguz (Microsoft), Rich Graham (ORNL), Richard Barrett > (ORNL), Torsten Hoefler (ISU), Alexander Supalov (Intel) just for the record, it's "IU" not "ISU" :-) > - Scope of the effort > - Rich > - Minimum subset consistent with the rest of MPI, for > performance/memory footprint optimization > - Danger of splitting MPI, hence against optional features in the > standard I back that (danger of optional features for portability). I'd propose to split the current standard into mostly self-contained subsets that have clearly defined interfaces to the rest of the standard. Note: this only defines logical interfaces, that does *not* define how those things are to be implemented. This makes it easier to understand the standard and have separate (portable) libraries for the subsets, it does not influence optimization possibilities by implementing everything in a monolithic block (i.e., central progress). > - Both blocking & nonblocking belong to the core > - Torsten > - Some collectives may go into selectable subsets I see three subsets: blocking colls, non-blocking colls and topological colls (maybe also blocking / non-blocking). > - MPI_ANY_SOURCE considered harmful I'd like to add datatypes and heterogeneity to this list (with regards to performance). Alexander mentioned the dynamics. I think we should have a lit of items ready that could influence optimization possibilities significanty if they were to be announced by the user before he can use them. That would give another strong argument for the subsetting. Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Indiana University | http://www.indiana.edu Open Systems Lab | http://osl.iu.edu/ 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA Lindley Hall Room 135 | +01 (812) 855-3608 _______________________________________________ Mpi3-subsetting mailing list Mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

Hi, > Present: Leonid Meyerguz (Microsoft), Rich Graham (ORNL), Richard Barrett > (ORNL), Torsten Hoefler (ISU), Alexander Supalov (Intel) just for the record, it's "IU" not "ISU" :-) > - Scope of the effort > - Rich > - Minimum subset consistent with the rest of MPI, for > performance/memory footprint optimization > - Danger of splitting MPI, hence against optional features in the > standard I back that (danger of optional features for portability). I'd propose to split the current standard into mostly self-contained subsets that have clearly defined interfaces to the rest of the standard. Note: this only defines logical interfaces, that does *not* define how those things are to be implemented. This makes it easier to understand the standard and have separate (portable) libraries for the subsets, it does not influence optimization possibilities by implementing everything in a monolithic block (i.e., central progress). > - Both blocking & nonblocking belong to the core > - Torsten > - Some collectives may go into selectable subsets I see three subsets: blocking colls, non-blocking colls and topological colls (maybe also blocking / non-blocking). > - MPI_ANY_SOURCE considered harmful I'd like to add datatypes and heterogeneity to this list (with regards to performance). Alexander mentioned the dynamics. I think we should have a lit of items ready that could influence optimization possibilities significanty if they were to be announced by the user before he can use them. That would give another strong argument for the subsetting. Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Indiana University | http://www.indiana.edu Open Systems Lab | http://osl.iu.edu/ 150 S. Woodlawn Ave. | Bloomington, IN, 474045-7104 | USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [Mpi3-subsetting] agenda for subsetting kickoff telecon ww09
by unknown＠example.com 28 Feb '08

28 Feb '08

Hi everybody, Thank you for your time today. It was a very good discussion. Here's what I captured (please add/modify what I may have missed): Present: Leonid Meyerguz (Microsoft), Rich Graham (ORNL), Richard Barrett (ORNL), Torsten Hoefler (ISU), Alexander Supalov (Intel) - Opens & introductions - Scope of the effort - Rich - Minimum subset consistent with the rest of MPI, for performance/memory footprint optimization - Danger of splitting MPI, hence against optional features in the standard - Both blocking & nonblocking belong to the core - Torsten - Some collectives may go into selectable subsets - MPI_ANY_SOURCE considered harmful - Leonid - Flexible support for optional features, means for choosing and advertising level of compliance/set of features - See enclosed email for Alexander's POV - General discussion snapshots - Support of subsets: some or all? If some, possible linkage problems in static apps (or dead calls). If all, where's the gain? - Optional: really optional (may be not present) or selectable (are present but may be unused)? - Performance penalty for unused subsets: implementation matter or standard choice? - Portability may be limited to certain class of applications (think FT, master-slave runs) - All we design needs to be implementable, complexity needs to be controlled - An ability to use certain set of subsets should not preclude pulling in other modules if necessary - Whatever we do, it should not conflict with the ABI efforts - Need to stay nice and be nicer wrt to the libraries (think threading) and keep things simple - The simplification argument, if put first, may not be liked by some - Next steps - Please comment on these minutes, and add/modify what I may have missed - I'll prepare a couple of slides by next week summarizing our discussion so far; again, your feedback will be most welcome - At the meeting, it may be great to meet F2F briefly and discuss any eventual loose ends before the presentation at the Forum; I'll see to this Best regards. Alexander -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. <strong>attached mail follows:</strong> Hi everybody, In the run-up to our kick-off, here's my POV on the subsetting and its possible scope/role in the MPI-3. Your comments and suggestions are most welcome. We start this activity because: 1) Certain industrial customers complain about MPI complexity and inadequacy 2) Complexity is going to grow in MPI-3 3) Growing complexity may have growing performance implications As a result of the above, customers drift away from the MPI to home-grown libraries, usually based on sockets. This effectively eliminates fast networks from their scope, unless they can profit from fast IP emulation layers. Moreover, this customer drift, if continued, may make MPI irrelevant in some HPC areas and lead to creation of alternative interfaces there. The main purpose of the Forum, as well as the subsetting WG, is thus to react to customer demand and make MPI faster and easier to use, especially in those areas that are subjected to the increasing customer drift (think, e.g., massive master-slave computations). Basing on these premises, the subsetting, in my mind, should try to: 1) Make MPI standard modular. This may include: a) Splitting the standard functionality into coherent groups that users will be able to select/deselect at init time b) Making implementation of some modules/functionality optional (think dynamic process support) as they are anyway now c) Addressing not only functional groups but also certain aspects of the standard that may not be needed in certain use cases (think communicator management, message tagging, derived datatypes, MPI_ANY_SOURCE support, non-blocking communication, etc.) 2) As part of the modularization, optionally identify the minimum functional MPI subset. This may be: a) Those 6 calls (Init, Rank, Size, Send, Recv, Finalize), possibly w/o communicator management and derived datatypes. b) A more flexible combination of modules actually needed by the user 3) Connect subsetting with other MPI-3 activities (FT, ABI, collectives, etc.) As a result of modularization, we should strive to achieve 1) Simplification of the standard for the newcomers 2) Performance advantages for reasonable module combinations 3) Influence upon the overall shape of the MPI-3 standard There are certainly quite a few concerns here: 1) We may end up complicating the standard and its implementation even further 2) We may facilitate a split of the standard into several mutually incompatible implementation "families" 3) We may cause some valid MPI-3 applications break if they use optional modules not available in the implementation involved 4) We may get carried away by academic considerations and miss the actual customer demands in the process 5) We may be engaging into a lost battle because the MPI standard is way too rigid by design/purpose to be simplified To guard against all this, we need to work closely with other WGs and the Forum as a whole, define our goals as early as possible, and solicit extensive Forum and customer feedback. >From all this, by the time of the Forum meeting in March, we should have at least a couple of slides reflecting our intentions and plans. Best regards. Alexander -- Dr Alexander Supalov Intel GmbH Hermuelheimer Strasse 8a 50321 Bruehl, Germany Phone: +49 2232 209034 Mobile: +49 173 511 8735 Fax: +49 2232 209029 *

1 0