- Mpi-22 - mailman3.cels.anl.gov

Re: [Mpi-22] Ticket #33 (Scalable Graph Topology Interface)
by unknown＠example.com 18 Jun '09

18 Jun '09

On Jun 18, 2009, at 3:47 AM, Supalov, Alexander wrote: > I wonder whether we may want to consider defining a prototype more > strictly, say, by adding a couple of qualifiers like "demonstrating > the intended functionality and performance potential" or something. > This will definitely increase the quality of the prototypes, and may > positively influence the quality of the resulting standard, which is > after all what all of us are after here. Given that most of the > intended MPI-2.2 tickets already appear to have good prototypes, it > should not be a problem to clarify the rules a bit here. > Actually, it would be better to make these "ad hoc" rules more definitive. Torsten is the latest in the series of people who have been bitten by thinking that they were acting by the rules but then someone said, "no, the rule is actually ..." Having somewhat subjective rules is fine, but having nebulous, not- written-down, sometimes-seemingly-arbitrary rules is what is causing a problem. How about someone making a shot at writing down exactly what the rules are? Leaving leeway for some subjectivity is fine (and good), but at leaving having *something* to point to when issues come under debate like this would be most helpful -- and will save a lot of time during the MPI-3 process. -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi-22] Ticket #33 (Scalable Graph Topology Interface)
by unknown＠example.com 18 Jun '09

18 Jun '09

Hi everybody, Thanks, having a good prototype is definitely a great step forward. I hope that if we happen to have a couple of other MPI-2.2 tickets, the prototypes in which could be improved in a similar manner, their authors may find it possible to do so now. I wonder whether we may want to consider defining a prototype more strictly, say, by adding a couple of qualifiers like "demonstrating the intended functionality and performance potential" or something. This will definitely increase the quality of the prototypes, and may positively influence the quality of the resulting standard, which is after all what all of us are after here. Given that most of the intended MPI-2.2 tickets already appear to have good prototypes, it should not be a problem to clarify the rules a bit here. As was mentioned back in April, to facilitate commercial adoption of the MPI-2.2, it might also be good to resolve the ABI and IP matters at the same time, in application to this ticket and all other actual and potential MPI-2.2 entries. Even though it looks fairly unlikely that an addition of an extra class proposed in #33 will break anything in the C++ ABI, it just might. Hence, we need a determination by the Forum what kind of ABI changes is acceptable for MPI-2.2, possibly individually for each language binding, and what kind of procedure should be used to check the ABI backward compatibility where it is required - like testing, consultation with compiler writers, etc. A reasonable compromise wrt individual language bindings may help to ensure that the scope of the MPI-2.2 standard stays unchanged. The IP matter is basically a question of a license the prototype is being made available under. To simplify commercial adoption, it would be great to have the prototypes carry a BSD-style or an equally "commercial friendly" open source license. This may require some clarification on part of the authors with their respective institution, but by my records, the matter was in rather satisfactory state back in April. So, it should be no problem to make a slight change to the rules here either. I guess we should consider returning to these thee topics (prototype, ABI, IP) at the July meeting and making a decision there. Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Wednesday, June 17, 2009 6:02 PM To: MPI 2.2 Subject: Re: [Mpi-22] Ticket #33 (Scalable Graph Topology Interface) Torsten -- Many thanks for your persistence in this ticket. From your description, it sounds like the implementation should be sufficient for many of those who abstained or voted against because it now does show tangible scalability improvements. All: please chime in if you feel this is not the case. #33 was perhaps the most "troublesome" ticket, so reaching consensus here would be great. Thanks! On Jun 17, 2009, at 11:59 AM, Torsten Hoefler wrote: > Hello, > everybody who abstained or voted against the proposal, please read > ticket #33 and discuss any open issue or question either here or > with me. > I would be happy to answer any questions or address any doubts. > > A new implementation for all versions of the proposed scalable graph > interface is attached to ticket #33 (virtual_graph_scal.cpp) > (https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/33) > > This version does not implement the new interface on top of the old > interface and thus reduces the memory consumption per process from > O(N*N) to O(N) (assuming a reasonable specification). > > The implementation is trivial (took me about 2 hours) and the most > complex function has 76 lines of code. The whole proposal including an > example has 199 lines of code (many of this are interface > definitions). > > As discussed, the communication overhead for the adjacent interface is > zero and the worst-case communication overhead for the general > interface > is O(N) per process, however, the expected case is O(1) (constant). > > The attached implementation now achieves all goals of ticket #33, it > increases the scalability of the graph interface by reducing the > memory > on the user-side to a lower complexity class and the communication is > optimized. > > Ths significance of improvement is shown in the following table which > lists the maximum memory consumption in bytes for different > communicator > sizes with the old and the new interface (assuming 8 byte integers). > > processes MPI-2.1 interface proposed interface > 512 2 MiB 2 kiB > 1024 8 MiB 8 kiB > 4096 134 MiB 32 kiB > 16384 2.1 GiB 131 kiB > 65536 34.4 GiB 524 kiB > > This is an upper bound, if you are interested in sparse graphs, > multiply > both numbers by 0.01 (1% sparsity). The relative scaling remains the > same. > > All issues related to process remapping are not part of the ticket and > are not touched (the main intent of this ticket is to enable the use > of > virtual graph topologies at scale at all). However, if you are > interested in this matter, there is an extensive body of research in > this area: > > "Rank Reordering Strategy for MPI Topology Creation Functions" > http://www.springerlink.com/content/f3hqq5u2v14tpu53/ > "Topology mapping for Blue Gene/L supercomputer" > http://portal.acm.org/citation.cfm?id=1188576 > "Implementing the MPI process topology mechanism" > http://portal.acm.org/citation.cfm?id=762761.762767 > "Scheduling regular and irregular communication patterns on the CM-5" > http://portal.acm.org/citation.cfm?id=147877.148034 > > Thanks & All the Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Torsten Hoefler, Ph.D. | Postdoctoral Fellow > Open Systems Lab | Indiana University > 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA > Lindley Hall Room 135 | +01 (812) 855-3608 > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 -- Jeff Squyres Cisco Systems _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Re: [Mpi-22] Ticket #33 (Scalable Graph Topology Interface)
by unknown＠example.com 17 Jun '09

17 Jun '09

Torsten -- Many thanks for your persistence in this ticket. From your description, it sounds like the implementation should be sufficient for many of those who abstained or voted against because it now does show tangible scalability improvements. All: please chime in if you feel this is not the case. #33 was perhaps the most "troublesome" ticket, so reaching consensus here would be great. Thanks! On Jun 17, 2009, at 11:59 AM, Torsten Hoefler wrote: > Hello, > everybody who abstained or voted against the proposal, please read > ticket #33 and discuss any open issue or question either here or > with me. > I would be happy to answer any questions or address any doubts. > > A new implementation for all versions of the proposed scalable graph > interface is attached to ticket #33 (virtual_graph_scal.cpp) > (https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/33) > > This version does not implement the new interface on top of the old > interface and thus reduces the memory consumption per process from > O(N*N) to O(N) (assuming a reasonable specification). > > The implementation is trivial (took me about 2 hours) and the most > complex function has 76 lines of code. The whole proposal including an > example has 199 lines of code (many of this are interface > definitions). > > As discussed, the communication overhead for the adjacent interface is > zero and the worst-case communication overhead for the general > interface > is O(N) per process, however, the expected case is O(1) (constant). > > The attached implementation now achieves all goals of ticket #33, it > increases the scalability of the graph interface by reducing the > memory > on the user-side to a lower complexity class and the communication is > optimized. > > Ths significance of improvement is shown in the following table which > lists the maximum memory consumption in bytes for different > communicator > sizes with the old and the new interface (assuming 8 byte integers). > > processes MPI-2.1 interface proposed interface > 512 2 MiB 2 kiB > 1024 8 MiB 8 kiB > 4096 134 MiB 32 kiB > 16384 2.1 GiB 131 kiB > 65536 34.4 GiB 524 kiB > > This is an upper bound, if you are interested in sparse graphs, > multiply > both numbers by 0.01 (1% sparsity). The relative scaling remains the > same. > > All issues related to process remapping are not part of the ticket and > are not touched (the main intent of this ticket is to enable the use > of > virtual graph topologies at scale at all). However, if you are > interested in this matter, there is an extensive body of research in > this area: > > "Rank Reordering Strategy for MPI Topology Creation Functions" > http://www.springerlink.com/content/f3hqq5u2v14tpu53/ > "Topology mapping for Blue Gene/L supercomputer" > http://portal.acm.org/citation.cfm?id=1188576 > "Implementing the MPI process topology mechanism" > http://portal.acm.org/citation.cfm?id=762761.762767 > "Scheduling regular and irregular communication patterns on the CM-5" > http://portal.acm.org/citation.cfm?id=147877.148034 > > Thanks & All the Best, > Torsten > > -- > bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- > Torsten Hoefler, Ph.D. | Postdoctoral Fellow > Open Systems Lab | Indiana University > 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA > Lindley Hall Room 135 | +01 (812) 855-3608 > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 -- Jeff Squyres Cisco Systems

1 0

Ticket #33 (Scalable Graph Topology Interface)
by unknown＠example.com 17 Jun '09

17 Jun '09

Hello, everybody who abstained or voted against the proposal, please read ticket #33 and discuss any open issue or question either here or with me. I would be happy to answer any questions or address any doubts. A new implementation for all versions of the proposed scalable graph interface is attached to ticket #33 (virtual_graph_scal.cpp) (https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/33) This version does not implement the new interface on top of the old interface and thus reduces the memory consumption per process from O(N*N) to O(N) (assuming a reasonable specification). The implementation is trivial (took me about 2 hours) and the most complex function has 76 lines of code. The whole proposal including an example has 199 lines of code (many of this are interface definitions). As discussed, the communication overhead for the adjacent interface is zero and the worst-case communication overhead for the general interface is O(N) per process, however, the expected case is O(1) (constant). The attached implementation now achieves all goals of ticket #33, it increases the scalability of the graph interface by reducing the memory on the user-side to a lower complexity class and the communication is optimized. Ths significance of improvement is shown in the following table which lists the maximum memory consumption in bytes for different communicator sizes with the old and the new interface (assuming 8 byte integers). processes MPI-2.1 interface proposed interface 512 2 MiB 2 kiB 1024 8 MiB 8 kiB 4096 134 MiB 32 kiB 16384 2.1 GiB 131 kiB 65536 34.4 GiB 524 kiB This is an upper bound, if you are interested in sparse graphs, multiply both numbers by 0.01 (1% sparsity). The relative scaling remains the same. All issues related to process remapping are not part of the ticket and are not touched (the main intent of this ticket is to enable the use of virtual graph topologies at scale at all). However, if you are interested in this matter, there is an extensive body of research in this area: "Rank Reordering Strategy for MPI Topology Creation Functions" http://www.springerlink.com/content/f3hqq5u2v14tpu53/ "Topology mapping for Blue Gene/L supercomputer" http://portal.acm.org/citation.cfm?id=1188576 "Implementing the MPI process topology mechanism" http://portal.acm.org/citation.cfm?id=762761.762767 "Scheduling regular and irregular communication patterns on the CM-5" http://portal.acm.org/citation.cfm?id=147877.148034 Thanks & All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler, Ph.D. | Postdoctoral Fellow Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608

1 0

Re: [Mpi-22] Authors now on tickets
by unknown＠example.com 16 Jun '09

16 Jun '09

Thanks, Jeff. I'm in charge of T&C. Bill On Jun 16, 2009, at 2:22 PM, Jeff Squyres wrote: > There are now checkboxes for the relevant chapter authors on the Trac > tickets. > > You can use these fields in Trac custom queries to find tickets that > are in your chapter (for example). > > Who is in charge of the Terms and Conventions chapter? On https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MpiTwoTwoWikiPage > , I see the following list, but it does not include T&C: > > Responsible Chapter Authors > > * Bill Gropp, Frontmatter, Introduction, and Bibliography > * Richard Graham, Point-to-Point Communications > * Adam Moody amd Torsten Hoefler, Collective Communication > * Richard Treumann, Groups, Contexts, and Communicators > * Jesper Larsson Traeff, Process Topologies, Info-Object, and One- > Sided Communications > * George Bosilca, Environmental Management > * David Solt, Process Creation and Management > * Bronis de Supinski, External Interfaces, and Profiling > * Rajeev Thakur, I/O > * Jeff Squyres, Language Bindings > * Rolf Rabenseifner, Deprecated Functions, and Annex Change-Log > * Alexander Supalov, Annex Language bindings > > -- > Jeff Squyres > Cisco Systems > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Authors now on tickets
by unknown＠example.com 16 Jun '09

16 Jun '09

There are now checkboxes for the relevant chapter authors on the Trac tickets. You can use these fields in Trac custom queries to find tickets that are in your chapter (for example). Who is in charge of the Terms and Conventions chapter? On https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/MpiTwoTwoWikiPage , I see the following list, but it does not include T&C: Responsible Chapter Authors * Bill Gropp, Frontmatter, Introduction, and Bibliography * Richard Graham, Point-to-Point Communications * Adam Moody amd Torsten Hoefler, Collective Communication * Richard Treumann, Groups, Contexts, and Communicators * Jesper Larsson Traeff, Process Topologies, Info-Object, and One- Sided Communications * George Bosilca, Environmental Management * David Solt, Process Creation and Management * Bronis de Supinski, External Interfaces, and Profiling * Rajeev Thakur, I/O * Jeff Squyres, Language Bindings * Rolf Rabenseifner, Deprecated Functions, and Annex Change-Log * Alexander Supalov, Annex Language bindings -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi-22] SVN committing MPI-2.2 latex
by unknown＠example.com 16 Jun '09

16 Jun '09

On Jun 16, 2009, at 3:04 PM, William Gropp wrote: > I would like the author check boxes in addition to the new states; > that makes it easier to keep track of who needs to be poked :) > Ok; I'll go add them. > And FWIW, I'd also prefer to have a single data source, but I need one > that will give me a quick overview of the state of the world (and > would do so without giving me constant tracebacks). There's still > info in my spreadsheet that isn't easily extracted from Trac. > Fair enough. -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi-22] SVN committing MPI-2.2 latex
by unknown＠example.com 16 Jun '09

16 Jun '09

I would like the author check boxes in addition to the new states; that makes it easier to keep track of who needs to be poked :) And FWIW, I'd also prefer to have a single data source, but I need one that will give me a quick overview of the state of the world (and would do so without giving me constant tracebacks). There's still info in my spreadsheet that isn't easily extracted from Trac. Bill On Jun 16, 2009, at 1:50 PM, Jeff Squyres wrote: > On Jun 13, 2009, at 3:53 PM, William Gropp wrote: > >> I prefer 2 (check boxes for each chapter author); the diagram is >> fine. I'm keeping a separate spreadsheet because it takes a *lot* >> less time to access the overall data that way. >> > > I'm confused. I added the new states ("waiting for PDF reviews" / > "ticket complete") and renamed the other two ("waiting for proposal > reviews" / "proposal reviewed"), but do you want me to add the author > checkboxes or not? > > FWIW: I think it'll be confusing if two different data systems are > used to track the tickets (trac + a spreadsheet). > > -- > Jeff Squyres > Cisco Systems > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0

Re: [Mpi-22] SVN committing MPI-2.2 latex
by unknown＠example.com 16 Jun '09

16 Jun '09

On Jun 13, 2009, at 3:53 PM, William Gropp wrote: > I prefer 2 (check boxes for each chapter author); the diagram is > fine. I'm keeping a separate spreadsheet because it takes a *lot* > less time to access the overall data that way. > I'm confused. I added the new states ("waiting for PDF reviews" / "ticket complete") and renamed the other two ("waiting for proposal reviews" / "proposal reviewed"), but do you want me to add the author checkboxes or not? FWIW: I think it'll be confusing if two different data systems are used to track the tickets (trac + a spreadsheet). -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi-22] SVN committing MPI-2.2 latex
by unknown＠example.com 13 Jun '09

13 Jun '09

I prefer 2 (check boxes for each chapter author); the diagram is fine. I'm keeping a separate spreadsheet because it takes a *lot* less time to access the overall data that way. Bill On Jun 12, 2009, at 11:21 AM, Jeff Squyres wrote: > Ok. Let's get a few more approvals of this, and if everyone likes it, > I'll add the extra states to Trac. > > On Jun 12, 2009, at 9:17 AM, Rolf Rabenseifner wrote: > >> Jeff, >> For me, it is now perfect. >> I hope that all chapter authors are lucky with it. >> Rolf >> >> On Fri, 12 Jun 2009 08:47:48 -0700 >> Jeff Squyres <jsquyres_at_[hidden]> wrote: >>> Ok, try this diagram. >>> >>> On Jun 12, 2009, at 5:53 AM, Rolf Rabenseifner wrote: >>> >>>> Aswers below >>>> >>>> On Fri, 12 Jun 2009 05:09:51 -0700 >>>> Jeff Squyres <jsquyres_at_[hidden]> wrote: >>>>> On Jun 11, 2009, at 11:20 AM, Rolf Rabenseifner wrote: >>>>> >>>>>> In your state diagram, you may differentiate between ticket >>>> author >>>>>> and chapter author. Please can you use a new color for the >>>> chapter- >>>>>> author trasitions. >>>>>> >>>>> >>>>> Can do. >>>>> >>>>>> If a reviewer detects a bug, he should set the state back to >>>>>> "Passed". A new color for the reviewer is also needed for this >>>>>> transition. >>>>>> >>>>> >>>>> I think that this is what I was thinking when I had an extra >> state >>>>> in my first diagram -- that the states would go between "text >>>>> committed" and "waiting for PDF reviews" until it was finally >>>>> "ticket complete". >>>>> >>>>> So do you want to make the reviewers responsible for changing the >>>>> state back away from "waiting for PDF reviews" to the previous >>>>> state? That would be different than what we did for proposals, >>>>> where the proposal author or chair would move the ticket between >>>>> "forum feedback requested", "not ready", and "waiting for >>>> (proposal) >>>>> reviews". >>>> >>>> Yes, because we have three parties: The ticket author who has >> only a >>>> controling role, the reviewers, and the chapter author. >>>> It must be absolutely sure, that the chapter author savely detects >>>> when he has to repair something. Therefore I would like that >>>> the status goes back to "Passed". This should be done by the >>>> reviewer >>>> as soon as he detects a problem. This reduces the risk, that >>>> something >>>> is overseen. >>>> >>>> The ticket author is doing only the final transition to "Ticket >>>> complete" >>>> after he and his 4 reviewers have checked the ticket-text in >>>> the MPI-2.2 standard. >>>> The ticket author may also be active, if the chapter author >>>> has not seen that a problem was detected and he has to repair >>>> something. >>>> >>>> Please, can you mention at the forward transition >>>> from "Passed" to "Waiting for PDF reviews": >>>> >>>> Set by a chapter author when all involved chapter authors >>>> have finished the MPI-2.2 latex text and produced a sniplet >>>> of pdf pages of this ticket. >>>> >>>> Please, can you mention at the backward transition >>>> from "Waiting for PDF reviews" to "Passed": >>>> >>>> Set back by a reviewer if he detects a bug in the pdf file. >>>> >>>> We must be sure, that all 85 tickets are handled correctly >>>> by the 13 chapter authors. >>>> >>>> Rolf >>>>> >>>>> -- >>>>> Jeff Squyres >>>>> Cisco Systems >>>>> >>>>> _______________________________________________ >>>>> mpi-22 mailing list >>>>> mpi-22_at_[hidden] >>>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >>>> >>>> Dr. Rolf Rabenseifner . . . . . . . . . .. email >>>> rabenseifner_at_[hidden] >>>> High Performance Computing Center (HLRS) . phone >>>> ++49(0)711/685-65530 >>>> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / >>>> 685-65832 >>>> Head of Dpmt Parallel Computing . . . >>>> www.hlrs.de/people/rabenseifner >>>> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) >>>> _______________________________________________ >>>> mpi-22 mailing list >>>> mpi-22_at_[hidden] >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >>>> >>> >>> >>> -- >>> Jeff Squyres >>> Cisco Systems >> >> Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner_at_[hidden] >> High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 >> University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 >> Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner >> Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30) >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >> > > > -- > Jeff Squyres > Cisco Systems > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 William Gropp Deputy Director for Research Institute for Advanced Computing Applications and Technologies Paul and Cynthia Saylor Professor of Computer Science University of Illinois Urbana-Champaign

1 0