December 2011 - mpi-forum - mailman3.cels.anl.gov

Formal reading of Ticket 271 - get/set info on comm/win
by Moody, Adam T. 28 Dec '11

28 Dec '11

Hi All, I would like to prepare for a formal reading of ticket #271 (get/set info on comm/win) at the January meeting. This ticket had an informal reading at our last meeting, and there were no major objections. I've incorporated the suggestions made at that meeting into the ticket271_v2 pdf that is attached to the ticket. https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/271 https://svn.mpi-forum.org/trac/mpi-forum-web/attachment/ticket/271/ticket27… Happy Holidays! -Adam

1 0

Formal Reading of Ticket 273
by Mohamad Chaarawi 25 Dec '11

25 Dec '11

Hi All, We will have a formal reading of ticket #273 (immediate nonblocking collective I/O routines) at the January meeting. The draft is attached to the ticket: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/273 Changes are marked up. Any comment is welcome. Thanks and Happy Holidays! Mohamad

1 0

Re: [Mpi-forum] Per communicator parameterisation
by Torsten Hoefler 21 Dec '11

21 Dec '11

On Wed, Dec 21, 2011 at 09:05:49PM +0000, Martyn Foster wrote: > On 21 December 2011 18:14, Torsten Hoefler <htor(a)illinois.edu> wrote: > > You are correct and this is indeed a missing functionality in MPI-2.2. > > There is a proposal that allows to attach info objects during > > communicator duplication (see item #2 "MPI_COMM_DUP_INFO" in > > https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/271) This has been > > discussed in the Forum but I don't know the status of this ticket. Adam, > > can you comment? > <snip> > > Would this functionality suffice for your use-case? > > Hi Torsten, > > Yes, thanks. I think this is pretty much exactly what I was looking > for. The only comments that I'd make on the ticket proposal would > revolve around providing a method to do it in a single step (I suspect > for several things a "comm_set_info" would be equivalent to recreating > the communicator and no cheaper than the dup method). I can see the > API problem, and I wouldn't want to see a huge explosion in interfaces > - would it be better to be able to set the 'info' object to be used > for subsequent communicator creation as well as the "creation and then > modification" approach? e.g. > > CALL MPI_Comm_set_default_info(My_Info_Sets(colour)) ! Set default > info to be used based on my role. > CALL MPI_Comm_split(Comm_In,colour,key,Comm_Out) > CALL MPI_Comm_set_default_info(MPI_Info_Null) ! lets avoid accidental > deadlock in future comm creation calls. The problem with those things where you put state in the library is always thread-safety. We just eliminated the last occurrence of "library state" without explicit handles by introducing MPI_Mprobe and friends. I imagine that the Forum would be opposed to re-introduce such problems. We discussed this issue at depth and decided that the comm_dup approach is most appropriate for a number of reasons. The best reasons are probably (1) that it can be used with any communicator and (2) that it sets all infos at creation time, i.e., the library can do special one-time optimizations while the communicator is guaranteed to be "empty" (no outstanding ops yet). > It would also be nice to have a reserved set of info keys for the > things that are common to most/many MPI/HW implementations- protocol > thresholds etc, and a recommended namespacing for vendor specific > stuff... but maybe the scope of that is too broad. Yes, it would be, definitely, however, it is very tricky to define such generic info keys. The Forum tried to add such infos wherever possible but we are rather conservative (MPI lived 15 years so far and will live much longer, and it is hard to predict future architectures), i.e., most predefined info keys are only relative to the MPI interface, not to specific hardware implementations. You are most welcome to join our discussions and meetings! However, version 3.technically in a feature-freeze right now and we move new non-critical aspects to 3.1. All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Re: [Mpi-forum] Per communicator parameterisation
by Martyn Foster 21 Dec '11

21 Dec '11

On 21 December 2011 18:14, Torsten Hoefler <htor(a)illinois.edu> wrote: > You are correct and this is indeed a missing functionality in MPI-2.2. > There is a proposal that allows to attach info objects during > communicator duplication (see item #2 "MPI_COMM_DUP_INFO" in > https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/271) This has been > discussed in the Forum but I don't know the status of this ticket. Adam, > can you comment? <snip> > Would this functionality suffice for your use-case? Hi Torsten, Yes, thanks. I think this is pretty much exactly what I was looking for. The only comments that I'd make on the ticket proposal would revolve around providing a method to do it in a single step (I suspect for several things a "comm_set_info" would be equivalent to recreating the communicator and no cheaper than the dup method). I can see the API problem, and I wouldn't want to see a huge explosion in interfaces - would it be better to be able to set the 'info' object to be used for subsequent communicator creation as well as the "creation and then modification" approach? e.g. CALL MPI_Comm_set_default_info(My_Info_Sets(colour)) ! Set default info to be used based on my role. CALL MPI_Comm_split(Comm_In,colour,key,Comm_Out) CALL MPI_Comm_set_default_info(MPI_Info_Null) ! lets avoid accidental deadlock in future comm creation calls. It would also be nice to have a reserved set of info keys for the things that are common to most/many MPI/HW implementations- protocol thresholds etc, and a recommended namespacing for vendor specific stuff... but maybe the scope of that is too broad. - Martyn

1 0

Re: [Mpi-forum] Jan MPI Forum meeting
by Adam T. Moody 21 Dec '11

21 Dec '11

Hi Rich, I'll need to 15 minutes in our Jan meeting for a formal reading of ticket 271 https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/271 -Adam

1 0

Re: [Mpi-forum] Per communicator parameterisation
by Adam T. Moody 21 Dec '11

21 Dec '11

Torsten Hoefler wrote: >Dear Martyn, > >Thanks for bringing this up, the Forum appreciates any user input and >comments. See my answers below. > > > >>Firstly, apologies if this is the wrong place to post - I couldn't see >>a particularly relevant subgroup or forum for discussion, so if this >>is the wrong place, redirection is more than welcome.... >> >> >The right place would probably be the collectives and topology working >group, but I think the general forum list is ok as long as the thread >does not grow too much. > > > >>[snip] >> >>As it stands there is no practical way of setting this up - even with >>the generous support of my vendor. This is in stark contrast to MPI-IO >>where I have data structures available to pass either vendor specific >>or generic hints allowing me to modify behaviour on a per-file basis, >>in application, at runtime. I really want to be able to perform a >>similar thing when I create communicators, such that when I perform, >>say, a comm_split, I can create unique characteristics for each of the >>new communicators. >> >> >You are correct and this is indeed a missing functionality in MPI-2.2. >There is a proposal that allows to attach info objects during >communicator duplication (see item #2 "MPI_COMM_DUP_INFO" in >https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/271) This has been >discussed in the Forum but I don't know the status of this ticket. Adam, >can you comment? > > Hi Martyn, Yes, the plan is to enable applications to apply info hints to communicators at creation time. We didn't want to duplicate all of the communicator creation calls, so instead we decided to add a single MPI_COMM_DUP_INFO call that takes an info object and applies it to the dup'd communication as it is created. To use this technique, you'd have to create a temporary communicator through an existing creation call, then create a copy of it with the info applied via COMM_DUP_INFO, and finally free the temporary. Of course, you'd also need your MPI vendor to provide relevant hints. Except for some minor wording changes, this ticket was generally well received by the forum, and the plan is to get it accepted with MPI 3.0. -Adam > > >>So I guess the first question is has any work been taking place in >>this area, and if not, how do I go about spurring the discussion? >> >> >Yes, Adam and I worked on this issue a while ago and Adam created the >referenced ticket. Would this functionality suffice for your use-case? > >All the Best, > Torsten > > >

1 0

Re: [Mpi-forum] Per communicator parameterisation
by Torsten Hoefler 21 Dec '11

21 Dec '11

Dear Martyn, Thanks for bringing this up, the Forum appreciates any user input and comments. See my answers below. > Firstly, apologies if this is the wrong place to post - I couldn't see > a particularly relevant subgroup or forum for discussion, so if this > is the wrong place, redirection is more than welcome.... The right place would probably be the collectives and topology working group, but I think the general forum list is ok as long as the thread does not grow too much. > [snip] > > As it stands there is no practical way of setting this up - even with > the generous support of my vendor. This is in stark contrast to MPI-IO > where I have data structures available to pass either vendor specific > or generic hints allowing me to modify behaviour on a per-file basis, > in application, at runtime. I really want to be able to perform a > similar thing when I create communicators, such that when I perform, > say, a comm_split, I can create unique characteristics for each of the > new communicators. You are correct and this is indeed a missing functionality in MPI-2.2. There is a proposal that allows to attach info objects during communicator duplication (see item #2 "MPI_COMM_DUP_INFO" in https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/271) This has been discussed in the Forum but I don't know the status of this ticket. Adam, can you comment? > So I guess the first question is has any work been taking place in > this area, and if not, how do I go about spurring the discussion? Yes, Adam and I worked on this issue a while ago and Adam created the referenced ticket. Would this functionality suffice for your use-case? All the Best, Torsten -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Performance Modeling and Simulation Lead Blue Waters Directorate | University of Illinois (UIUC) 1205 W Clark Street | Urbana, IL, 61801 NCSA Building | +01 (217) 244-7736

1 0

Per communicator parameterisation
by Martyn Foster 21 Dec '11

21 Dec '11

Dear all, Firstly, apologies if this is the wrong place to post - I couldn't see a particularly relevant subgroup or forum for discussion, so if this is the wrong place, redirection is more than welcome.... I have a program. Actually my program is three programs coupled together. The entire application lives in MPI_COMM_WORLD, and each of the components operates mainly within its own private communicator. Another three communicators exist that handle the coupling between each pair of subcomponents. I run my program on a machine - it has some special features that allow me to alter how MPI behaves - I can take advantage of different protocols for point to point messaging, I can elect to use hardware acceleration for collectives, and various other things that radically change how MPI behaves and performs. I can switch these on and off, say, via the environment and via my MPI launcher program. For the purposes of tuning, I can isolate the three applications as well as the inter component communication patterns - I discover that the parametrisation needed for each to be optimal is rather different. Tuning for the whole ensemble leads to none of the individual parts performing optimally. After discussion with my MPI vendor, it rapidly became apparent that there was no way of articulating from outside the application that I wanted different (as yet uncreated) communicators on different task sets to behave differently in anything remotely approaching a sensible fashion - the communicators are completely opaque, and there is no way to reference them! Here's a simple example to illustrate < ---------- COMM_WORLD ---------------> < main app > ï¿½ ï¿½ ï¿½ < post processing > --- long messages one way--> My machine has, say, two p2p messaging modes, low latency driven by the CPU, and high latency offloaded to supporting hardware. The main app is tightly synchronised and benefits from the low latency mode even though it consumes CPU cycles. Communications between the main app and the post processor is unidirectional and slow delivery does not hurt overall performance (but involving the CPU in dispatching them does). Optimal performance is acheived by setting comm_world to use the high latency offload mode, and comm_main_app to use the low latency CPU mode. An even simpler example would be a monolithic application, where optimal settings are not apparent until part way through execution - i.e. data dependent. As it stands there is no practical way of setting this up - even with the generous support of my vendor. This is in stark contrast to MPI-IO where I have data structures available to pass either vendor specific or generic hints allowing me to modify behaviour on a per-file basis, in application, at runtime. I really want to be able to perform a similar thing when I create communicators, such that when I perform, say, a comm_split, I can create unique characteristics for each of the new communicators. So I guess the first question is has any work been taking place in this area, and if not, how do I go about spurring the discussion? Best regards, Martyn

1 0

Draft for FT Run-Through Stabilization available / Ticket #276
by Josh Hursey 20 Dec '11

20 Dec '11

A new draft of the Run-Through Stabilization Fault Tolerance proposal is attached to the ticket: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/276 There will be a plenary session on Monday, Jan. 9 during which we will provide an overview of the proposal. This will be followed by a first reading on Wednesday, Jan. 11 for this proposal. Thanks to everyone that has provided feedback so far. We look forward to additional discussion at the next meeting. Josh -- Joshua Hursey Postdoctoral Research Associate Oak Ridge National Laboratory http://users.nccs.gov/~jjhursey

1 0

Formal Reading Ticket #280 (MPI_Type_create_hindexed_block
by Mohamad Chaarawi 19 Dec '11

19 Dec '11

Hi All, A formal reading of ticket #280 is planned for the January meeting. I have uploaded a pdf of the proposal to the ticket: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/280 All changes are marked up and can be found by the usual search for ticket280. All comments are welcome! Thanks, Mohamad

1 0