May 2008 - Mpi-22 - mailman3.cels.anl.gov

Re: [Mpi-22] MPI_INIT_ASSERTED proposal for MPI-2.2
by unknown＠example.com 21 May '08

21 May '08

On Fri, 2008-05-16 at 14:51 -0400, Richard Treumann wrote: > MPI_NO_REDUCTION_DETERMINISM > > The MPI standard either requires or strongly urges that an > MPI_REDUCE/MPI_ALLREDUCE give exactly the > same answer every time. There are applications that do not need a > "same answer" promise as long as > they can assume they will get a "correct" answer. Maybe they can be > provided a faster reduction algorithm. This makes zero sense as a hint and would make the standard contradictory if adopted. I've already made it clear I would prefer assertions and this option would be useful in that context, if hints turn out the be chosen over assertions then this flag should be dropped. Clarification of the standard over the results of floating point reduction results would probably be a good thing anyway, for example is it OK for the different communicators of the same size but with different process layouts to give different answers given the same input data? I suspect the standard forbids this although without it it's probably impossible to do any shared memory optimisations. Ashley Pittman.

1 0

MPI_INIT_ASSERTED proposal for MPI-2.2
by unknown＠example.com 16 May '08

16 May '08

Proposal for MPI Initialization time assertions in MPI 2.2 Define two new MPI functions: MPI_INIT_ASSERTED(assertions, threads, provided) [ IN assertions ] assertions about application behavior [ IN threads] desired level of thread support (integer) [ OUT provided] provided level of thread support (integer) int MPI_Init_asserted(int *argc, char *((*argv)[]), int assertions, int threads, int *provided) MPI_INIT_ASSERTED(ASSERTIONS, THREADS, PROVIDED, IERROR) INTEGER ASSERTIONS, THREADS, PROVIDED, IERROR (I will need help for C++ bindings ) The assertions arguments is to be provided as the logical OR of any subset of the assertions defined by the MPI standard. Zero is valid for an application that does not wish to make any assertion. The assertions are constants defined in mpi.h. MPI_QUERY_ASSERTIONS(stated, exploited) [ OUT stated] assertions as give in the MPI_INIT_ASSERTED call [ OUT exploited ] assertions the MPI implementation is exploiting int MPI_Query_assertions(int *stated, int *exploited) MPI_QUERY_ASSERTIONS(STATED, EXPLOITED, IERROR) INTEGER STATED, EXPLOITED, IERROR The outputs, STATED and EXPLOITED, can be checked for specific assertions with logical masks. For example to see if the application has used MPI_NO_SEND_CANCEL, library code could use: MPI_Query_assertions(&stated, &exploited); if (stated & MPI_NO_SEND_CANCEL) .... To confirm that no assertions are being exploited the library could use if (exploited) ..... error ==================================================================== Rationale There are certain semantic guarantees or features required by the MPI standard that can add significant memory overheads or performance costs. If an application does not depend on one of these features and can declare that fact at MPI initialization, the MPI implementation may be able to provide better performance. Assertions on MPI_INIT_ASSERTED let an application declare it does not require a specific feature and allow the implementation to omit support for that feature if there is a savings from doing so. When an application calls MPI_INIT_ASSERTED with a particular assertion the MPI implementation may depend on the application to act as the assertion promises. The behavior of an MPI application that uses or depends on some feature of MPI after asserting it does not is undefined. The application cannot depend on the assertion being exploited. The MPI application must be valid for an MPI implementation that ignores the assertion. Advise to implementors - When it is practical, a high quality MPI implementation will raise an error if an application violates an assertion that was provided at initialization It will not be practical for the MPI implementation to detect all assertion violations. End advise to implementors - The following assertions are available (defined in mpi.h) MPI_NO_EAGER_THROTTLE The MPI standard allows an implementation to send some messages eagerly, meaning the message can complete at the sender before a matching receive is posted. The implementation is required to hold the message data until the matching receive gets posted. If there is no space to buffer the data the send must block. The MPI job must not fail because it has run out of space or lose the data. Providing this guarantee can demand an expensive flow control. In many well structured applications there will never be a need to buffer significant numbers of eager messages but eager sends do help performance. These applications can tell the MPI implementation the guarantee is optional. MPI_NO_REDUCTION_DETERMINISM The MPI standard either requires or strongly urges that an MPI_REDUCE/MPI_ALLREDUCE give exactly the same answer every time. There are applications that do not need a "same answer" promise as long as they can assume they will get a "correct" answer. Maybe they can be provided a faster reduction algorithm. MPI_NO_DATATYPE_HETEROGENEITY MPI_Datatypes that are made from multiple types can be optimized if it is known there will never be a need to translate the data between heterogeneous nodes. A structure made of an MPI_INT followed by an MPI_FLOAT is likely to be 8 contiguous bytes with an extent of 8. An MPI implementation that "knows" it will not need to deal with data conversion can simplify the datatype commit logic and resulting internal representation by discarding the MPI_INT/MPI_FLOAT distinctions and just recording that the type is 8 bytes with a stride of 8. ============================== If we add MPI_INIT_ASSERTED to the MPI standard, MPI_INIT and MPI_INIT_THREAD continue to work as before. MPI_INIT_THREAD is equivalent to MPI_INIT_ASSERTED with zero as the assertion. We can consider deprecating the older initialization calls. Above are the assertions I have decided make most sense for the first pass. There are others that may be justified but a rationale is needed. If MPI 2.2 defines support for the assertion above we can consider others as part of MPI 3.0, Below are some That may be justified but will need a better rationale. MPI_NO_THREAD_CONTENTION (I am not sure this different from asking for MPI_THREAD_SERIALIZED but here is what it would mean) If the MPI library may be called by contending threads it must do something to protect internal state from conflicting updates. Whether this involves mutex locks or some other scheme, it is likely to have a performance cost. This assertion promises that the application will not make conflicting MPI calls. The MPI implementation can bypass any logic that defends state against multiple concurrent calls to MPI. The application may be multi-threaded in other aspects. MPI_NO_SEND_CANCEL To support MPI_CANCEL of a send, the MPI implementation may need to add extra information to every message header. If the application can assert that it does not try to cancel any sends then perhaps less state must be maintained and smaller message headers can be used. MPI_NO_ANY_SOURCE Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 *

1 0

Re: [Mpi-22] MPI_INIT assertions
by unknown＠example.com 16 May '08

16 May '08

Thank you Ashley - This is exactly the distinction between assertions and hints and the reason this narrow proposal advocates assertions and does not advocate hints. An MPI_NO_SEND_CANCEL hint still requires the MPI implementation to invest whatever it takes, across the entire application life to do MPI_CANCEL correctly. An MPI_NO_SEND_CANCEL assertion allows the MPI implementation to forgo those costs and just refuse to try to MPI_CANCEL if the application has an MPI_CANCEL call. The assertion does relax a semantic guarentee (I say relax rather than change because "change" has broader implications but an assertion does change semantics in a narrow and well defined way). An MPI Implementation that is prepared to do MPI_CANCEL correctly has a different semantic than one that will issue an error message if MPI_CANCEL is called. As a footnote - MPI 2 defined some hints and MPI 3 may define more hints. That is OK. There are cases where a hint will allow an MPI implementation to give better performance without changing its semantic. In a situation where a hint is enough, the MPI standard should avoid defining an assertion. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/15/2008 05:59:51 PM: <snip> > > To me I see little benefit in offering MPI_NO_SEND_CANCEL as a hint, all > that does is say to the library that a cancel is unlikely to happen and > we all know that anyway. As a cancel is still possible I see no > opportunity for optimisation of any kind making itself possible, all the > extra space in headers and longer code paths would still be needed > anyway. > > As an assertion it would offer some scope for library writers to make > simplifications leading to better performance, as a hint I can't see how > it would change anything. > > Ashley, > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 *

1 0

Re: [Mpi-22] MPI_INIT assertions
by unknown＠example.com 15 May '08

15 May '08

On Thu, 2008-05-15 at 17:44 -0400, Richard Treumann wrote: > > The MPI_NO_SEND_CANCEL is only useful for cases where libmpi must > devote extra memory, longer code paths or extra space in message > headers across an entire run just to have the information it needs to > do MPI_CANCEL correctly is somebody happens to call it. If a > particular MPI implementation does not incur extra costs to be ready > for a potential MPI_CANCEL then the implementation will ignore this > assertion. To me I see little benefit in offering MPI_NO_SEND_CANCEL as a hint, all that does is say to the library that a cancel is unlikely to happen and we all know that anyway. As a cancel is still possible I see no opportunity for optimisation of any kind making itself possible, all the extra space in headers and longer code paths would still be needed anyway. As an assertion it would offer some scope for library writers to make simplifications leading to better performance, as a hint I can't see how it would change anything. Ashley,

1 0

Re: [Mpi-22] MPI_INIT assertions
by unknown＠example.com 15 May '08

15 May '08

I think we are splitting hairs on wording here. In one case the user is saying ³I don¹t care about the performance of cancel² and in the other ³I promise not to call cancel². Neither is changing the semantics of MPI. Rich On 5/15/08 5:44 PM, "Richard Treumann" <treumann_at_[hidden]> wrote: > Hi Rich > > The difference between a hint and an assertion is that an hint cannot change > semantic in any way. An assertion will allow libmpi to relax a specific > semantic guarantee. Lets take a simple case: > > The application wants to say it will not try to cancel any MPI_ISEND so it > passes MPI_NO_SEND_CANCEL to MPI_INIT > > If MPI_INIT were passed MPI_NO_SEND_CANCEL as a hint and then the application > went ahead and called MPI_CANCEL anyway, it would be fine for the MPI > implementation to use a much slower MPI_CANCEL algorithm than if the hint had > not been given. It would not be OK for the implementation to hang or raise an > error or cancel the wrong message. > > If MPI_Init were passed MPI_NO_SEND_CANCEL as an assertion and then the > application went ahead and called MPI_CANCEL anyway, it would be OK for the > MPI implementation to raise an error for the MPI_CANCEL call. > > Every MPI Implementation must still be prepared to do MPI_CANCEL of an ISEND > correctly if MPI_CANCEL is called by an application that did not provide the > assertion. Adding assertion support in MPI 2.2 cannot break a correct MPI 2.1 > application. > > The MPI_NO_SEND_CANCEL is only useful for cases where libmpi must devote extra > memory, longer code paths or extra space in message headers across an entire > run just to have the information it needs to do MPI_CANCEL correctly is > somebody happens to call it. If a particular MPI implementation does not incur > extra costs to be ready for a potential MPI_CANCEL then the implementation > will ignore this assertion. > > Dick > > > > Dick Treumann - MPI Team/TCEM > IBM Systems & Technology Group > Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 05/15/2008 01:33:02 PM: > >> > Assertions should not change semantics  these are defined by the >> > standard. Assertions >> > may provide ³help² to the implementation, what ever that means. I >> > may be missing some thing, >> > but I really don¹t see a difference (aside from name) between >> > assertions an hints  both are >> > ways of the user conveying information to the library. >> > >> > Rich >> > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 *

1 0

Re: [Mpi-22] MPI_INIT assertions
by unknown＠example.com 15 May '08

15 May '08

Hi Rich The difference between a hint and an assertion is that an hint cannot change semantic in any way. An assertion will allow libmpi to relax a specific semantic guarantee. Lets take a simple case: The application wants to say it will not try to cancel any MPI_ISEND so it passes MPI_NO_SEND_CANCEL to MPI_INIT If MPI_INIT were passed MPI_NO_SEND_CANCEL as a hint and then the application went ahead and called MPI_CANCEL anyway, it would be fine for the MPI implementation to use a much slower MPI_CANCEL algorithm than if the hint had not been given. It would not be OK for the implementation to hang or raise an error or cancel the wrong message. If MPI_Init were passed MPI_NO_SEND_CANCEL as an assertion and then the application went ahead and called MPI_CANCEL anyway, it would be OK for the MPI implementation to raise an error for the MPI_CANCEL call. Every MPI Implementation must still be prepared to do MPI_CANCEL of an ISEND correctly if MPI_CANCEL is called by an application that did not provide the assertion. Adding assertion support in MPI 2.2 cannot break a correct MPI 2.1 application. The MPI_NO_SEND_CANCEL is only useful for cases where libmpi must devote extra memory, longer code paths or extra space in message headers across an entire run just to have the information it needs to do MPI_CANCEL correctly is somebody happens to call it. If a particular MPI implementation does not incur extra costs to be ready for a potential MPI_CANCEL then the implementation will ignore this assertion. Dick Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/15/2008 01:33:02 PM: > Assertions should not change semantics â these are defined by the > standard. Assertions > may provide âhelpâ to the implementation, what ever that means. I > may be missing some thing, > but I really donât see a difference (aside from name) between > assertions an hints â both are > ways of the user conveying information to the library. > > Rich > *

1 0

Re: [Mpi-22] MPI_INIT assertions
by unknown＠example.com 15 May '08

15 May '08

Assertions should not change semantics  these are defined by the standard. Assertions may provide ³help² to the implementation, what ever that means. I may be missing some thing, but I really don¹t see a difference (aside from name) between assertions an hints  both are ways of the user conveying information to the library. Rich On 5/15/08 12:58 PM, "Terry Jones" <trj_at_[hidden]> wrote: > > If we are to allow an extensible mechanism for Assertions, then I like the > idea of a "Hints/Info" like interface where one may query to find out what the > implementation supports. However, an Assertions version would need to be > clearly distinct from Hints to adequately convey that Assertions *may change > the semantic rules* (which Hints do not). > > An Extensible Assertions interface would presumably allow for many > optimizations by relaxing semantics that are unneeded by the application. > However, for Assertions the application must be aware because they can change > the outcome of the application. I mentioned read-ahead as an example. > Asserting a read-ahead of a given size would permit different results for > files opened Read+Write on a file system that provides strong consistency > semantics. > > I believe Extensible Assertions would be utilized by applications because as > application teams currently move from platform to platform they make > themselves aware of environment variable tunables and use such tunables. I > would like to see applications dynamically change these tunables during a run > to accommodate different application phases efficiently. Allowing such a > mechanism to alter semantics gives freedom for increased performance (albeit > at the cost that the application must be aware of what they are doing, and > there is added complexity for all involved). > > I would also support Dick's original thought of a small, pre-defined set of > Assertions. This is a small step in the direction of Extensible Assertions > (with smaller opportunity for misuse and 'easier to swallow' adoption), and > could be designed with an interface that would later permit Extensible > Assertions. But I would keep the interface sufficiently distinct from Hints to > avoid confusion. > > Regards, > Terry > > > At 11:25 AM -0400 5/15/08, Richard Graham wrote: >> We should be careful about making a change in MPI 2.2, knowing that we will >> likely turn around again >> in MPI 3.0 and change things again. If we are talking about changing the >> interface in 2.2, and >> then extending the assertions/hints in 3.0, this seems fine, but if we may >> want to change the >> interface yet again in 3.0, we should rethink things. >> I will add that if we are going to add some sort of "info" argument to >> 'MPI_Init()', we should deprecate >> MPI_Init() and MPI_Init_thread(), and include the threading specification in >> the "info" object. >> Finally, before we decide on how to pass hints/assertions (if we do) to >> MPI_Init(), we should >> define a consistent way across the standard for passing information between >> the application and >> the library, as this is not the only instance where this is useful, and a >> uniform way of doing this >> makes things much easier on users. >> >> Rich >> >> >> On 5/14/08 9:14 AM, "Richard Treumann" <treumann_at_[hidden]> wrote: >>> MPI_Init time assertions must be few and each must be valuable or the >>> concept will fall like a house of cards. >>> >>> There is nothing in my proposal for MPI_Init time assertions that rules out >>> providing other mechanisms in MPI 3 for giving guidance to the MPI >>> implementation. In MPI 3 we can consider more hints and we can add the >>> abiility to give stronger direction to MPI or provide it on a more granular >>> basis - If it makes sense. These extra mechanisms are far to complex to >>> consider as part of MPI 2.2. >>> >>> I would not use the phrase that Dries does when he says "Assertions are bad" >>> but I agree with the the sentiment behind his statement. I think there >>> should be a very small number of assertions defined in the standard and for >>> each there should be a good rationale. For MPI 2.2 there should be a great >>> rationale because we can come back in MPI 3 and add more assertions if we >>> miss some important ones. It is much harder to remove one that turns out to >>> be real trouble. >>> >>> Each assertion the the MPI Standard defines has the potential to break some >>> piece of code that is valid MPI but that depends on semantic the assertion >>> says is optional. The author of the routine that calls MPI_Init has the >>> power to declare assertions and the authors of other parts of an applicaton >>> must either live within the rules or explicitly shield against them. >>> >>> For project teams that develop complete applications, the decision to use an >>> assertion belongs to the team, team leader or architect. If it is decided >>> that an application will use a specific assertion it is the team lead who >>> must make sure all developers understand the decision and write appropriate >>> code. All testing will be done with the assertion in place. >>> >>> For third part libraries, the only option is to either forbid all assertions >>> or explicitly pick some to allow. If there are 4 potential assertions, it >>> is not very hard to decide for each one - "Will the library tolerate it?". >>> If there are 50 assertions, library authors will seldom allow them all and >>> will be more tempted to just say "No assertions allowed" because making >>> judgements about each of 50 is too difficult. >>> >>> For Community developed code where people contribute source but are not >>> under direct control of an architect or team lead, reviewing each submission >>> for compliance with one or two assertions may be acceptable but reviewing >>> for 50 each time somebody contributes new code is not. >>> >>> >>> >>> Dick Treumann - MPI Team/TCEM >>> IBM Systems & Technology Group >>> Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >>> Tele (845) 433-7846 Fax (845) 433-8363 >>> >>> >>> mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: >>> >>>> > * Terry Jones <trj_at_[hidden]> [2008-05-13 15:19:07]: >>>> > >>>>> > > You can also imagine other possibilities to provide helpful context. For >>>>> > > instance, perhaps the user could provide Assertions that would help >>>>> MPI IO >>>>> > > with read-ahead prefetching or write-behind, or even meta-data >>>>> operations >>>>> > > (e.g., later I will be creating one file per MPI task). >>>> > >>>> > Things like read-ahead or write-behind clearly shouldn't be assertions >>>> but >>>> > hints. (And, probably 'one file per MPI task' too -- if this is still >>>> going to >>>> > be needed in 2 years) >>>> > >>>> > MPI already has hints that can capture some of the things mentioned >>>> > above. >>>> > >>>> > access_style: (read_once, write_once, read_mostly, write_mostly, >>>> sequential, >>>> > reverse_sequential, and random) >>>> > sequential -> this can easily be used to turn on read-ahead >>>> > IF THE MPI LIBRARY decides this is useful >>>> > >>>> > Assertions are bad -- they break compatibility -- and should only be >>>> > tolerated if they provide real benefits and if the same cannot be >>>> obtained >>>> > through existing mechanisms (hints, ...). >>>> > >>>> > In the examples mentioned, this is not the case. >>>> > >>>> > Dries >>>> > >>>> > [attachment "attia6gr.dat" deleted by Richard >>>> > Treumann/Poughkeepsie/IBM] >>>> _______________________________________________ >>>> > mpi-22 mailing list >>>> > mpi-22_at_[hidden] >>>> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >>>> <http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22> >>> >>> _______________________________________________ >>> mpi-22 mailing list >>> mpi-22_at_[hidden] >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 >>> <http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22> >> >> >> _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 *

1 0

Re: [Mpi-22] MPI_INIT assertions
by unknown＠example.com 15 May '08

15 May '08

Hi, I bet some setting changes (like eager protocol threshold) may need to be global, or at least include all processes involved in a loosely synchronous, i.e., collective manner. Is this a part of the hints proposal? If not, changes to the settings effective on one side won't be visible on the other side. Enter chaos. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Terry Jones Sent: Thursday, May 15, 2008 6:58 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI_INIT assertions If we are to allow an extensible mechanism for Assertions, then I like the idea of a "Hints/Info" like interface where one may query to find out what the implementation supports. However, an Assertions version would need to be clearly distinct from Hints to adequately convey that Assertions *may change the semantic rules* (which Hints do not). An Extensible Assertions interface would presumably allow for many optimizations by relaxing semantics that are unneeded by the application. However, for Assertions the application must be aware because they can change the outcome of the application. I mentioned read-ahead as an example. Asserting a read-ahead of a given size would permit different results for files opened Read+Write on a file system that provides strong consistency semantics. I believe Extensible Assertions would be utilized by applications because as application teams currently move from platform to platform they make themselves aware of environment variable tunables and use such tunables. I would like to see applications dynamically change these tunables during a run to accommodate different application phases efficiently. Allowing such a mechanism to alter semantics gives freedom for increased performance (albeit at the cost that the application must be aware of what they are doing, and there is added complexity for all involved). I would also support Dick's original thought of a small, pre-defined set of Assertions. This is a small step in the direction of Extensible Assertions (with smaller opportunity for misuse and 'easier to swallow' adoption), and could be designed with an interface that would later permit Extensible Assertions. But I would keep the interface sufficiently distinct from Hints to avoid confusion. Regards, Terry At 11:25 AM -0400 5/15/08, Richard Graham wrote: We should be careful about making a change in MPI 2.2, knowing that we will likely turn around again in MPI 3.0 and change things again. If we are talking about changing the interface in 2.2, and then extending the assertions/hints in 3.0, this seems fine, but if we may want to change the interface yet again in 3.0, we should rethink things. I will add that if we are going to add some sort of "info" argument to 'MPI_Init()', we should deprecate MPI_Init() and MPI_Init_thread(), and include the threading specification in the "info" object. Finally, before we decide on how to pass hints/assertions (if we do) to MPI_Init(), we should define a consistent way across the standard for passing information between the application and the library, as this is not the only instance where this is useful, and a uniform way of doing this makes things much easier on users. Rich On 5/14/08 9:14 AM, "Richard Treumann" <treumann_at_[hidden]> wrote: MPI_Init time assertions must be few and each must be valuable or the concept will fall like a house of cards. There is nothing in my proposal for MPI_Init time assertions that rules out providing other mechanisms in MPI 3 for giving guidance to the MPI implementation. In MPI 3 we can consider more hints and we can add the abiility to give stronger direction to MPI or provide it on a more granular basis - If it makes sense. These extra mechanisms are far to complex to consider as part of MPI 2.2. I would not use the phrase that Dries does when he says "Assertions are bad" but I agree with the the sentiment behind his statement. I think there should be a very small number of assertions defined in the standard and for each there should be a good rationale. For MPI 2.2 there should be a great rationale because we can come back in MPI 3 and add more assertions if we miss some important ones. It is much harder to remove one that turns out to be real trouble. Each assertion the the MPI Standard defines has the potential to break some piece of code that is valid MPI but that depends on semantic the assertion says is optional. The author of the routine that calls MPI_Init has the power to declare assertions and the authors of other parts of an applicaton must either live within the rules or explicitly shield against them. For project teams that develop complete applications, the decision to use an assertion belongs to the team, team leader or architect. If it is decided that an application will use a specific assertion it is the team lead who must make sure all developers understand the decision and write appropriate code. All testing will be done with the assertion in place. For third part libraries, the only option is to either forbid all assertions or explicitly pick some to allow. If there are 4 potential assertions, it is not very hard to decide for each one - "Will the library tolerate it?". If there are 50 assertions, library authors will seldom allow them all and will be more tempted to just say "No assertions allowed" because making judgements about each of 50 is too difficult. For Community developed code where people contribute source but are not under direct control of an architect or team lead, reviewing each submission for compliance with one or two assertions may be acceptable but reviewing for 50 each time somebody contributes new code is not. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: > * Terry Jones <trj_at_[hidden]> [2008-05-13 15:19:07]: > > > You can also imagine other possibilities to provide helpful context. For > > instance, perhaps the user could provide Assertions that would help MPI IO > > with read-ahead prefetching or write-behind, or even meta-data operations > > (e.g., later I will be creating one file per MPI task). > > Things like read-ahead or write-behind clearly shouldn't be assertions but > hints. (And, probably 'one file per MPI task' too -- if this is still going to > be needed in 2 years) > > MPI already has hints that can capture some of the things mentioned > above. > > access_style: (read_once, write_once, read_mostly, write_mostly, sequential, > reverse_sequential, and random) > sequential -> this can easily be used to turn on read-ahead > IF THE MPI LIBRARY decides this is useful > > Assertions are bad -- they break compatibility -- and should only be > tolerated if they provide real benefits and if the same cannot be obtained > through existing mechanisms (hints, ...). > > In the examples mentioned, this is not the case. > > Dries > > [attachment "attia6gr.dat" deleted by Richard > Treumann/Poughkeepsie/IBM] _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 <http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22> ________________________________ _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 <http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22> _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. *

1 0

Re: [Mpi-22] MPI_INIT assertions
by unknown＠example.com 15 May '08

15 May '08

If we are to allow an extensible mechanism for Assertions, then I like the idea of a "Hints/Info" like interface where one may query to find out what the implementation supports. However, an Assertions version would need to be clearly distinct from Hints to adequately convey that Assertions *may change the semantic rules* (which Hints do not). An Extensible Assertions interface would presumably allow for many optimizations by relaxing semantics that are unneeded by the application. However, for Assertions the application must be aware because they can change the outcome of the application. I mentioned read-ahead as an example. Asserting a read-ahead of a given size would permit different results for files opened Read+Write on a file system that provides strong consistency semantics. I believe Extensible Assertions would be utilized by applications because as application teams currently move from platform to platform they make themselves aware of environment variable tunables and use such tunables. I would like to see applications dynamically change these tunables during a run to accommodate different application phases efficiently. Allowing such a mechanism to alter semantics gives freedom for increased performance (albeit at the cost that the application must be aware of what they are doing, and there is added complexity for all involved). I would also support Dick's original thought of a small, pre-defined set of Assertions. This is a small step in the direction of Extensible Assertions (with smaller opportunity for misuse and 'easier to swallow' adoption), and could be designed with an interface that would later permit Extensible Assertions. But I would keep the interface sufficiently distinct from Hints to avoid confusion. Regards, Terry At 11:25 AM -0400 5/15/08, Richard Graham wrote: >We should be careful about making a change in MPI 2.2, knowing that >we will likely turn around again > in MPI 3.0 and change things again. If we are talking about >changing the interface in 2.2, and > then extending the assertions/hints in 3.0, this seems fine, but if >we may want to change the > interface yet again in 3.0, we should rethink things. >I will add that if we are going to add some sort of "info" argument >to 'MPI_Init()', we should deprecate > MPI_Init() and MPI_Init_thread(), and include the threading >specification in the "info" object. >Finally, before we decide on how to pass hints/assertions (if we do) >to MPI_Init(), we should > define a consistent way across the standard for passing information >between the application and > the library, as this is not the only instance where this is useful, >and a uniform way of doing this > makes things much easier on users. > >Rich > > >On 5/14/08 9:14 AM, "Richard Treumann" <treumann_at_[hidden]> wrote: > >MPI_Init time assertions must be few and each must be valuable or >the concept will fall like a house of cards. > >There is nothing in my proposal for MPI_Init time assertions that >rules out providing other mechanisms in MPI 3 for giving guidance to >the MPI implementation. In MPI 3 we can consider more hints and we >can add the abiility to give stronger direction to MPI or provide it >on a more granular basis - If it makes sense. These extra mechanisms >are far to complex to consider as part of MPI 2.2. > >I would not use the phrase that Dries does when he says "Assertions >are bad" but I agree with the the sentiment behind his statement. I >think there should be a very small number of assertions defined in >the standard and for each there should be a good rationale. For MPI >2.2 there should be a great rationale because we can come back in >MPI 3 and add more assertions if we miss some important ones. It is >much harder to remove one that turns out to be real trouble. > >Each assertion the the MPI Standard defines has the potential to >break some piece of code that is valid MPI but that depends on >semantic the assertion says is optional. The author of the routine >that calls MPI_Init has the power to declare assertions and the >authors of other parts of an applicaton must either live within the >rules or explicitly shield against them. > >For project teams that develop complete applications, the decision >to use an assertion belongs to the team, team leader or architect. > If it is decided that an application will use a specific assertion >it is the team lead who must make sure all developers understand >the decision and write appropriate code. All testing will be done >with the assertion in place. > >For third part libraries, the only option is to either forbid all >assertions or explicitly pick some to allow. If there are 4 >potential assertions, it is not very hard to decide for each one - >"Will the library tolerate it?". If there are 50 assertions, >library authors will seldom allow them all and will be more tempted >to just say "No assertions allowed" because making judgements about >each of 50 is too difficult. > >For Community developed code where people contribute source but are >not under direct control of an architect or team lead, reviewing >each submission for compliance with one or two assertions may be >acceptable but reviewing for 50 each time somebody contributes new >code is not. > > > >Dick Treumann - MPI Team/TCEM >IBM Systems & Technology Group >Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 >Tele (845) 433-7846 Fax (845) 433-8363 > > >mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: > >> * Terry Jones <trj_at_[hidden]> [2008-05-13 15:19:07]: >> >> > You can also imagine other possibilities to provide helpful context. For >> > instance, perhaps the user could provide Assertions that would help MPI IO >> > with read-ahead prefetching or write-behind, or even meta-data operations >> > (e.g., later I will be creating one file per MPI task). >> >> Things like read-ahead or write-behind clearly shouldn't be assertions but >> hints. (And, probably 'one file per MPI task' too -- if this is >>still going to >> be needed in 2 years) >> >> MPI already has hints that can capture some of the things mentioned >> above. >> >> access_style: (read_once, write_once, read_mostly, write_mostly, sequential, >> reverse_sequential, and random) >> sequential -> this can easily be used to turn on read-ahead >> IF THE MPI LIBRARY decides this is useful >> >> Assertions are bad -- they break compatibility -- and should only be >> tolerated if they provide real benefits and if the same cannot be obtained >> through existing mechanisms (hints, ...). >> >> In the examples mentioned, this is not the case. >> >> Dries >> >> [attachment "attia6gr.dat" deleted by Richard >> Treumann/Poughkeepsie/IBM] _______________________________________________ >> mpi-22 mailing list >> mpi-22_at_[hidden] >> >><http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > >_______________________________________________ >mpi-22 mailing list >mpi-22_at_[hidden] ><http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > >_______________________________________________ >mpi-22 mailing list >mpi-22_at_[hidden] >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 *

1 0

Re: [Mpi-22] MPI_INIT assertions
by unknown＠example.com 15 May '08

15 May '08

Re. MPI_Info before MPI_Init: if special MPI initialized or owned memory is used for implementing MPI_Info, getting it initialized ahead of the MPI_Init may be tough. If bit flags are appropriate for assertions, I'd say we should go with that, and possibly fold required threading support level into them as well, as discussed earlier. ________________________________ From: Supalov, Alexander Sent: Thursday, May 15, 2008 5:31 PM To: 'MPI 2.2' Subject: RE: [Mpi-22] MPI_INIT assertions Hi, We can pretty well pass data back and forth via MPI_COMM_WORLD attribute already now. This may be too late for assertions proposed by Rick, but seems adequate for everything else, provided we can describe how an implementation should react to the attribute setting action, and what attributes it should attach to the MPI_COMM_WORLD by default. Same with windows and files. Best regards. Alexander ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Graham Sent: Thursday, May 15, 2008 5:26 PM To: MPI 2.2 Subject: Re: [Mpi-22] MPI_INIT assertions We should be careful about making a change in MPI 2.2, knowing that we will likely turn around again in MPI 3.0 and change things again. If we are talking about changing the interface in 2.2, and then extending the assertions/hints in 3.0, this seems fine, but if we may want to change the interface yet again in 3.0, we should rethink things. I will add that if we are going to add some sort of "info" argument to 'MPI_Init()', we should deprecate MPI_Init() and MPI_Init_thread(), and include the threading specification in the "info" object. Finally, before we decide on how to pass hints/assertions (if we do) to MPI_Init(), we should define a consistent way across the standard for passing information between the application and the library, as this is not the only instance where this is useful, and a uniform way of doing this makes things much easier on users. Rich On 5/14/08 9:14 AM, "Richard Treumann" <treumann_at_[hidden]> wrote: MPI_Init time assertions must be few and each must be valuable or the concept will fall like a house of cards. There is nothing in my proposal for MPI_Init time assertions that rules out providing other mechanisms in MPI 3 for giving guidance to the MPI implementation. In MPI 3 we can consider more hints and we can add the abiility to give stronger direction to MPI or provide it on a more granular basis - If it makes sense. These extra mechanisms are far to complex to consider as part of MPI 2.2. I would not use the phrase that Dries does when he says "Assertions are bad" but I agree with the the sentiment behind his statement. I think there should be a very small number of assertions defined in the standard and for each there should be a good rationale. For MPI 2.2 there should be a great rationale because we can come back in MPI 3 and add more assertions if we miss some important ones. It is much harder to remove one that turns out to be real trouble. Each assertion the the MPI Standard defines has the potential to break some piece of code that is valid MPI but that depends on semantic the assertion says is optional. The author of the routine that calls MPI_Init has the power to declare assertions and the authors of other parts of an applicaton must either live within the rules or explicitly shield against them. For project teams that develop complete applications, the decision to use an assertion belongs to the team, team leader or architect. If it is decided that an application will use a specific assertion it is the team lead who must make sure all developers understand the decision and write appropriate code. All testing will be done with the assertion in place. For third part libraries, the only option is to either forbid all assertions or explicitly pick some to allow. If there are 4 potential assertions, it is not very hard to decide for each one - "Will the library tolerate it?". If there are 50 assertions, library authors will seldom allow them all and will be more tempted to just say "No assertions allowed" because making judgements about each of 50 is too difficult. For Community developed code where people contribute source but are not under direct control of an architect or team lead, reviewing each submission for compliance with one or two assertions may be acceptable but reviewing for 50 each time somebody contributes new code is not. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 05/14/2008 07:21:41 AM: > * Terry Jones <trj_at_[hidden]> [2008-05-13 15:19:07]: > > > You can also imagine other possibilities to provide helpful context. For > > instance, perhaps the user could provide Assertions that would help MPI IO > > with read-ahead prefetching or write-behind, or even meta-data operations > > (e.g., later I will be creating one file per MPI task). > > Things like read-ahead or write-behind clearly shouldn't be assertions but > hints. (And, probably 'one file per MPI task' too -- if this is still going to > be needed in 2 years) > > MPI already has hints that can capture some of the things mentioned > above. > > access_style: (read_once, write_once, read_mostly, write_mostly, sequential, > reverse_sequential, and random) > sequential -> this can easily be used to turn on read-ahead > IF THE MPI LIBRARY decides this is useful > > Assertions are bad -- they break compatibility -- and should only be > tolerated if they provide real benefits and if the same cannot be obtained > through existing mechanisms (hints, ...). > > In the examples mentioned, this is not the case. > > Dries > > [attachment "attia6gr.dat" deleted by Richard > Treumann/Poughkeepsie/IBM] _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 ________________________________ _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. *

1 0