Rainer: There are two kinds of optional. In the paragraph you were responding to, I meant that the user would be explicitly required to include the profiling interface. This requirement is generally a bad idea. Recompilation is a huge barrier to tool adoption. As I have tried to make clear in my comments, requiring recompilation is not a good solution. You would have to demonstrate that the performance benefits are significant across a range of architectures and implementations to justify it. I do not believe that you can demonstrate that since most implementations would eventually make some function call. Eliminating the API level call would not provide much benefit in that case; you could instead inline everything below the API and not make anything public. That solution comes at no cost. Alternatively, linking to a non-standard library is reasonable solution for the rare case when it does make a difference (and the code is not constantly in development, as is the case for MOST large scale applications). Bronis On Fri, 7 Mar 2008, Rainer Keller wrote:

Hello Bronis, thanks for Your comments.

On Thursday 06 March 2008 17:58, Bronis R. de Supinski wrote:

...

There are some significant problems with this proposal. The premise that the profiling interface is only used early in the development is not correct. The PMPI interface is useful at almost every stage of the application's life time. One of the things that makes it successful is that it is always available and does not require recompilation. Making it optional would destroy that. This would not make it optional. It still is required by the standard. However, if the user wishes to do so, he may ask the implementation to inline functions (and on top of that specify some characterstics of his/her app).

...

I can tell you that one of the biggest problems for OpenMP has been the lack of a profiling interface. We recently had one sanctioned by the OpenMP ARB but that took a lot of effort since it is very hard to add it in after the fact. Your proposal would not be as drastic as eliminating the interface but the effect would be almost as catastrophic. Hmm, I do follow Your argument on OpenMP that is a real issue... However fail to see, why it's so dramatic in our case (see above).

...

Perhaps just as importantly to the general need of the interface, the idea that inlining of MPI will provide substantial benefits is inaccurate. MPI is a library interface. Many implementations are distributed only in binary form; They do provide at least one header file, now. They could provide other header files. They could provide obviouscated header files which call into specialized functions, that do not check for any_source ,-]

...

unless you are proposing link time inlining (possible but not without its pitfalls), No.

...

they will not gain any benefit from this. Even if we consider only open source MPI implementations, the VAST majority of users do NOT want to compile the MPI library. The possible performance gain is far too small for them to take on this task. Errr, they do not need to compile the MPI library at all? They just #define MPI_HINT_INLINE #include "mpi.h"

and the magic happens.

...

A properly designed subsetting approach will NOT try to partition the profiling interface into its own subset. Yeah, that was probably the biggest misunderstanding. Please see the other mail ,-]

...

Instead, the interface must be partitioned into different subsets, with each PMPI function in the same subset as the corresponding MPI function. Yes.

Thanks, Rainer

...

On Wed, 5 Mar 2008, Rainer Keller wrote:

...

Dear Alexander, dear all, at the previous Chicago meeting, some of us (Rich Graham, Jeff Squyres, Hubert Ritzdorf) have been discussing about the MPI-Standard's requirement to provide PMPI-functions for each MPI-Call into the library (except MPI_Wtime, MPI_Wtick).

This is in my eyes a limitation: there could be some gains for the common case of large-scale applications: - Not using PMPI-based tools (dynamically loaded in) - Not using MPI_PROC_NULL - Not using MPI_STATUS_IGNORE

So, if this proposal would fit Your definition of MPI3-Subsets, I would like to discuss the removal of the PMPI-function requirement, allowing inlining of MPI-functions and, based on that, pre-processor hints, e.g. MPI_HINT_NO_THREADS or MPI_HINT_NO_ANY_SOURCE, so the compiler might be able to eliminate several if-statements in the fast-path, allowing inlining, allowing dead code elimination. (of course, there is a whole lot of further issues involved, here -- please see the last section in the document).

I have included some timings with NetPipe on Open MPI using these hints (further information section 4 -- Ompi Patch availble upon request, as its work in progress). Please take the timings with a grain of salt. I would like to get better figures on a more well-defined cluster environment.

Any comments would be welcome.

With best regards, Rainer -- ---------------------------------------------------------------- Dipl.-Inf. Rainer Keller http://www.hlrs.de/people/keller HLRS Tel: ++49 (0)711-685 6 5858 Nobelstrasse 19 Fax: ++49 (0)711-685 6 5832 70550 Stuttgart email: keller_at_[hidden] Germany AIM/Skype:rusraink

_______________________________________________ mpi3-subsetting mailing list mpi3-subsetting_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-subsetting

-- ---------------------------------------------------------------- Dipl.-Inf. Rainer Keller http://www.hlrs.de/people/keller HLRS Tel: ++49 (0)711-685 6 5858 Nobelstrasse 19 Fax: ++49 (0)711-685 6 5832 70550 Stuttgart email: keller_at_[hidden] Germany AIM/Skype:rusraink