September 2008 - Mpi-22 - mailman3.cels.anl.gov

Re: [Mpi-22] Dynamic thread levels
by unknown＠example.com 04 Sep '08

04 Sep '08

Dick, I think there are two independent questions here -- one is about the potential performance improvement, and another is about the ease of use (or rather complexity of use) for application writers. To some degree, I agree with both the concerns, but here are some notes for each one. With respect to the first, though latency improvement is probably only marginal (especially on fast processors), the degradation in the message rate is usually quite high. I've implemented this in MPICH2 and placed a copy here: https://svn.mcs.anl.gov/repos/mpi/mpich2/branches/dev/dyn_thread_level. Based on this implementation, I did some measurements using a single threaded application, but using MPI_Init_thread() vs. MPI_Init(). Experiment 1: Each process sends to MPI_PROC_NULL (emulating an infinitely fast network) MPI_Init -- 29.94 million messages per second MPI_Init_thread -- 15.86 million messages per second Difference -- almost 2X Experiment 2: Two processes communicating over TCP MPI_Init -- 3.947 million messages per second MPI_Init_thread -- 3.231 million messages per second Difference -- about 20% For fast networks, the difference might be somewhere in between. With respect to the second question, I agree that we are placing a lot of faith in the application developers for this, but this is probably not too different from trusting the application in that only one thread will call a Barrier or a Bcast, for example. If multiple threads call barriers in applications where only one thread is supposed to call, bad things can happen, but we trust that the user will not do that. Basically, there is a potential for bugs here, but I don't think that's any more than what is already present. If there are any concerns, or if you think we can do this in a better way, please do let me know. I'm open to editing this (even substantially if needed). Thanks. -- Pavan On 09/04/2008 09:35 AM, Richard Treumann wrote: > Pavan, > > Can you explain what improvements you expect if MPI added this? > > If an MPI implementation that is capable of supporting > MPI_THREAD_MULTIPLE is running in MPI_THREAD_SINGLE mode, that probably > means all protection against multiple threads is being bypassed at > rather modest latency savings. The savings probably come from branching > over lock operations. Ongoing checking for concurrent threads making MPI > calls is expensive enough so doing it would eat up much of the savings > from branching over lock operations. Basically, the MPI_THREAD_MULTIPLE > capable MPI running in MPI_THREAD_SINGLE mode is probably defenseless > against multi thread applications. > > What this means for MPI_Set_thread_level(int required, int * provided); > > is that the application would need to take full responsibility for > making sure there is only one thread making MPI calls at the moment a > mode switch is requested. If the application gets it wrong there is > probably nothing (affordable) the MPI implementation can do to detect > the danger. Applications that turned thread safety on and off but were > misusing the call would never get an error message and might run without > problems 99 times out of 100. 1% of the runs would have mysterious > failures and the failures might each look quite different. > > It is certainly possible for an MPI implementation that only supports > MPI_THREAD_SINGLE to do some things faster by using simpler global data > structures but such an MPI implementation could not honor a runtime > request to convert to MPI_THREAD_MULTIPLE. It may even be possible for > an MPI implementation at INIT time to chose simple vs. thread safe data > structures but again, a switch once everything is up and running is > probably not practical. > > This does not seem to me to offer enough payoff to justify the dangers. > > Dick > > Dick Treumann - MPI Team > IBM Systems & Technology Group > Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 > Tele (845) 433-7846 Fax (845) 433-8363 > > > mpi-22-bounces_at_[hidden] wrote on 09/04/2008 04:30:23 AM: > > > [image removed] > > > > [Mpi-22] Dynamic thread levels > > > > Pavan Balaji > > > > to: > > > > mpi-22 > > > > 09/04/2008 04:39 AM > > > > Sent by: > > > > mpi-22-bounces_at_[hidden] > > > > Please respond to "MPI 2.2" > > > > Hi all, > > > > I would like to propose an addition of a function call to dynamically > > modify the thread level required by the application, instead of at > > MPI_Init_thread() time. I'm sending this over email for comments > > (attached with this email); I'll upload it on the wiki this evening. > > > > -- Pavan > > > > -- > > Pavan Balaji > > http://www.mcs.anl.gov/~balaji > > Dynamic Thread Levels > > > > Author: Pavan Balaji, Argonne National Laboratory > > > > Background: > > > > MPI 2.1 standard allows users to set thread level only at > > MPI_Init_thread time. This means that for applications that > > dynamically create threads (e.g., hybrid MPI + OpenMP applications), > > even non-threaded portions of the application have to rely on the > > maximum thread level used in the application (e.g., MULTIPLE). > > > > > > Proposal: > > > > Add an additional function call to dynamically set thread-level during > > the application. > > > > int MPI_Set_thread_level(int required, int * provided); > > > > > > Rational: > > > > The requirement to specify the thread-level at MPI_Init_thread time is > > too restrictive for applications that perform a small amount of > > communication requiring a high-level of thread support. For > > correctness, the standard requires all of the code to follow the same > > thread-level, and provides the applications with no way to give the > > MPI library more information about their behavior. > > > > > > Impact on MPI implementations: > > > > Most MPI implementations already provide runtime support for > > thread-level, i.e., locks are compiled in, but whether they are > > invoked or not is decided at runtime. For implementations that choose > > not to respect this option, MPI_Set_thread_level() can just set the > > provided level to the current level, by ignoring the required level > > specified by the user. > > > > > > Impact on MPI applications: > > > > Existing MPI applications do not need to be modified at all. But newer > > applications can benefit from this additional functionality. > > _______________________________________________ > > mpi-22 mailing list > > mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > ------------------------------------------------------------------------ > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [Mpi-22] New proposal: Support for large message counts
by unknown＠example.com 04 Sep '08

04 Sep '08

Terabytes are pushed back and forth already now, maybe not yet in one piece. MPI-3 is a couple of years away, by which time who knows what may happen. Anyway, you're right, this may not be an immediate concern. At the moment, IBM Power apparently implements lp64 model. We typically see the same thing on Intel 64 based platforms. Our compiler's long long is 64 bit in that case, too. Same on Itanium. The latter already has several multiply/add commands with an intermediate full 128-bit product out of 64-bit int operands, and the corresponding compiler intrinsics, however. ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Thursday, September 04, 2008 5:25 PM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts Are there systems today that use a 128 bit long long? Are there virtual memory systems or filesystems coming in the reasonable future that use a 128 bit address range? On IBM Power we support both 32 bit and 64 bit executables. For both 32 bit and 64 bit executables, the C compilers treat int as 32 bits and long long as 64 bit. C long differs so is 32 bits in a 32 bit executable and 64 bits in a 64 bit executable (I.E. big enough to hold an address/pointer). The jump from 32 bits to 64 bits is a factor of 4 billion so it may be goodness in the abstract to think about what comes after 64 bits gives way to 128 bits, should we consider this a real problem? I do not see much reason to worry about the fact that multiplying 2 arbitrary 64 bit values can overflow a 64 bit result unless we think some real situation will call for 128 bit results. Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 Alexander wrote <snip> > Another thing is that if we multiply to long values, the result may > potentially overflow a long. So, for a library that allows long > counts and long datatype extents, the internals of the library will > have to be long long. long long (128 bit) arithmetic may be rather > expensive on some CPUs. So, the 64-bit interface should probably be optional. <snip> --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. *

1 0

Re: [Mpi-22] New proposal: Support for large message counts
by unknown＠example.com 04 Sep '08

04 Sep '08

Are there systems today that use a 128 bit long long? Are there virtual memory systems or filesystems coming in the reasonable future that use a 128 bit address range? On IBM Power we support both 32 bit and 64 bit executables. For both 32 bit and 64 bit executables, the C compilers treat int as 32 bits and long long as 64 bit. C long differs so is 32 bits in a 32 bit executable and 64 bits in a 64 bit executable (I.E. big enough to hold an address/pointer). The jump from 32 bits to 64 bits is a factor of 4 billion so it may be goodness in the abstract to think about what comes after 64 bits gives way to 128 bits, should we consider this a real problem? I do not see much reason to worry about the fact that multiplying 2 arbitrary 64 bit values can overflow a 64 bit result unless we think some real situation will call for 128 bit results. Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 Alexander wrote <snip> > Another thing is that if we multiply to long values, the result may > potentially overflow a long. So, for a library that allows long > counts and long datatype extents, the internals of the library will > have to be long long. long long (128 bit) arithmetic may be rather > expensive on some CPUs. So, the 64-bit interface should probably be optional. <snip> *

1 0

Re: [Mpi-22] Dynamic thread levels
by unknown＠example.com 04 Sep '08

04 Sep '08

Pavan, Can you explain what improvements you expect if MPI added this? If an MPI implementation that is capable of supporting MPI_THREAD_MULTIPLE is running in MPI_THREAD_SINGLE mode, that probably means all protection against multiple threads is being bypassed at rather modest latency savings. The savings probably come from branching over lock operations. Ongoing checking for concurrent threads making MPI calls is expensive enough so doing it would eat up much of the savings from branching over lock operations. Basically, the MPI_THREAD_MULTIPLE capable MPI running in MPI_THREAD_SINGLE mode is probably defenseless against multi thread applications. What this means for MPI_Set_thread_level(int required, int * provided); is that the application would need to take full responsibility for making sure there is only one thread making MPI calls at the moment a mode switch is requested. If the application gets it wrong there is probably nothing (affordable) the MPI implementation can do to detect the danger. Applications that turned thread safety on and off but were misusing the call would never get an error message and might run without problems 99 times out of 100. 1% of the runs would have mysterious failures and the failures might each look quite different. It is certainly possible for an MPI implementation that only supports MPI_THREAD_SINGLE to do some things faster by using simpler global data structures but such an MPI implementation could not honor a runtime request to convert to MPI_THREAD_MULTIPLE. It may even be possible for an MPI implementation at INIT time to chose simple vs. thread safe data structures but again, a switch once everything is up and running is probably not practical. This does not seem to me to offer enough payoff to justify the dangers. Dick Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 09/04/2008 04:30:23 AM: > [image removed] > > [Mpi-22] Dynamic thread levels > > Pavan Balaji > > to: > > mpi-22 > > 09/04/2008 04:39 AM > > Sent by: > > mpi-22-bounces_at_[hidden] > > Please respond to "MPI 2.2" > > Hi all, > > I would like to propose an addition of a function call to dynamically > modify the thread level required by the application, instead of at > MPI_Init_thread() time. I'm sending this over email for comments > (attached with this email); I'll upload it on the wiki this evening. > > -- Pavan > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > Dynamic Thread Levels > > Author: Pavan Balaji, Argonne National Laboratory > > Background: > > MPI 2.1 standard allows users to set thread level only at > MPI_Init_thread time. This means that for applications that > dynamically create threads (e.g., hybrid MPI + OpenMP applications), > even non-threaded portions of the application have to rely on the > maximum thread level used in the application (e.g., MULTIPLE). > > > Proposal: > > Add an additional function call to dynamically set thread-level during > the application. > > int MPI_Set_thread_level(int required, int * provided); > > > Rational: > > The requirement to specify the thread-level at MPI_Init_thread time is > too restrictive for applications that perform a small amount of > communication requiring a high-level of thread support. For > correctness, the standard requires all of the code to follow the same > thread-level, and provides the applications with no way to give the > MPI library more information about their behavior. > > > Impact on MPI implementations: > > Most MPI implementations already provide runtime support for > thread-level, i.e., locks are compiled in, but whether they are > invoked or not is decided at runtime. For implementations that choose > not to respect this option, MPI_Set_thread_level() can just set the > provided level to the current level, by ignoring the required level > specified by the user. > > > Impact on MPI applications: > > Existing MPI applications do not need to be modified at all. But newer > applications can benefit from this additional functionality. > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 *

1 0

Re: [Mpi-22] New proposal: Support for large message counts
by unknown＠example.com 04 Sep '08

04 Sep '08

Alexander, We do have use-cases of customers requireing a mix of 32-bit and 64-bit executables (as strange as it sounds). These are not typical MPI applications, though... Thanx! Kannan -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Thursday, September 04, 2008 3:58 AM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts Hi, Yes, I was looking into maintaining one library source code with the ability to build several versions out of it. Since 1) and 2) will be essentially ABI compatible, one can ship one version of the 32-bit dynamic library to cover both cases. The 64-bit library will be different, and this is I think what customers want anyway - they don't seem to want mixing 32- and 64-bit calls in one application. Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Narasimhan, Kannan Sent: Wednesday, September 03, 2008 7:32 PM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts Alexander, Thanx for the suggestion. 1. Maybe I'm missing something here, but without name change, we break backward compatibility with "old" binaries coded to the existing APIs with 32-bit counts, and the new MPI 2.2 compatible MPI shared library that supports 64-bit counts --- without a re-build/relink step. Or are you proposing that implementations have multiple libraries, one that supports 32 bit counts, one that supports 64 bit counts, etc.? With a single library, (1) and (2) addresses backward compatibility, but maintains status-quo on support for large message counts, and (3) and (4) requires rebuild. However, having different names will take care of this. 2. If needed, we can explore normalization to a single API that supports MPI_Count in the MPI 3.0 time frame, since MPI 3.0 is open changes that can break backward compatibility. 3. I cannot think of any use cases for a "signed" count, but I'll let others chime in here. 4. I agree that new applications will have no issues with your approach. Thanx! Kannan -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Wednesday, September 03, 2008 11:39 AM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts Thanks. We may even do without changing the MPI routine names I think. Let's consider this: 1) #define MPI_Count int will yield the current MPI interface. 2) typedef int MPI_Count will yield the new 32-bit interface, still backward binary compatible with 1). 3) typedef long MPI_Count will yield new 64-bit interface. 4) typedef long long MPI_Count will safely bring us into XXII century (or earlier). Old applications may be rebuilt using 1) without any changes, dynamically linked against MPI library built using 2), and will have to be rebuilt to use MPI library built using 3) or 4). New applications will use library with the MPI_Count definitions, and, properly rebuilt, will work in either 32-, 64-, or 128-bit mode, using MPI libraries built using MPI bindings 2), 3), or 4), respectively. By the way, should MPI_Count really be signed? -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Narasimhan, Kannan Sent: Wednesday, September 03, 2008 5:55 PM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts We can certainly consider this option, since it abstracts along the same lines as MPI_Offset. Thanx! Kannan -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Wednesday, September 03, 2008 10:19 AM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts Dear Kannan, Thanks. Why don't we define MPI_Count datatype instead, and make that grow if necessary? Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Narasimhan, Kannan Sent: Friday, August 29, 2008 6:15 PM To: mpi-22_at_[hidden] Subject: [Mpi-22] New proposal: Support for large message counts I have submitted a new 2.2 proposal to addresses the need for large message counts (i.e. counts greater than the size of 32-bit integer) for MPI calls that communicate messages. Please refer to https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/LargeMsgCounts for details. Comments and suggestions are welcome. Thanx! Kannan _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

1 0

Re: [Mpi-22] New proposal: Support for large message counts
by unknown＠example.com 04 Sep '08

04 Sep '08

Good point. At face value, this seems to suggest that we should have "extended" constructors available on a level with the traditional ones, thus favoring the original name-shifted proposal. If I think about the library internals in this case, however, this addition will effectively push all of the library extent calculations to 64-bit (or more, see below). Another thing is that if we multiply to long values, the result may potentially overflow a long. So, for a library that allows long counts and long datatype extents, the internals of the library will have to be long long. long long (128 bit) arithmetic may be rather expensive on some CPUs. So, the 64-bit interface should probably be optional. ________________________________ From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Richard Treumann Sent: Wednesday, September 03, 2008 7:24 PM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts MPI_Aint has the right number of bits to hold an address. For operations that are address space oriented, count can be the same size as an MPI_Aint MPI_Offset has the right number of bits to deal with the range in a file. For operations that are file oriented, arguments like count should be an MPI_Offset A problem is that MPI_Datatype constructors are intended to apply to both and it is common for a 32 bit address space to operate on files with 64 bit offsets. What type should "count" on the new "long" MPI_Datatype constructors be in this case? The proposal says MPI_Aint but that is awkward for datatypes as fileviews. Dick Treumann - MPI Team IBM Systems & Technology Group Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 09/03/2008 12:38:35 PM: > [image removed] > > Re: [Mpi-22] New proposal: Support for large message counts > > Supalov, Alexander > > to: > > MPI 2.2 > > 09/03/2008 12:48 PM > > Sent by: > > mpi-22-bounces_at_[hidden] > > Please respond to "MPI 2.2" > > Thanks. We may even do without changing the MPI routine names I think. > Let's consider this: > > 1) #define MPI_Count int will yield the current MPI interface. > 2) typedef int MPI_Count will yield the new 32-bit interface, still > backward binary compatible with 1). > 3) typedef long MPI_Count will yield new 64-bit interface. > 4) typedef long long MPI_Count will safely bring us into XXII century > (or earlier). > > Old applications may be rebuilt using 1) without any changes, > dynamically linked against MPI library built using 2), and will have to > be rebuilt to use MPI library built using 3) or 4). > > New applications will use library with the MPI_Count definitions, and, > properly rebuilt, will work in either 32-, 64-, or 128-bit mode, using > MPI libraries built using MPI bindings 2), 3), or 4), respectively. > > By the way, should MPI_Count really be signed? > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] > [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Narasimhan, > Kannan > Sent: Wednesday, September 03, 2008 5:55 PM > To: MPI 2.2 > Subject: Re: [Mpi-22] New proposal: Support for large message counts > > We can certainly consider this option, since it abstracts along the same > lines as MPI_Offset. > > Thanx! > Kannan > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] > [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, > Alexander > Sent: Wednesday, September 03, 2008 10:19 AM > To: MPI 2.2 > Subject: Re: [Mpi-22] New proposal: Support for large message counts > > Dear Kannan, > > Thanks. Why don't we define MPI_Count datatype instead, and make that > grow if necessary? > > Best regards. > > Alexander > > -----Original Message----- > From: mpi-22-bounces_at_[hidden] > [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Narasimhan, > Kannan > Sent: Friday, August 29, 2008 6:15 PM > To: mpi-22_at_[hidden] > Subject: [Mpi-22] New proposal: Support for large message counts > > I have submitted a new 2.2 proposal to addresses the need for large > message counts (i.e. counts greater than the size of 32-bit integer) for > MPI calls that communicate messages. > > Please refer to > https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/LargeMsgCounts > for details. > > Comments and suggestions are welcome. > > Thanx! > Kannan > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution by > others is strictly prohibited. If you are not the intended recipient, > please contact the sender and delete all copies. > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > --------------------------------------------------------------------- > Intel GmbH > Dornacher Strasse 1 > 85622 Feldkirchen/Muenchen Germany > Sitz der Gesellschaft: Feldkirchen bei Muenchen > Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer > Registergericht: Muenchen HRB 47456 Ust.-IdNr. > VAT Registration No.: DE129385895 > Citibank Frankfurt (BLZ 502 109 00) 600119052 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > mpi-22 mailing list > mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. *

1 0

Re: [Mpi-22] New proposal: Support for large message counts
by unknown＠example.com 04 Sep '08

04 Sep '08

Hi, Yes, I was looking into maintaining one library source code with the ability to build several versions out of it. Since 1) and 2) will be essentially ABI compatible, one can ship one version of the 32-bit dynamic library to cover both cases. The 64-bit library will be different, and this is I think what customers want anyway - they don't seem to want mixing 32- and 64-bit calls in one application. Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Narasimhan, Kannan Sent: Wednesday, September 03, 2008 7:32 PM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts Alexander, Thanx for the suggestion. 1. Maybe I'm missing something here, but without name change, we break backward compatibility with "old" binaries coded to the existing APIs with 32-bit counts, and the new MPI 2.2 compatible MPI shared library that supports 64-bit counts --- without a re-build/relink step. Or are you proposing that implementations have multiple libraries, one that supports 32 bit counts, one that supports 64 bit counts, etc.? With a single library, (1) and (2) addresses backward compatibility, but maintains status-quo on support for large message counts, and (3) and (4) requires rebuild. However, having different names will take care of this. 2. If needed, we can explore normalization to a single API that supports MPI_Count in the MPI 3.0 time frame, since MPI 3.0 is open changes that can break backward compatibility. 3. I cannot think of any use cases for a "signed" count, but I'll let others chime in here. 4. I agree that new applications will have no issues with your approach. Thanx! Kannan -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Wednesday, September 03, 2008 11:39 AM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts Thanks. We may even do without changing the MPI routine names I think. Let's consider this: 1) #define MPI_Count int will yield the current MPI interface. 2) typedef int MPI_Count will yield the new 32-bit interface, still backward binary compatible with 1). 3) typedef long MPI_Count will yield new 64-bit interface. 4) typedef long long MPI_Count will safely bring us into XXII century (or earlier). Old applications may be rebuilt using 1) without any changes, dynamically linked against MPI library built using 2), and will have to be rebuilt to use MPI library built using 3) or 4). New applications will use library with the MPI_Count definitions, and, properly rebuilt, will work in either 32-, 64-, or 128-bit mode, using MPI libraries built using MPI bindings 2), 3), or 4), respectively. By the way, should MPI_Count really be signed? -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Narasimhan, Kannan Sent: Wednesday, September 03, 2008 5:55 PM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts We can certainly consider this option, since it abstracts along the same lines as MPI_Offset. Thanx! Kannan -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Supalov, Alexander Sent: Wednesday, September 03, 2008 10:19 AM To: MPI 2.2 Subject: Re: [Mpi-22] New proposal: Support for large message counts Dear Kannan, Thanks. Why don't we define MPI_Count datatype instead, and make that grow if necessary? Best regards. Alexander -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Narasimhan, Kannan Sent: Friday, August 29, 2008 6:15 PM To: mpi-22_at_[hidden] Subject: [Mpi-22] New proposal: Support for large message counts I have submitted a new 2.2 proposal to addresses the need for large message counts (i.e. counts greater than the size of 32-bit integer) for MPI calls that communicate messages. Please refer to https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/LargeMsgCounts for details. Comments and suggestions are welcome. Thanx! Kannan _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies. _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 --------------------------------------------------------------------- Intel GmbH Dornacher Strasse 1 85622 Feldkirchen/Muenchen Germany Sitz der Gesellschaft: Feldkirchen bei Muenchen Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer Registergericht: Muenchen HRB 47456 Ust.-IdNr. VAT Registration No.: DE129385895 Citibank Frankfurt (BLZ 502 109 00) 600119052 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

1 0

Dynamic thread levels
by unknown＠example.com 04 Sep '08

04 Sep '08

Hi all, I would like to propose an addition of a function call to dynamically modify the thread level required by the application, instead of at MPI_Init_thread() time. I'm sending this over email for comments (attached with this email); I'll upload it on the wiki this evening. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji Dynamic Thread Levels Author: Pavan Balaji, Argonne National Laboratory Background: MPI 2.1 standard allows users to set thread level only at MPI_Init_thread time. This means that for applications that dynamically create threads (e.g., hybrid MPI + OpenMP applications), even non-threaded portions of the application have to rely on the maximum thread level used in the application (e.g., MULTIPLE). Proposal: Add an additional function call to dynamically set thread-level during the application. int MPI_Set_thread_level(int required, int * provided); Rational: The requirement to specify the thread-level at MPI_Init_thread time is too restrictive for applications that perform a small amount of communication requiring a high-level of thread support. For correctness, the standard requires all of the code to follow the same thread-level, and provides the applications with no way to give the MPI library more information about their behavior. Impact on MPI implementations: Most MPI implementations already provide runtime support for thread-level, i.e., locks are compiled in, but whether they are invoked or not is decided at runtime. For implementations that choose not to respect this option, MPI_Set_thread_level() can just set the provided level to the current level, by ignoring the required level specified by the user. Impact on MPI applications: Existing MPI applications do not need to be modified at all. But newer applications can benefit from this additional functionality.

1 0

Re: [Mpi-22] MPI::Grequest::Start proposal
by unknown＠example.com 04 Sep '08

04 Sep '08

I think that this is okay; when we discussed it, I thought it was a different fix. The proposal is just fine. -----Original Message----- From: mpi-22-bounces_at_[hidden] [mailto:mpi-22-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Thursday, September 04, 2008 7:26 AM To: MPI 2.2 Subject: [Mpi-22] MPI::Grequest::Start proposal Regarding https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/GrequestStartFnPtrA… : I have confirmed in Open MPI that if you change this: static Grequest Start(Query_function, Free_function, Cancel_function, void *); to static Grequest Start(Query_function *, Free_function *, Cancel_function *, void *); Recompile and reinstall, both the implementation and same C++ user code compiles without warning/error. Erez raised the point that this would change the type of any implementation and user-declared variables that hold the function pointer. It does not; such a variable type must always be (Free_function*); it's only the type that is passed through a function argument that can be either (Free_function) or (Free_function*). Someone smarter than me in C++ can explain why. :-) Specifically, the following compiles and runs without warning/error: ----- #include <stdio.h> typedef int Free_function(void *); int my_function(void*) { printf("In my_function\n"); return 0; } void foo1(Free_function ptr) { Free_function *save = ptr; save(0); } void foo2(Free_function* ptr) { Free_function *save = ptr; save(0); } int main (int argc, char*argv[]) { foo1(my_function); foo2(my_function); return 0; } ----- I tried 4 different C++ compilers (gnu, intel, pgi, pathscale): ----- [23:19] svbu-mpi:~/tmp % g++ foo.cc -o foo && ./foo In my_function In my_function [23:21] svbu-mpi:~/tmp % icpc foo.cc -o foo && ./foo In my_function In my_function [23:21] svbu-mpi:~/tmp % pgCC foo.cc -o foo && ./foo In my_function In my_function [23:22] svbu-mpi:~/tmp % pathCC foo.cc -o foo && ./foo In my_function In my_function [23:22] svbu-mpi:~/tmp % ------ So I think the proposal stands as it is written. -- Jeff Squyres Cisco Systems _______________________________________________ mpi-22 mailing list mpi-22_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22

1 0

MPI::Grequest::Start proposal
by unknown＠example.com 04 Sep '08

04 Sep '08

Regarding https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/mpi22/GrequestStartFnPtrA… : I have confirmed in Open MPI that if you change this: static Grequest Start(Query_function, Free_function, Cancel_function, void *); to static Grequest Start(Query_function *, Free_function *, Cancel_function *, void *); Recompile and reinstall, both the implementation and same C++ user code compiles without warning/error. Erez raised the point that this would change the type of any implementation and user-declared variables that hold the function pointer. It does not; such a variable type must always be (Free_function*); it's only the type that is passed through a function argument that can be either (Free_function) or (Free_function*). Someone smarter than me in C++ can explain why. :-) Specifically, the following compiles and runs without warning/error: ----- #include <stdio.h> typedef int Free_function(void *); int my_function(void*) { printf("In my_function\n"); return 0; } void foo1(Free_function ptr) { Free_function *save = ptr; save(0); } void foo2(Free_function* ptr) { Free_function *save = ptr; save(0); } int main (int argc, char*argv[]) { foo1(my_function); foo2(my_function); return 0; } ----- I tried 4 different C++ compilers (gnu, intel, pgi, pathscale): ----- [23:19] svbu-mpi:~/tmp % g++ foo.cc -o foo && ./foo In my_function In my_function [23:21] svbu-mpi:~/tmp % icpc foo.cc -o foo && ./foo In my_function In my_function [23:21] svbu-mpi:~/tmp % pgCC foo.cc -o foo && ./foo In my_function In my_function [23:22] svbu-mpi:~/tmp % pathCC foo.cc -o foo && ./foo In my_function In my_function [23:22] svbu-mpi:~/tmp % ------ So I think the proposal stands as it is written. -- Jeff Squyres Cisco Systems

1 0