- Mpi-22 - mailman3.cels.anl.gov

Re: [Mpi-22] determine if running in a heterogenous environment
by unknown＠example.com 03 Mar '08

03 Mar '08

You have to draw the line somewhere ;-) Especially since -- assuming the user uses MPI_BYTE to describe everything -- the MPI implementation cannot know if any of those datatypes needing adjustment are used. Of course, the application could know this if detailed information about the So, as soon as one of the predefined MPI datatypes is different I would flag the peer (or communicator) as heterogen(e)ous. For example, X86 vs X86_64 and gcc: All C types, except for pointers and long integers are the same; Still, I would say they are heterogen(e)ous without giving more details. Dries * Jeff Squyres <jsquyres_at_[hidden]> [2008-03-03 13:24:11]: > My point is that some datatypes may require translation while others > may not. It's not always just endian differences -- sometimes there's > floating point representation differences, or size differences (e.g., > sizeof(int) != sizeof(int)). *

1 0

Re: [Mpi-22] determine if running in a heterogenous environment
by unknown＠example.com 03 Mar '08

03 Mar '08

The simple question seems to be: Can MPI_BYTE be used safely on this communicator? If even one datatype requires translation then the answer is "no". How about just calling the per communicator attribute something like "MPI_BYTE_SAFE" and leave other debates about the meaning of heterogeneous aside? There is clearly room to add complexity but is there a real need? It seems to me that communicator attribute "MPI_BYTE_SAFE" provides 99% of the value and is easy to do. Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 03/03/2008 01:24:11 PM: > On Mar 3, 2008, at 12:54 PM, Dries Kimpe wrote: > > >> Narrowing the scope to "whether translation functionality is > >> required" > >> is good. But note that whether the translation functionality is used > >> may be specific to a given (communicator,peer,datatype) tuple. > > > > You're right that this depends on the communicator and on the peer; > > ...and the datatype. > > > It would not be uncommon to have one 'different (memory representation > > wise)' node in the communicator. This could be a login node, a > > visualisation node, ... > > > > So, from a performance point of view (and also for the mpi > > implementation > > internally), it's actually better to record this information for > > (communicator, source, dest). However, my main reason for this > > proposal > > was to enable an application that doesn't care about datatypes > > (might be > > more common than you'd think, looking at some of the mpi-subset > > ideas) to > > abort execution if it detects that data conversion is needed when > > communicating to its peers. > > > My point is that some datatypes may require translation while others > may not. It's not always just endian differences -- sometimes there's > floating point representation differences, or size differences (e.g., > sizeof(int) != sizeof(int)). > > -- > Jeff Squyres > Cisco Systems > > _______________________________________________ > Mpi-22 mailing list > Mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 *

1 0

Re: [Mpi-22] determine if running in a heterogenous environment
by unknown＠example.com 03 Mar '08

03 Mar '08

On Mar 3, 2008, at 12:54 PM, Dries Kimpe wrote: >> Narrowing the scope to "whether translation functionality is >> required" >> is good. But note that whether the translation functionality is used >> may be specific to a given (communicator,peer,datatype) tuple. > > You're right that this depends on the communicator and on the peer; ...and the datatype. > It would not be uncommon to have one 'different (memory representation > wise)' node in the communicator. This could be a login node, a > visualisation node, ... > > So, from a performance point of view (and also for the mpi > implementation > internally), it's actually better to record this information for > (communicator, source, dest). However, my main reason for this > proposal > was to enable an application that doesn't care about datatypes > (might be > more common than you'd think, looking at some of the mpi-subset > ideas) to > abort execution if it detects that data conversion is needed when > communicating to its peers. My point is that some datatypes may require translation while others may not. It's not always just endian differences -- sometimes there's floating point representation differences, or size differences (e.g., sizeof(int) != sizeof(int)). -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi-22] determine if running in a heterogenous environment
by unknown＠example.com 03 Mar '08

03 Mar '08

* Jeff Squyres <jsquyres_at_[hidden]> [2008-03-03 12:08:47]: > > I think the proposal has merit but we need to be specific that only > > data representation conversion for data transfer routines is > > involved in the meaning of "heterogeneous". > Narrowing the scope to "whether translation functionality is required" > is good. But note that whether the translation functionality is used > may be specific to a given (communicator,peer,datatype) tuple. You're right that this depends on the communicator and on the peer; It would not be uncommon to have one 'different (memory representation wise)' node in the communicator. This could be a login node, a visualisation node, ... So, from a performance point of view (and also for the mpi implementation internally), it's actually better to record this information for (communicator, source, dest). However, my main reason for this proposal was to enable an application that doesn't care about datatypes (might be more common than you'd think, looking at some of the mpi-subset ideas) to abort execution if it detects that data conversion is needed when communicating to its peers. Greetings, Dries *

1 0

Re: [Mpi-22] determine if running in a heterogenous environment
by unknown＠example.com 03 Mar '08

03 Mar '08

* Richard Treumann <treumann_at_[hidden]> [2008-03-03 12:01:12]: > The real concern here is: > Does the MPI implementation need to provide data conversion services > between any pair of tasks in the communiator? If I have some tasks on a > slow node and other tasks on a fast one we could debate whether that is > heterogeneous. > I think the proposal has merit but we need to be specific that only data > representation conversion for data transfer routines is involved in the > meaning of "heterogeneous". This is what I meant: that's why I talked about not specifying the correct datatype, but instead using MPI_BYTE and relying on the fact that the in-memory representation on both ranks is the same. So, my definition of 'heterogenous': not having the same in-memory data representation. Greetings, Dries > > > Some applications are not written with heterogenous environments in > > > mind; > > > They do not (always) use correct datatype descriptions when sending or > > > receiving data, but instead treat the data as an array of bytes, > > > relying on all datatypes having the same memory representation on both > > > sender and receiver. Most often, this is done to avoid the added > > > complexity > > > of creating the correct datatypes. *

1 0

Re: [Mpi-22] determine if running in a heterogenous environment
by unknown＠example.com 03 Mar '08

03 Mar '08

On Mar 3, 2008, at 12:01 PM, Richard Treumann wrote: > The real concern here is: > > Does the MPI implementation need to provide data conversion services > between any pair of tasks in the communiator? If I have some tasks > on a slow node and other tasks on a fast one we could debate whether > that is heterogeneous. > > I think the proposal has merit but we need to be specific that only > data representation conversion for data transfer routines is > involved in the meaning of "heterogeneous". > Narrowing the scope to "whether translation functionality is required" is good. But note that whether the translation functionality is used may be specific to a given (communicator,peer,datatype) tuple. -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi-22] determine if running in a heterogenous environment
by unknown＠example.com 03 Mar '08

03 Mar '08

The real concern here is: Does the MPI implementation need to provide data conversion services between any pair of tasks in the communiator? If I have some tasks on a slow node and other tasks on a fast one we could debate whether that is heterogeneous. I think the proposal has merit but we need to be specific that only data representation conversion for data transfer routines is involved in the meaning of "heterogeneous". Dick Treumann - MPI Team/TCEM IBM Systems & Technology Group Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601 Tele (845) 433-7846 Fax (845) 433-8363 mpi-22-bounces_at_[hidden] wrote on 03/03/2008 11:09:13 AM: > I think that coming up with a precise definition for "heterogeneous" > could be problematic... > > > On Mar 2, 2008, at 7:01 AM, Dries Kimpe wrote: > > > Below: proposal for MPI-2.2 that adds support for querying if a > > communicator is heterogenous. > > > > ---------------------------------------------------------------------------- > > Proposal: > > ---------------------------------------------------------------------------- > > > > Provide some method to determine if a communicator is heterogenous. > > > > There are a number of different possibilities to provide this > > capability: > > > > 1) Provide a predefined integer valued attribute which can be used > > to query a > > communicator. (for example MPI_HETEROGENOUS) > > > > -- or -- > > > > 2) Create a seperate function: MPI_Comm_is_heterogenous (MPI_Comm > > comm, > > int * flag); > > > > ---------------------------------------------------------------------------- > > Rationale: > > ---------------------------------------------------------------------------- > > > > Currently, MPI-2.0 does not provide a portable way for an > > application to > > determine if it is running in a heterogenous environment. > > > > Some applications are not written with heterogenous environments in > > mind; > > They do not (always) use correct datatype descriptions when sending or > > receiving data, but instead treat the data as an array of bytes, > > relying on all datatypes having the same memory representation on both > > sender and receiver. Most often, this is done to avoid the added > > complexity > > of creating the correct datatypes. > > > > Although the pack/unpack functions provide an alternative, the do > > not have > > the same memory requirements (need an extra buffer to receive the > > data) or > > performance characteristics (need an extra copy). > > > > Adding a way for an application to test if it is currently running > > in a > > heterogenous environment enables it take appropriate action (aborting, > > switching to type-safe functions, ...) > > > > The MPI implementation -- if it supports heterogenous environments -- > > already needs to determine this information because it is > > responsible for > > performing type conversions in heterogenous communicators. > > > > Providing this information on a per-communicator base instead of > > returning > > it for the whole MPI_UNIVERSE / MPI_COMM_WORLD enables both the MPI > > implementation and the user application to avoid overhead > > in case of a homogenous communicator that is a subset of an > > inhomogenous > > MPI_COMM_WORLD. > > > > ---------------------------------------------------------------------------- > > Alternative ways to get the same information without modifying the > > standard: > > ---------------------------------------------------------------------------- > > > > * Store information about the architecture when compiling the > > application; > > Compare this information at runtime with all other members of the > > communicator. > > > > * (not 100% correct): calculate at run time the size of a number of > > elementary datatypes, compare information with the other ranks in the > > communicator. > > > > _______________________________________________ > > Mpi-22 mailing list > > Mpi-22_at_[hidden] > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 > > > -- > Jeff Squyres > Cisco Systems > > _______________________________________________ > Mpi-22 mailing list > Mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 *

1 0

Re: [Mpi-22] determine if running in a heterogenous environment
by unknown＠example.com 03 Mar '08

03 Mar '08

I think that coming up with a precise definition for "heterogeneous" could be problematic... On Mar 2, 2008, at 7:01 AM, Dries Kimpe wrote: > Below: proposal for MPI-2.2 that adds support for querying if a > communicator is heterogenous. > > ---------------------------------------------------------------------------- > Proposal: > ---------------------------------------------------------------------------- > > Provide some method to determine if a communicator is heterogenous. > > There are a number of different possibilities to provide this > capability: > > 1) Provide a predefined integer valued attribute which can be used > to query a > communicator. (for example MPI_HETEROGENOUS) > > -- or -- > > 2) Create a seperate function: MPI_Comm_is_heterogenous (MPI_Comm > comm, > int * flag); > > ---------------------------------------------------------------------------- > Rationale: > ---------------------------------------------------------------------------- > > Currently, MPI-2.0 does not provide a portable way for an > application to > determine if it is running in a heterogenous environment. > > Some applications are not written with heterogenous environments in > mind; > They do not (always) use correct datatype descriptions when sending or > receiving data, but instead treat the data as an array of bytes, > relying on all datatypes having the same memory representation on both > sender and receiver. Most often, this is done to avoid the added > complexity > of creating the correct datatypes. > > Although the pack/unpack functions provide an alternative, the do > not have > the same memory requirements (need an extra buffer to receive the > data) or > performance characteristics (need an extra copy). > > Adding a way for an application to test if it is currently running > in a > heterogenous environment enables it take appropriate action (aborting, > switching to type-safe functions, ...) > > The MPI implementation -- if it supports heterogenous environments -- > already needs to determine this information because it is > responsible for > performing type conversions in heterogenous communicators. > > Providing this information on a per-communicator base instead of > returning > it for the whole MPI_UNIVERSE / MPI_COMM_WORLD enables both the MPI > implementation and the user application to avoid overhead > in case of a homogenous communicator that is a subset of an > inhomogenous > MPI_COMM_WORLD. > > ---------------------------------------------------------------------------- > Alternative ways to get the same information without modifying the > standard: > ---------------------------------------------------------------------------- > > * Store information about the architecture when compiling the > application; > Compare this information at runtime with all other members of the > communicator. > > * (not 100% correct): calculate at run time the size of a number of > elementary datatypes, compare information with the other ranks in the > communicator. > > _______________________________________________ > Mpi-22 mailing list > Mpi-22_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-22 -- Jeff Squyres Cisco Systems

1 0

determine if running in a heterogenous environment
by unknown＠example.com 02 Mar '08

02 Mar '08

Below: proposal for MPI-2.2 that adds support for querying if a communicator is heterogenous. ---------------------------------------------------------------------------- Proposal: ---------------------------------------------------------------------------- Provide some method to determine if a communicator is heterogenous. There are a number of different possibilities to provide this capability: 1) Provide a predefined integer valued attribute which can be used to query a communicator. (for example MPI_HETEROGENOUS) -- or -- 2) Create a seperate function: MPI_Comm_is_heterogenous (MPI_Comm comm, int * flag); ---------------------------------------------------------------------------- Rationale: ---------------------------------------------------------------------------- Currently, MPI-2.0 does not provide a portable way for an application to determine if it is running in a heterogenous environment. Some applications are not written with heterogenous environments in mind; They do not (always) use correct datatype descriptions when sending or receiving data, but instead treat the data as an array of bytes, relying on all datatypes having the same memory representation on both sender and receiver. Most often, this is done to avoid the added complexity of creating the correct datatypes. Although the pack/unpack functions provide an alternative, the do not have the same memory requirements (need an extra buffer to receive the data) or performance characteristics (need an extra copy). Adding a way for an application to test if it is currently running in a heterogenous environment enables it take appropriate action (aborting, switching to type-safe functions, ...) The MPI implementation -- if it supports heterogenous environments -- already needs to determine this information because it is responsible for performing type conversions in heterogenous communicators. Providing this information on a per-communicator base instead of returning it for the whole MPI_UNIVERSE / MPI_COMM_WORLD enables both the MPI implementation and the user application to avoid overhead in case of a homogenous communicator that is a subset of an inhomogenous MPI_COMM_WORLD. ---------------------------------------------------------------------------- Alternative ways to get the same information without modifying the standard: ---------------------------------------------------------------------------- * Store information about the architecture when compiling the application; Compare this information at runtime with all other members of the communicator. * (not 100% correct): calculate at run time the size of a number of elementary datatypes, compare information with the other ranks in the communicator. *

1 0

Proposed amendment to application cleanup callback text
by unknown＠example.com 01 Mar '08

01 Mar '08

Hi all, I've got a problem which I think might be a candidate for a fix in the 2.2 standard: Is it legal to make MPI calls in the 'delete' callback for an attribute attached to MPI_COMM_SELF which is being freed during a call to MPI_Finalize? Section 4.8 in the 2.0 standard doesn't really say, I think. Here's the text: > 4.8 Allowing User Functions at Process Termination > > There are times in which it would be convenient to have actions > happen when an MPI process > ï¬nishes. For example, a routine may do initializations that are > useful until the MPI job (or > that part of the job that being terminated in the case of > dynamically created processes) is > ï¬nished. This can be accomplished in MPI-2 by attaching an > attribute to MPI_COMM_SELF > with a callback function. When MPI_FINALIZE is called, it will ï¬rst > execute the equivalent > of an MPI_COMM_FREE on MPI_COMM_SELF. This will cause the delete > callback function > to be executed on all keys associated with MPI_COMM_SELF, in an > arbitrary order. If no > key has been attached to MPI_COMM_SELF, then no callback is invoked. > The âfreeingâ of > MPI_COMM_SELF occurs before any other parts of MPI are aï¬ected. > Thus, for example, > calling MPI_FINALIZED will return false in any of these callback > functions. Once done with > MPI_COMM_SELF, the order and rest of the actions taken by > MPI_FINALIZE is not speciï¬ed. > > Advice to implementors. Since attributes can be added from any > supported language, > the MPI implementation needs to remember the creating language so > the correct > callback is made. (End of advice to implementors.) Here's my use case: we cache modified file information in the HDF5 library and need to be able to flush that data to the file if the user calls MPI_Finalize without calling the HDF5 routines to close (or flush) that file first. It looks like adding an attribute onto MPI_COMM_SELF is the right thing to do, and the standard seems to imply that it's possible to make MPI calls from that callback, but it's not certain to me. (Obviously, the HDF5 library needs to make I/ O calls and possible may need to exchange data or invoke a barrier, etc) Is it reasonable to add a sentence to the effect that "all MPI operations (except MPI_Finalize and MPI_Init) are possible from the attribute's 'delete' callback"? Here's a copy of section 4.8, with text to that effect worked in: > 4.8 Allowing User Functions at Process Termination > > There are times in which it would be convenient to have actions > happen when an MPI process > ï¬nishes. For example, a routine may do initializations that are > useful until the MPI job (or > that part of the job that being terminated in the case of > dynamically created processes) is > ï¬nished. This can be accomplished in MPI-2 by attaching an > attribute to MPI_COMM_SELF > with a callback function. When MPI_FINALIZE is called, it will ï¬rst > execute the equivalent > of an MPI_COMM_FREE on MPI_COMM_SELF. This will cause the delete > callback function > to be executed on all keys associated with MPI_COMM_SELF, in an > arbitrary order. If no > key has been attached to MPI_COMM_SELF, then no callback is invoked. > The âfreeingâ of > MPI_COMM_SELF occurs before any other parts of MPI are aï¬ected and > all MPI operations (except MPI_Finalize and MPI_Init) are available > to the attribute's 'delete' callback. Thus, for example, > calling MPI_FINALIZED will return false in any of these callback > functions. Once done with > MPI_COMM_SELF, the order and rest of the actions taken by > MPI_FINALIZE is not speciï¬ed. > > Advice to implementors. Since attributes can be added from any > supported language, > the MPI implementation needs to remember the creating language so > the correct > callback is made. (End of advice to implementors.) Thanks, Quincey Koziol The HDF Group

1 0