Hi Doug, On 02/07/2011 07:31 AM, Douglas Miller wrote:

*balanced *- if set to true, the MPI implementation may assume that all threads in the team will actively participate by making MPI calls (or calling MPI_HELPER_TEAM_LEAVE). If false, the implementation must allow for some threads spending the bulk of their time performing computation or other non-MPI activities.

With respect to the first point, threads don't need to "actively participate" from the user's perspective. The user just needs to be aware that all MPI calls might synchronize all or some of the threads. For the second point, I think this is incorrect. The threads spending the bulk of their time in computation is fine even in "balanced" mode. If the user's code is willing to accept the "synchronization overhead" during the MPI call, that's fine. For example, we already discussed that the following is valid: MPI_Helper_team_join() compute(random_time) pthread_barrier(); if (thread == 0) { MPI_SOMETHING(); } else { MPI_SOMETHING_ELSE(); } MPI_Helper_team_leave() But the following might be valid too (pthread_barrier is gone): MPI_Helper_team_join() compute(random_time) if (thread == 0) { MPI_SOMETHING(); } else { MPI_SOMETHING_ELSE(); } MPI_Helper_team_leave() In the second example, if the MPI implementation internally decides to synchronize the threads, that's fine, since the application would have done that anyway. This should be left to the user. The only thing that will be invalid is: MPI_Helper_team_join() compute(random_time) if (thread == 0) { MPI_SOMETHING(); } pthread_barrier(); MPI_Helper_team_leave() This is invalid because the MPI implementation is expecting to synchronize with other threads during the MPI_SOMETHING call, but it can't as all other threads are blocked in pthread_barrier().

...also, wouldn't setting 'singular' to true also imply/force 'balanced' to true? It almost seems like we have a three-state value rather than two booleans. From strict to weak: (singular), (!singular,balanced), (!singular,!balanced). I'm not clear just how much difference there is between singular (true) and balanced (true). Maybe we need only 'balanced'?

Yes, 'singular' is a superset of 'balanced'. I'll point that out. But as we discussed in the telecon, 'singular' can give more benefit for implementations as they know that other MPI calls will not be scheduled, so it's easier to optimize. -- Pavan -- Pavan Balaji http://www.mcs.anl.gov/~balaji