Rolf: Your proposal is basically fine with me. Wording comments below. Bronis On Tue, 8 Mar 2011, Rolf Rabenseifner wrote:

Summary of my proposal: -----------------------

I propose to clarify: MPI-2.2, page 248, line 33 reads

If ndims is zero then a zero-dimensional Cartesian topology is created.

but should read

If ndims > 0 and the product of dims[i] is zero then MPI_COMM_NULL is returned in all processes.

The above sentence is fine.

If ndims is zero then and in each process, in comm_cart a communicator is retunred that contains only the process itself and that is accociated with a zero-dimensional Cartesian topology (i.e., comm_cart would be the same if all processes in the group of comm would have called MPI_CART_CREATE with ndims=0 and MPI_COMM_SELF as input communicator).

The above sentence has some typos plus the beginning of it does not parse properly. Replace with: If ndims is zero then a communicator that contains only the local process accociated with a zero-dimensional Cartesian topology is returned in comm_cart in each process (i.e., the same comm_cart is returned in each process in the group of comm as if the process called MPI_CART_CREATE with ndims = 0 and comm_old = MPI_COMM_SELF). and

---------------------- Details and reasons:

I agree that we should clarify the standard. To be compatible with existing implementations (they may handle it as an error), I would propose:

Add in MPI-2.2, page 248, line 34, before "If ndims is zero":

If ndims > 0 and the product of dims[i] is zero then MPI_COMM_NULL is returned in all processes.

This solution is in analogy to the definition of MPI_GRAPH_CREATE in the case of nnodes==0, see MPI-2.2, page 250, lines 30-31. (For MPI_DIST_GRAPH_CREATE(_ADJACENT), we decided another solution: Here, "isolated processes" are allowed, see MPI-2.2, page 253, lines 30-33, and page page 255, lines 15-17.

Additionally, I propose to clarify the next sentence, i.e., MPI-2.2, page 248, line 33: "If ndims is zero then a zero-dimensional Cartesian topology is created."

Relevant is only the text in MPI_CART_CREATE and MPI_CART_SUB. The idea behind this additions in the standardization of MPI-2.2 was, that MPI_CART_SUB had not forbidden that all remain_dims[i] are false, and threfore implementations had to return zero-dimensional single-process-communicators in all processes.

In MPI_CART_CREATE, the outcome is not such clear, because the number of processes that return such a zero-dimensional topology (i.e. a point), - can be ONE (and ann others return MPI_COMM_NULL) - or SIZE(comm).

The idea behind the new wording in MPI-2.2 was clearly to have it identical to the outcome of MPI_COMM_SPLIT, i.e., all process return such a zero-dimensional topology.

Therefore, I propose to clarify: MPI-2.2, page 248, line 33 reads

If ndims is zero then a zero-dimensional Cartesian topology is created.

but should read

If ndims > 0 and the product of dims[i] is zero then MPI_COMM_NULL is returned in all processes. If ndims is zero then and in each process, comm_cart contains only the process itself and comm_cart is accociated with a zero-dimensional Cartesian topology (i.e., the outcme is identical to calling this routine by all processes in the group of comm, but with MPI_COMM_SELF as input communicator).

Related lines to "zero-dimensional" are - MPI_CARTDIM_GET, MPI-2.2, page 259, lines 35-37 - MPI_CART_RANK, MPI-2.2, page 260, lines 45-46 - MPI_CART_COORDS, MPI-2.2, page 261, lines 19-20 - MPI_CART_SHIFT, MPI-2.2, page 266, lines 1-4 - MPI_CART_SUB, MPI-2.2, page 267, lines 1-4

Best regards Rolf

----- Original Message -----

...

From: "Bronis R. de Supinski" <[email protected]> To: "Main MPI Forum mailing list" <[email protected]> Sent: Monday, March 7, 2011 4:22:09 PM Subject: Re: [Mpi-forum] Question on Cartesian Topologies Michael:

I do not agree with your inference. The values in the dims array are not ranks so you cannot extend rank restrictions to them

I can envision algorithms for which providing 0 for a dimension indicates that the communicator should be empty. It would be inefficient for the application to check if any dimension is 0. As the MPICH-2 implementation demonstrates, this choice can be supported. I'll admit that your interpretation is reasonable although I think returning MPI_COMM_NULL is better. I also agree that an error should be returned if any of the dims is negative.

Regardless, the issue should not be left to inference. We need to update the standard to clarify this issue.

Bronis

On Mon, 7 Mar 2011, Michael Raymond wrote:

...

While there's nothing specified in the definition of MPI_Cart_create(), you might be able to infer this from the definition of MPI_Cart_rank(). It specifies that a coordinate < 0 or >= the size of that dimension is out of range. 0 is >= 0, so it must be out of range.

Tobias Hilbrich wrote:

...

Hi all,

LANL, LLNL and the University of Dresden are working on a follow up tool of the runtime correctness checking tools Marmot and Umpire, which is done in cooperation by developers of both tools.

We have some issues with basic correctness checks for integers. Were for most of the handles used in MPI calls there are fairly clear rules on what is allowed and what not, there are sometimes close to no rules for integer arguments. Most of the time common sense helps to come up with a reasonable assumption, but for the definition of cartesian communicators we are somewhat stuck, maybe you can help. (Maybe we missed some lines in the standard or are not aware of some base rule)

What is unclear to us is "what is a correct definition for a cartesian topology?", we found no clear definition for that. So basically we care about the "ndims" and "dims" values in calls like MPI_Cart_create. Is it allowed to specify a value of 0 for one of the entries in the "dims" array? The standard specifies no restrictions in the text following MPI_Cart_create, I am referring to MPI-2.2 here. E.g. is the following correct or not:

int dims[3] dims[0] = 1; dims[1] = 1; dims[2] = 0; /*Is this ok/suspicious/erroneous ?*/ MPI_Cart_create( ... , 3 /*ndims*/, dims , ...);

Our little testing revealed that it causes OpenMPI, LAM-MPI, MPICH-1 to crash, whereas MPICH-2 returns MPI_COMM_NULL here.

many thanks, Tobias

P.S.: This is probably not the last os such questions, if this is a bad way to ask these we would be glad to get suggestions on how to raise them. A tool that validates whether an MPI application behaves according to the standard or not, hopefully provides some motivates to think about these troublesome questions ;)

-- Dipl.-Inf. Tobias Hilbrich Wissenschaftlicher Mitarbeiter

Technische Universitaet Dresden Zentrum fuer Informationsdienste und Hochleistungsrechnen (ZIH) (Center for Information Services and High Performance Computing (ZIH)) Interdisziplinäre Anwenderunterstützung und Koordination (Interdisciplinary Application Development and Coordination) 01062 Dresden Tel.: +49 (351) 463-32041 Fax: +49 (351) 463-37773 E-Mail: [email protected]

_______________________________________________ mpi-forum mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

-- Michael A. Raymond Message Passing Toolkit Team Silicon Graphics Inc (651) 683-3434

_______________________________________________ mpi-forum mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

_______________________________________________ mpi-forum mailing list [email protected] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi-forum

-- Dr. Rolf Rabenseifner . . . . . . . . . .. email [email protected] High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . (Office: Allmandring 30)