Is creating two communicators prohibitive?

Sent from my iPhone

On Jul 23, 2018, at 8:02 AM, Junchao Zhang <jczhang@mcs.anl.gov> wrote:

Jeff,

 No. I do not want to send messages to other ranks (not in comm_dist_graph).

 My question is: I create a graph with MPI_Dist_graph_create_adjacent, which specifies a communication direction. I can do MPI_Neighbor_alltoallv on it. That is great.  But I also want to do backward communication (reverse the direction of the edges in graph). How can I do it without creating a new communicator?

The background of this question is: In PETSc, we have an operation called VecScatter, which scatters some entries of a parallel vector x to another parallel vector y. Sometime, we want to reverse the operation, to scatter y entries to x.

--Junchao Zhang

On Mon, Jul 23, 2018 at 9:49 AM, Jeff Hammond <jeff.science@gmail.com> wrote:

On Mon, Jul 23, 2018 at 7:45 AM, Junchao Zhang <jczhang@mcs.anl.gov> wrote:

Hello, Dan,
 I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not see one can send a message to a neighbor not specified in the graph created by for example, MPI_Dist_graph_create_adjacent.

Why would you expect this?  The whole point of graph communicators is to specify the communication topology so that it can be optimized.  If you want to communicate with another ranks, use the parent communicator that supports communication to all ranks.

Jeff

 

--Junchao Zhang

On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes@epcc.ed.ac.uk> wrote:

Hi Junchao Zhang,

My understanding of the current API for MPI-3.1 is that:

1) the virtual topology does not actually restrict communication via the communicator to the edges specified in the topology - messages can be sent along any edge in either direction, and even between pairs of processes for which no edge was specified.

2) the virtual topology can be specified as a symmetric graph - for every ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to A) can be included as well.

3) there is already language in the MPI Standard regarding how MPI handles symmetric and non-symmetric graph topologies for neighbourhood collective operations.

Thus, there is no need to create two distributed graph topology communicators to achieve ‘forward and backward communication along the edges’.

Cheers,

Dan.

—
Dr Daniel Holmes PhD
Applications Consultant in HPC Research
d.holmes@epcc.ed.ac.uk
Phone: +44 (0) 131 651 3465
Mobile: +44 (0) 7940 524 088
Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD

—

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

—

On 15 Jul 2018, at 10:42, Anthony Skjellum <tony@runtimecomputing.com> wrote:

Hi, I just saw this. 

We definitely need to consider this concern.  

I also need to go review the APIs.

Thanks,

Tony

On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang@mcs.anl.gov> wrote:

 I want to try MPI neighborhood collectives. I have a communication graph and want to do both forward and backward communication along the edges. With current APIs, it looks I need to create two comm_dist_graphs. It is a waste since MPI implementations can do similar optimizations for the two.

 Should MPI support this scenario?  Thanks.

--Junchao Zhang

_______________________________________________
mpiwg-coll mailing list
mpiwg-coll@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll

--

Tony Skjellum, PhD
RunTime Computing Solutions, LLC
tony@runtimecomputing.com
direct: +1-205-918-7514
cell: +1-205-807-4968

_______________________________________________
mpiwg-coll mailing list
mpiwg-coll@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

_______________________________________________
mpiwg-coll mailing list
mpiwg-coll@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll

_______________________________________________
mpiwg-coll mailing list
mpiwg-coll@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll

--

Jeff Hammond
jeff.science@gmail.com
http://jeffhammond.github.io/

_______________________________________________
mpiwg-coll mailing list
mpiwg-coll@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll

_______________________________________________
mpiwg-coll mailing list
mpiwg-coll@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll

_______________________________________________
mpiwg-coll mailing list
mpiwg-coll@lists.mpi-forum.org
https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll