Hi Junchao Zhang, You are correct - the neighbourhood collectives send messages to (and receive messages from) only neighbouring MPI processes, as defined by the virtual topology. However, it is possible to use point-to-point, single-sided, or normal collective functions to send (and receive) messages between *any* MPI processes in a communicator, irrespective of the presence/structure of a virtual topology. For your use-case (if I understand it correctly), you should specify a symmetric graph - each process specifies to MPI_DIST_GRAPH_CREATE_ADJACENT all the processes it wishes to communicate with (both sending and receiving) during subsequent calls to MPI_NEIGHBOR_ALLTOALLV. Cheers, Dan. — Dr Daniel Holmes PhD Applications Consultant in HPC Research [email protected]<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 23 Jul 2018, at 15:45, Junchao Zhang <[email protected]<mailto:[email protected]>> wrote: Hello, Dan, I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not see one can send a message to a neighbor not specified in the graph created by for example, MPI_Dist_graph_create_adjacent. --Junchao Zhang On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <[email protected]<mailto:[email protected]>> wrote: Hi Junchao Zhang, My understanding of the current API for MPI-3.1 is that: 1) the virtual topology does not actually restrict communication via the communicator to the edges specified in the topology - messages can be sent along any edge in either direction, and even between pairs of processes for which no edge was specified. 2) the virtual topology can be specified as a symmetric graph - for every ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to A) can be included as well. 3) there is already language in the MPI Standard regarding how MPI handles symmetric and non-symmetric graph topologies for neighbourhood collective operations. Thus, there is no need to create two distributed graph topology communicators to achieve ‘forward and backward communication along the edges’. Cheers, Dan. — Dr Daniel Holmes PhD Applications Consultant in HPC Research [email protected]<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 15 Jul 2018, at 10:42, Anthony Skjellum <[email protected]<mailto:[email protected]>> wrote: Hi, I just saw this. We definitely need to consider this concern. I also need to go review the APIs. Thanks, Tony On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <[email protected]<mailto:[email protected]>> wrote: I want to try MPI neighborhood collectives. I have a communication graph and want to do both forward and backward communication along the edges. With current APIs, it looks I need to create two comm_dist_graphs. It is a waste since MPI implementations can do similar optimizations for the two. Should MPI support this scenario? Thanks. --Junchao Zhang _______________________________________________ mpiwg-coll mailing list [email protected]<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll -- Tony Skjellum, PhD RunTime Computing Solutions, LLC [email protected]<mailto:[email protected]> direct: +1-205-918-7514 cell: +1-205-807-4968 _______________________________________________ mpiwg-coll mailing list [email protected]<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ mpiwg-coll mailing list [email protected]<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll _______________________________________________ mpiwg-coll mailing list [email protected]<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.