July 2018 - mpiwg-coll - mailman3.cels.anl.gov

Re: [mpiwg-coll] backward communication in neighborhood collectives
by Junchao Zhang 23 Jul '18

23 Jul '18

No, not at all. --Junchao Zhang On Mon, Jul 23, 2018 at 11:04 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > Is creating two communicators prohibitive? > > Sent from my iPhone > > On Jul 23, 2018, at 8:02 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > > Jeff, > No. I do not want to send messages to other ranks (not in > comm_dist_graph). > My question is: I create a graph with MPI_Dist_graph_create_adjacent, > which specifies a communication direction. I can do MPI_Neighbor_alltoallv > on it. That is great. But I also want to do backward communication > (reverse the direction of the edges in graph). How can I do it without > creating a new communicator? > The background of this question is: In PETSc, we have an operation called > VecScatter, which scatters some entries of a parallel vector x to another > parallel vector y. Sometime, we want to reverse the operation, to scatter y > entries to x. > > > --Junchao Zhang > > On Mon, Jul 23, 2018 at 9:49 AM, Jeff Hammond <jeff.science(a)gmail.com> > wrote: > >> On Mon, Jul 23, 2018 at 7:45 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> >> wrote: >> >>> Hello, Dan, >>> I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not >>> see one can send a message to a neighbor not specified in the graph created >>> by for example, MPI_Dist_graph_create_adjacent. >>> >> >> Why would you expect this? The whole point of graph communicators is to >> specify the communication topology so that it can be optimized. If you >> want to communicate with another ranks, use the parent communicator that >> supports communication to all ranks. >> >> Jeff >> >> >>> --Junchao Zhang >>> >>> On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> >>> wrote: >>> >>>> Hi Junchao Zhang, >>>> >>>> My understanding of the current API for MPI-3.1 is that: >>>> >>>> 1) the virtual topology does not actually restrict communication via >>>> the communicator to the edges specified in the topology - messages can be >>>> sent along any edge in either direction, and even between pairs of >>>> processes for which no edge was specified. >>>> >>>> 2) the virtual topology can be specified as a symmetric graph - for >>>> every ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B >>>> to A) can be included as well. >>>> >>>> 3) there is already language in the MPI Standard regarding how MPI >>>> handles symmetric and non-symmetric graph topologies for neighbourhood >>>> collective operations. >>>> >>>> Thus, there is no need to create two distributed graph topology >>>> communicators to achieve ‘forward and backward communication along the >>>> edges’. >>>> >>>> Cheers, >>>> Dan. >>>> — >>>> Dr Daniel Holmes PhD >>>> Applications Consultant in HPC Research >>>> d.holmes(a)epcc.ed.ac.uk >>>> Phone: +44 (0) 131 651 3465 >>>> Mobile: +44 (0) 7940 524 088 >>>> Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD >>>> — >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> — >>>> >>>> On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com> >>>> wrote: >>>> >>>> Hi, I just saw this. >>>> >>>> We definitely need to consider this concern. >>>> >>>> I also need to go review the APIs. >>>> >>>> Thanks, >>>> Tony >>>> >>>> >>>> On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> >>>> wrote: >>>> >>>>> I want to try MPI neighborhood collectives. I have a communication >>>>> graph and want to do both forward and backward communication along the >>>>> edges. With current APIs, it looks I need to create two comm_dist_graphs. >>>>> It is a waste since MPI implementations can do similar optimizations for >>>>> the two. >>>>> Should MPI support this scenario? Thanks. >>>>> >>>>> --Junchao Zhang >>>>> >>>>> _______________________________________________ >>>>> mpiwg-coll mailing list >>>>> mpiwg-coll(a)lists.mpi-forum.org >>>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>>>> >>>>> >>>> >>>> >>>> -- >>>> Tony Skjellum, PhD >>>> RunTime Computing Solutions, LLC >>>> tony(a)runtimecomputing.com >>>> direct: +1-205-918-7514 >>>> cell: +1-205-807-4968 >>>> _______________________________________________ >>>> mpiwg-coll mailing list >>>> mpiwg-coll(a)lists.mpi-forum.org >>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>>> >>>> >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> >>>> _______________________________________________ >>>> mpiwg-coll mailing list >>>> mpiwg-coll(a)lists.mpi-forum.org >>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>>> >>>> >>> >>> _______________________________________________ >>> mpiwg-coll mailing list >>> mpiwg-coll(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>> >>> >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> http://jeffhammond.github.io/ >> >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> >> > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > > > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > >

1 0

Re: [mpiwg-coll] backward communication in neighborhood collectives
by Jeff Hammond 23 Jul '18

23 Jul '18

Is creating two communicators prohibitive? Sent from my iPhone > On Jul 23, 2018, at 8:02 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > > Jeff, > No. I do not want to send messages to other ranks (not in comm_dist_graph). > My question is: I create a graph with MPI_Dist_graph_create_adjacent, which specifies a communication direction. I can do MPI_Neighbor_alltoallv on it. That is great. But I also want to do backward communication (reverse the direction of the edges in graph). How can I do it without creating a new communicator? > The background of this question is: In PETSc, we have an operation called VecScatter, which scatters some entries of a parallel vector x to another parallel vector y. Sometime, we want to reverse the operation, to scatter y entries to x. > > > --Junchao Zhang > >> On Mon, Jul 23, 2018 at 9:49 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>> On Mon, Jul 23, 2018 at 7:45 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: >>> Hello, Dan, >>> I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not see one can send a message to a neighbor not specified in the graph created by for example, MPI_Dist_graph_create_adjacent. >> >> Why would you expect this? The whole point of graph communicators is to specify the communication topology so that it can be optimized. If you want to communicate with another ranks, use the parent communicator that supports communication to all ranks. >> >> Jeff >> >>> --Junchao Zhang >>> >>>> On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> wrote: >>>> Hi Junchao Zhang, >>>> >>>> My understanding of the current API for MPI-3.1 is that: >>>> >>>> 1) the virtual topology does not actually restrict communication via the communicator to the edges specified in the topology - messages can be sent along any edge in either direction, and even between pairs of processes for which no edge was specified. >>>> >>>> 2) the virtual topology can be specified as a symmetric graph - for every ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to A) can be included as well. >>>> >>>> 3) there is already language in the MPI Standard regarding how MPI handles symmetric and non-symmetric graph topologies for neighbourhood collective operations. >>>> >>>> Thus, there is no need to create two distributed graph topology communicators to achieve ‘forward and backward communication along the edges’. >>>> >>>> Cheers, >>>> Dan. >>>> — >>>> Dr Daniel Holmes PhD >>>> Applications Consultant in HPC Research >>>> d.holmes(a)epcc.ed.ac.uk >>>> Phone: +44 (0) 131 651 3465 >>>> Mobile: +44 (0) 7940 524 088 >>>> Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD >>>> — >>>> The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. >>>> — >>>> >>>>> On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com> wrote: >>>>> >>>>> Hi, I just saw this. >>>>> >>>>> We definitely need to consider this concern. >>>>> >>>>> I also need to go review the APIs. >>>>> >>>>> Thanks, >>>>> Tony >>>>> >>>>> >>>>>> On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: >>>>>> I want to try MPI neighborhood collectives. I have a communication graph and want to do both forward and backward communication along the edges. With current APIs, it looks I need to create two comm_dist_graphs. It is a waste since MPI implementations can do similar optimizations for the two. >>>>>> Should MPI support this scenario? Thanks. >>>>>> >>>>>> --Junchao Zhang >>>>>> >>>>>> _______________________________________________ >>>>>> mpiwg-coll mailing list >>>>>> mpiwg-coll(a)lists.mpi-forum.org >>>>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Tony Skjellum, PhD >>>>> RunTime Computing Solutions, LLC >>>>> tony(a)runtimecomputing.com >>>>> direct: +1-205-918-7514 >>>>> cell: +1-205-807-4968 >>>>> _______________________________________________ >>>>> mpiwg-coll mailing list >>>>> mpiwg-coll(a)lists.mpi-forum.org >>>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>>> >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> >>>> _______________________________________________ >>>> mpiwg-coll mailing list >>>> mpiwg-coll(a)lists.mpi-forum.org >>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>>> >>> >>> >>> _______________________________________________ >>> mpiwg-coll mailing list >>> mpiwg-coll(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>> >> >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> http://jeffhammond.github.io/ >> >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> > > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll

1 0

Re: [mpiwg-coll] backward communication in neighborhood collectives
by Junchao Zhang 23 Jul '18

23 Jul '18

I do not want to combine the forward/backward graphs. Creating two communicators is OK with me. My original thought was: with one graph, if MPI also supports its reverse operation, then MPI implementations can do symmetric optimizations and save some resources. --Junchao Zhang On Mon, Jul 23, 2018 at 10:35 AM, Jed Brown <jed(a)jedbrown.org> wrote: > For ghost value updates in domain decomposition PDE solvers (usually the > most performance-sensitive use of VecScatter), the graph will usually be > symmetric. It is often nonsymmetric for multilevel restriction and > prolongation and for some field-split methods in multiphysics > applications, in which case I think you'll want to make two neighborhood > communicators rather than the union of the forward and reverse/transpose > operations with some empty messages. > > Junchao Zhang <jczhang(a)mcs.anl.gov> writes: > > > Dan, > > Using your approach, I have to specify zero messages in arguments to > > MPI_NEIGHBOR_ALLTOALLV, > > because I only do communication in one direction in one time. Also, in > my > > graph, there may have edges A->B and B->A. They may have different > weights. > > Using your approach, it is quite confusing. > > One uses MPI_NEIGHBOR_ALLTOALLV to have better performance. We have to > > keep it in mind. > > > > --Junchao Zhang > > > > On Mon, Jul 23, 2018 at 9:59 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > > wrote: > > > >> Hi Junchao Zhang, > >> > >> You are correct - the neighbourhood collectives send messages to (and > >> receive messages from) only neighbouring MPI processes, as defined by > the > >> virtual topology. However, it is possible to use point-to-point, > >> single-sided, or normal collective functions to send (and receive) > messages > >> between *any* MPI processes in a communicator, irrespective of the > >> presence/structure of a virtual topology. > >> > >> For your use-case (if I understand it correctly), you should specify a > >> symmetric graph - each process specifies to > MPI_DIST_GRAPH_CREATE_ADJACENT > >> all the processes it wishes to communicate with (both sending and > >> receiving) during subsequent calls to MPI_NEIGHBOR_ALLTOALLV. > >> > >> Cheers, > >> Dan. > >> — > >> Dr Daniel Holmes PhD > >> Applications Consultant in HPC Research > >> d.holmes(a)epcc.ed.ac.uk > >> Phone: +44 (0) 131 651 3465 > >> Mobile: +44 (0) 7940 524 088 > >> Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD > >> — > >> The University of Edinburgh is a charitable body, registered in > Scotland, > >> with registration number SC005336. > >> — > >> > >> On 23 Jul 2018, at 15:45, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > >> > >> Hello, Dan, > >> I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not > >> see one can send a message to a neighbor not specified in the graph > created > >> by for example, MPI_Dist_graph_create_adjacent. > >> > >> --Junchao Zhang > >> > >> On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > >> wrote: > >> > >>> Hi Junchao Zhang, > >>> > >>> My understanding of the current API for MPI-3.1 is that: > >>> > >>> 1) the virtual topology does not actually restrict communication via > the > >>> communicator to the edges specified in the topology - messages can be > sent > >>> along any edge in either direction, and even between pairs of > processes for > >>> which no edge was specified. > >>> > >>> 2) the virtual topology can be specified as a symmetric graph - for > every > >>> ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to > A) > >>> can be included as well. > >>> > >>> 3) there is already language in the MPI Standard regarding how MPI > >>> handles symmetric and non-symmetric graph topologies for neighbourhood > >>> collective operations. > >>> > >>> Thus, there is no need to create two distributed graph topology > >>> communicators to achieve ‘forward and backward communication along the > >>> edges’. > >>> > >>> Cheers, > >>> Dan. > >>> — > >>> Dr Daniel Holmes PhD > >>> Applications Consultant in HPC Research > >>> d.holmes(a)epcc.ed.ac.uk > >>> Phone: +44 (0) 131 651 3465 > >>> Mobile: +44 (0) 7940 524 088 > >>> Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD > >>> — > >>> The University of Edinburgh is a charitable body, registered in > Scotland, > >>> with registration number SC005336. > >>> — > >>> > >>> On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com> > >>> wrote: > >>> > >>> Hi, I just saw this. > >>> > >>> We definitely need to consider this concern. > >>> > >>> I also need to go review the APIs. > >>> > >>> Thanks, > >>> Tony > >>> > >>> > >>> On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> > >>> wrote: > >>> > >>>> I want to try MPI neighborhood collectives. I have a communication > >>>> graph and want to do both forward and backward communication along the > >>>> edges. With current APIs, it looks I need to create two > comm_dist_graphs. > >>>> It is a waste since MPI implementations can do similar optimizations > for > >>>> the two. > >>>> Should MPI support this scenario? Thanks. > >>>> > >>>> --Junchao Zhang > >>>> > >>>> _______________________________________________ > >>>> mpiwg-coll mailing list > >>>> mpiwg-coll(a)lists.mpi-forum.org > >>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > >>>> > >>>> > >>> > >>> > >>> -- > >>> Tony Skjellum, PhD > >>> RunTime Computing Solutions, LLC > >>> tony(a)runtimecomputing.com > >>> direct: +1-205-918-7514 > >>> cell: +1-205-807-4968 > >>> _______________________________________________ > >>> mpiwg-coll mailing list > >>> mpiwg-coll(a)lists.mpi-forum.org > >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > >>> > >>> > >>> > >>> The University of Edinburgh is a charitable body, registered in > >>> Scotland, with registration number SC005336. > >>> > >>> _______________________________________________ > >>> mpiwg-coll mailing list > >>> mpiwg-coll(a)lists.mpi-forum.org > >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > >>> > >>> > >> _______________________________________________ > >> mpiwg-coll mailing list > >> mpiwg-coll(a)lists.mpi-forum.org > >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > >> > >> > >> > >> The University of Edinburgh is a charitable body, registered in > >> Scotland, with registration number SC005336. > >> > >> _______________________________________________ > >> mpiwg-coll mailing list > >> mpiwg-coll(a)lists.mpi-forum.org > >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > >> > >> > > _______________________________________________ > > mpiwg-coll mailing list > > mpiwg-coll(a)lists.mpi-forum.org > > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >

1 0

Re: [mpiwg-coll] backward communication in neighborhood collectives
by Jed Brown 23 Jul '18

23 Jul '18

For ghost value updates in domain decomposition PDE solvers (usually the most performance-sensitive use of VecScatter), the graph will usually be symmetric. It is often nonsymmetric for multilevel restriction and prolongation and for some field-split methods in multiphysics applications, in which case I think you'll want to make two neighborhood communicators rather than the union of the forward and reverse/transpose operations with some empty messages. Junchao Zhang <jczhang(a)mcs.anl.gov> writes: > Dan, > Using your approach, I have to specify zero messages in arguments to > MPI_NEIGHBOR_ALLTOALLV, > because I only do communication in one direction in one time. Also, in my > graph, there may have edges A->B and B->A. They may have different weights. > Using your approach, it is quite confusing. > One uses MPI_NEIGHBOR_ALLTOALLV to have better performance. We have to > keep it in mind. > > --Junchao Zhang > > On Mon, Jul 23, 2018 at 9:59 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > wrote: > >> Hi Junchao Zhang, >> >> You are correct - the neighbourhood collectives send messages to (and >> receive messages from) only neighbouring MPI processes, as defined by the >> virtual topology. However, it is possible to use point-to-point, >> single-sided, or normal collective functions to send (and receive) messages >> between *any* MPI processes in a communicator, irrespective of the >> presence/structure of a virtual topology. >> >> For your use-case (if I understand it correctly), you should specify a >> symmetric graph - each process specifies to MPI_DIST_GRAPH_CREATE_ADJACENT >> all the processes it wishes to communicate with (both sending and >> receiving) during subsequent calls to MPI_NEIGHBOR_ALLTOALLV. >> >> Cheers, >> Dan. >> — >> Dr Daniel Holmes PhD >> Applications Consultant in HPC Research >> d.holmes(a)epcc.ed.ac.uk >> Phone: +44 (0) 131 651 3465 >> Mobile: +44 (0) 7940 524 088 >> Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD >> — >> The University of Edinburgh is a charitable body, registered in Scotland, >> with registration number SC005336. >> — >> >> On 23 Jul 2018, at 15:45, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: >> >> Hello, Dan, >> I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not >> see one can send a message to a neighbor not specified in the graph created >> by for example, MPI_Dist_graph_create_adjacent. >> >> --Junchao Zhang >> >> On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> >> wrote: >> >>> Hi Junchao Zhang, >>> >>> My understanding of the current API for MPI-3.1 is that: >>> >>> 1) the virtual topology does not actually restrict communication via the >>> communicator to the edges specified in the topology - messages can be sent >>> along any edge in either direction, and even between pairs of processes for >>> which no edge was specified. >>> >>> 2) the virtual topology can be specified as a symmetric graph - for every >>> ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to A) >>> can be included as well. >>> >>> 3) there is already language in the MPI Standard regarding how MPI >>> handles symmetric and non-symmetric graph topologies for neighbourhood >>> collective operations. >>> >>> Thus, there is no need to create two distributed graph topology >>> communicators to achieve ‘forward and backward communication along the >>> edges’. >>> >>> Cheers, >>> Dan. >>> — >>> Dr Daniel Holmes PhD >>> Applications Consultant in HPC Research >>> d.holmes(a)epcc.ed.ac.uk >>> Phone: +44 (0) 131 651 3465 >>> Mobile: +44 (0) 7940 524 088 >>> Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD >>> — >>> The University of Edinburgh is a charitable body, registered in Scotland, >>> with registration number SC005336. >>> — >>> >>> On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com> >>> wrote: >>> >>> Hi, I just saw this. >>> >>> We definitely need to consider this concern. >>> >>> I also need to go review the APIs. >>> >>> Thanks, >>> Tony >>> >>> >>> On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> >>> wrote: >>> >>>> I want to try MPI neighborhood collectives. I have a communication >>>> graph and want to do both forward and backward communication along the >>>> edges. With current APIs, it looks I need to create two comm_dist_graphs. >>>> It is a waste since MPI implementations can do similar optimizations for >>>> the two. >>>> Should MPI support this scenario? Thanks. >>>> >>>> --Junchao Zhang >>>> >>>> _______________________________________________ >>>> mpiwg-coll mailing list >>>> mpiwg-coll(a)lists.mpi-forum.org >>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>>> >>>> >>> >>> >>> -- >>> Tony Skjellum, PhD >>> RunTime Computing Solutions, LLC >>> tony(a)runtimecomputing.com >>> direct: +1-205-918-7514 >>> cell: +1-205-807-4968 >>> _______________________________________________ >>> mpiwg-coll mailing list >>> mpiwg-coll(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>> >>> >>> >>> The University of Edinburgh is a charitable body, registered in >>> Scotland, with registration number SC005336. >>> >>> _______________________________________________ >>> mpiwg-coll mailing list >>> mpiwg-coll(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>> >>> >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> >> >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> >> > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll

1 0

Re: [mpiwg-coll] backward communication in neighborhood collectives
by Junchao Zhang 23 Jul '18

23 Jul '18

Dan, Using your approach, I have to specify zero messages in arguments to MPI_NEIGHBOR_ALLTOALLV, because I only do communication in one direction in one time. Also, in my graph, there may have edges A->B and B->A. They may have different weights. Using your approach, it is quite confusing. One uses MPI_NEIGHBOR_ALLTOALLV to have better performance. We have to keep it in mind. --Junchao Zhang On Mon, Jul 23, 2018 at 9:59 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> wrote: > Hi Junchao Zhang, > > You are correct - the neighbourhood collectives send messages to (and > receive messages from) only neighbouring MPI processes, as defined by the > virtual topology. However, it is possible to use point-to-point, > single-sided, or normal collective functions to send (and receive) messages > between *any* MPI processes in a communicator, irrespective of the > presence/structure of a virtual topology. > > For your use-case (if I understand it correctly), you should specify a > symmetric graph - each process specifies to MPI_DIST_GRAPH_CREATE_ADJACENT > all the processes it wishes to communicate with (both sending and > receiving) during subsequent calls to MPI_NEIGHBOR_ALLTOALLV. > > Cheers, > Dan. > — > Dr Daniel Holmes PhD > Applications Consultant in HPC Research > d.holmes(a)epcc.ed.ac.uk > Phone: +44 (0) 131 651 3465 > Mobile: +44 (0) 7940 524 088 > Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD > — > The University of Edinburgh is a charitable body, registered in Scotland, > with registration number SC005336. > — > > On 23 Jul 2018, at 15:45, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > > Hello, Dan, > I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not > see one can send a message to a neighbor not specified in the graph created > by for example, MPI_Dist_graph_create_adjacent. > > --Junchao Zhang > > On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > wrote: > >> Hi Junchao Zhang, >> >> My understanding of the current API for MPI-3.1 is that: >> >> 1) the virtual topology does not actually restrict communication via the >> communicator to the edges specified in the topology - messages can be sent >> along any edge in either direction, and even between pairs of processes for >> which no edge was specified. >> >> 2) the virtual topology can be specified as a symmetric graph - for every >> ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to A) >> can be included as well. >> >> 3) there is already language in the MPI Standard regarding how MPI >> handles symmetric and non-symmetric graph topologies for neighbourhood >> collective operations. >> >> Thus, there is no need to create two distributed graph topology >> communicators to achieve ‘forward and backward communication along the >> edges’. >> >> Cheers, >> Dan. >> — >> Dr Daniel Holmes PhD >> Applications Consultant in HPC Research >> d.holmes(a)epcc.ed.ac.uk >> Phone: +44 (0) 131 651 3465 >> Mobile: +44 (0) 7940 524 088 >> Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD >> — >> The University of Edinburgh is a charitable body, registered in Scotland, >> with registration number SC005336. >> — >> >> On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com> >> wrote: >> >> Hi, I just saw this. >> >> We definitely need to consider this concern. >> >> I also need to go review the APIs. >> >> Thanks, >> Tony >> >> >> On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> >> wrote: >> >>> I want to try MPI neighborhood collectives. I have a communication >>> graph and want to do both forward and backward communication along the >>> edges. With current APIs, it looks I need to create two comm_dist_graphs. >>> It is a waste since MPI implementations can do similar optimizations for >>> the two. >>> Should MPI support this scenario? Thanks. >>> >>> --Junchao Zhang >>> >>> _______________________________________________ >>> mpiwg-coll mailing list >>> mpiwg-coll(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>> >>> >> >> >> -- >> Tony Skjellum, PhD >> RunTime Computing Solutions, LLC >> tony(a)runtimecomputing.com >> direct: +1-205-918-7514 >> cell: +1-205-807-4968 >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> >> >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> >> > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > > > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > >

1 0

Re: [mpiwg-coll] backward communication in neighborhood collectives
by Junchao Zhang 23 Jul '18

23 Jul '18

Jeff, No. I do not want to send messages to other ranks (not in comm_dist_graph). My question is: I create a graph with MPI_Dist_graph_create_adjacent, which specifies a communication direction. I can do MPI_Neighbor_alltoallv on it. That is great. But I also want to do backward communication (reverse the direction of the edges in graph). How can I do it without creating a new communicator? The background of this question is: In PETSc, we have an operation called VecScatter, which scatters some entries of a parallel vector x to another parallel vector y. Sometime, we want to reverse the operation, to scatter y entries to x. --Junchao Zhang On Mon, Jul 23, 2018 at 9:49 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > On Mon, Jul 23, 2018 at 7:45 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> > wrote: > >> Hello, Dan, >> I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not >> see one can send a message to a neighbor not specified in the graph created >> by for example, MPI_Dist_graph_create_adjacent. >> > > Why would you expect this? The whole point of graph communicators is to > specify the communication topology so that it can be optimized. If you > want to communicate with another ranks, use the parent communicator that > supports communication to all ranks. > > Jeff > > >> --Junchao Zhang >> >> On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> >> wrote: >> >>> Hi Junchao Zhang, >>> >>> My understanding of the current API for MPI-3.1 is that: >>> >>> 1) the virtual topology does not actually restrict communication via the >>> communicator to the edges specified in the topology - messages can be sent >>> along any edge in either direction, and even between pairs of processes for >>> which no edge was specified. >>> >>> 2) the virtual topology can be specified as a symmetric graph - for >>> every ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B >>> to A) can be included as well. >>> >>> 3) there is already language in the MPI Standard regarding how MPI >>> handles symmetric and non-symmetric graph topologies for neighbourhood >>> collective operations. >>> >>> Thus, there is no need to create two distributed graph topology >>> communicators to achieve ‘forward and backward communication along the >>> edges’. >>> >>> Cheers, >>> Dan. >>> — >>> Dr Daniel Holmes PhD >>> Applications Consultant in HPC Research >>> d.holmes(a)epcc.ed.ac.uk >>> Phone: +44 (0) 131 651 3465 >>> Mobile: +44 (0) 7940 524 088 >>> Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD >>> — >>> The University of Edinburgh is a charitable body, registered in >>> Scotland, with registration number SC005336. >>> — >>> >>> On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com> >>> wrote: >>> >>> Hi, I just saw this. >>> >>> We definitely need to consider this concern. >>> >>> I also need to go review the APIs. >>> >>> Thanks, >>> Tony >>> >>> >>> On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> >>> wrote: >>> >>>> I want to try MPI neighborhood collectives. I have a communication >>>> graph and want to do both forward and backward communication along the >>>> edges. With current APIs, it looks I need to create two comm_dist_graphs. >>>> It is a waste since MPI implementations can do similar optimizations for >>>> the two. >>>> Should MPI support this scenario? Thanks. >>>> >>>> --Junchao Zhang >>>> >>>> _______________________________________________ >>>> mpiwg-coll mailing list >>>> mpiwg-coll(a)lists.mpi-forum.org >>>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>>> >>>> >>> >>> >>> -- >>> Tony Skjellum, PhD >>> RunTime Computing Solutions, LLC >>> tony(a)runtimecomputing.com >>> direct: +1-205-918-7514 >>> cell: +1-205-807-4968 >>> _______________________________________________ >>> mpiwg-coll mailing list >>> mpiwg-coll(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>> >>> >>> >>> The University of Edinburgh is a charitable body, registered in >>> Scotland, with registration number SC005336. >>> >>> _______________________________________________ >>> mpiwg-coll mailing list >>> mpiwg-coll(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>> >>> >> >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> >> > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > >

1 0

Re: [mpiwg-coll] backward communication in neighborhood collectives
by HOLMES Daniel 23 Jul '18

23 Jul '18

Hi Junchao Zhang, You are correct - the neighbourhood collectives send messages to (and receive messages from) only neighbouring MPI processes, as defined by the virtual topology. However, it is possible to use point-to-point, single-sided, or normal collective functions to send (and receive) messages between *any* MPI processes in a communicator, irrespective of the presence/structure of a virtual topology. For your use-case (if I understand it correctly), you should specify a symmetric graph - each process specifies to MPI_DIST_GRAPH_CREATE_ADJACENT all the processes it wishes to communicate with (both sending and receiving) during subsequent calls to MPI_NEIGHBOR_ALLTOALLV. Cheers, Dan. — Dr Daniel Holmes PhD Applications Consultant in HPC Research d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 23 Jul 2018, at 15:45, Junchao Zhang <jczhang(a)mcs.anl.gov<mailto:[email protected]>> wrote: Hello, Dan, I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not see one can send a message to a neighbor not specified in the graph created by for example, MPI_Dist_graph_create_adjacent. --Junchao Zhang On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]>> wrote: Hi Junchao Zhang, My understanding of the current API for MPI-3.1 is that: 1) the virtual topology does not actually restrict communication via the communicator to the edges specified in the topology - messages can be sent along any edge in either direction, and even between pairs of processes for which no edge was specified. 2) the virtual topology can be specified as a symmetric graph - for every ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to A) can be included as well. 3) there is already language in the MPI Standard regarding how MPI handles symmetric and non-symmetric graph topologies for neighbourhood collective operations. Thus, there is no need to create two distributed graph topology communicators to achieve ‘forward and backward communication along the edges’. Cheers, Dan. — Dr Daniel Holmes PhD Applications Consultant in HPC Research d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com<mailto:[email protected]>> wrote: Hi, I just saw this. We definitely need to consider this concern. I also need to go review the APIs. Thanks, Tony On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov<mailto:[email protected]>> wrote: I want to try MPI neighborhood collectives. I have a communication graph and want to do both forward and backward communication along the edges. With current APIs, it looks I need to create two comm_dist_graphs. It is a waste since MPI implementations can do similar optimizations for the two. Should MPI support this scenario? Thanks. --Junchao Zhang _______________________________________________ mpiwg-coll mailing list mpiwg-coll(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll -- Tony Skjellum, PhD RunTime Computing Solutions, LLC tony(a)runtimecomputing.com<mailto:[email protected]> direct: +1-205-918-7514 cell: +1-205-807-4968 _______________________________________________ mpiwg-coll mailing list mpiwg-coll(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ mpiwg-coll mailing list mpiwg-coll(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll _______________________________________________ mpiwg-coll mailing list mpiwg-coll(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0

Re: [mpiwg-coll] backward communication in neighborhood collectives
by Jeff Hammond 23 Jul '18

23 Jul '18

On Mon, Jul 23, 2018 at 7:45 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > Hello, Dan, > I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not > see one can send a message to a neighbor not specified in the graph created > by for example, MPI_Dist_graph_create_adjacent. > Why would you expect this? The whole point of graph communicators is to specify the communication topology so that it can be optimized. If you want to communicate with another ranks, use the parent communicator that supports communication to all ranks. Jeff > --Junchao Zhang > > On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> > wrote: > >> Hi Junchao Zhang, >> >> My understanding of the current API for MPI-3.1 is that: >> >> 1) the virtual topology does not actually restrict communication via the >> communicator to the edges specified in the topology - messages can be sent >> along any edge in either direction, and even between pairs of processes for >> which no edge was specified. >> >> 2) the virtual topology can be specified as a symmetric graph - for every >> ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to A) >> can be included as well. >> >> 3) there is already language in the MPI Standard regarding how MPI >> handles symmetric and non-symmetric graph topologies for neighbourhood >> collective operations. >> >> Thus, there is no need to create two distributed graph topology >> communicators to achieve ‘forward and backward communication along the >> edges’. >> >> Cheers, >> Dan. >> — >> Dr Daniel Holmes PhD >> Applications Consultant in HPC Research >> d.holmes(a)epcc.ed.ac.uk >> Phone: +44 (0) 131 651 3465 >> Mobile: +44 (0) 7940 524 088 >> Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD >> — >> The University of Edinburgh is a charitable body, registered in Scotland, >> with registration number SC005336. >> — >> >> On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com> >> wrote: >> >> Hi, I just saw this. >> >> We definitely need to consider this concern. >> >> I also need to go review the APIs. >> >> Thanks, >> Tony >> >> >> On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> >> wrote: >> >>> I want to try MPI neighborhood collectives. I have a communication >>> graph and want to do both forward and backward communication along the >>> edges. With current APIs, it looks I need to create two comm_dist_graphs. >>> It is a waste since MPI implementations can do similar optimizations for >>> the two. >>> Should MPI support this scenario? Thanks. >>> >>> --Junchao Zhang >>> >>> _______________________________________________ >>> mpiwg-coll mailing list >>> mpiwg-coll(a)lists.mpi-forum.org >>> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >>> >>> >> >> >> -- >> Tony Skjellum, PhD >> RunTime Computing Solutions, LLC >> tony(a)runtimecomputing.com >> direct: +1-205-918-7514 >> cell: +1-205-807-4968 >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> >> >> >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. >> >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> >> > > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > > -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-coll] backward communication in neighborhood collectives
by Junchao Zhang 23 Jul '18

23 Jul '18

Hello, Dan, I am interested in MPI_Neighbor_alltoallv. From its arguments, I do not see one can send a message to a neighbor not specified in the graph created by for example, MPI_Dist_graph_create_adjacent. --Junchao Zhang On Mon, Jul 23, 2018 at 5:51 AM, HOLMES Daniel <d.holmes(a)epcc.ed.ac.uk> wrote: > Hi Junchao Zhang, > > My understanding of the current API for MPI-3.1 is that: > > 1) the virtual topology does not actually restrict communication via the > communicator to the edges specified in the topology - messages can be sent > along any edge in either direction, and even between pairs of processes for > which no edge was specified. > > 2) the virtual topology can be specified as a symmetric graph - for every > ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to A) > can be included as well. > > 3) there is already language in the MPI Standard regarding how MPI handles > symmetric and non-symmetric graph topologies for neighbourhood collective > operations. > > Thus, there is no need to create two distributed graph topology > communicators to achieve ‘forward and backward communication along the > edges’. > > Cheers, > Dan. > — > Dr Daniel Holmes PhD > Applications Consultant in HPC Research > d.holmes(a)epcc.ed.ac.uk > Phone: +44 (0) 131 651 3465 > Mobile: +44 (0) 7940 524 088 > Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD > — > The University of Edinburgh is a charitable body, registered in Scotland, > with registration number SC005336. > — > > On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com> > wrote: > > Hi, I just saw this. > > We definitely need to consider this concern. > > I also need to go review the APIs. > > Thanks, > Tony > > > On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> > wrote: > >> I want to try MPI neighborhood collectives. I have a communication graph >> and want to do both forward and backward communication along the edges. >> With current APIs, it looks I need to create two comm_dist_graphs. It is a >> waste since MPI implementations can do similar optimizations for the two. >> Should MPI support this scenario? Thanks. >> >> --Junchao Zhang >> >> _______________________________________________ >> mpiwg-coll mailing list >> mpiwg-coll(a)lists.mpi-forum.org >> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll >> >> > > > -- > Tony Skjellum, PhD > RunTime Computing Solutions, LLC > tony(a)runtimecomputing.com > direct: +1-205-918-7514 > cell: +1-205-807-4968 > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > > > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > _______________________________________________ > mpiwg-coll mailing list > mpiwg-coll(a)lists.mpi-forum.org > https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll > >

1 0

Re: [mpiwg-coll] backward communication in neighborhood collectives
by HOLMES Daniel 23 Jul '18

23 Jul '18

Hi Junchao Zhang, My understanding of the current API for MPI-3.1 is that: 1) the virtual topology does not actually restrict communication via the communicator to the edges specified in the topology - messages can be sent along any edge in either direction, and even between pairs of processes for which no edge was specified. 2) the virtual topology can be specified as a symmetric graph - for every ‘forward’ edge (e.g. from A to B), the ‘backward’ edge (i.e. from B to A) can be included as well. 3) there is already language in the MPI Standard regarding how MPI handles symmetric and non-symmetric graph topologies for neighbourhood collective operations. Thus, there is no need to create two distributed graph topology communicators to achieve ‘forward and backward communication along the edges’. Cheers, Dan. — Dr Daniel Holmes PhD Applications Consultant in HPC Research d.holmes(a)epcc.ed.ac.uk<mailto:[email protected]> Phone: +44 (0) 131 651 3465 Mobile: +44 (0) 7940 524 088 Address: Room 3415, JCMB, The King’s Buildings, Edinburgh, EH9 3FD — The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. — On 15 Jul 2018, at 10:42, Anthony Skjellum <tony(a)runtimecomputing.com<mailto:[email protected]>> wrote: Hi, I just saw this. We definitely need to consider this concern. I also need to go review the APIs. Thanks, Tony On Sat, Jul 14, 2018 at 12:27 AM, Junchao Zhang <jczhang(a)mcs.anl.gov<mailto:[email protected]>> wrote: I want to try MPI neighborhood collectives. I have a communication graph and want to do both forward and backward communication along the edges. With current APIs, it looks I need to create two comm_dist_graphs. It is a waste since MPI implementations can do similar optimizations for the two. Should MPI support this scenario? Thanks. --Junchao Zhang _______________________________________________ mpiwg-coll mailing list mpiwg-coll(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll -- Tony Skjellum, PhD RunTime Computing Solutions, LLC tony(a)runtimecomputing.com<mailto:[email protected]> direct: +1-205-918-7514 cell: +1-205-807-4968 _______________________________________________ mpiwg-coll mailing list mpiwg-coll(a)lists.mpi-forum.org<mailto:[email protected]> https://lists.mpi-forum.org/mailman/listinfo/mpiwg-coll The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

1 0