December 2013 - devel - mailman3.cels.anl.gov

Re: [mpich-devel] MPI_Dims_create() fails for nproc >1000000
by Pavan Balaji 19 Dec '13

19 Dec '13

Actually, I was favoring just increasing the static index set to 4792 indices instead of generating on the fly. This was mostly because it’s the smallest distance change without introducing any new bugs. 19KB is not too bad, since it’s one allocation per process irrespective of how many communicators you create. — Pavan On Dec 20, 2013, at 9:23 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > Indeed, I also wrote a C program to generate primes and found that it was fast enough to justify generating the list on-the-fly for every call. > > No one expects MPI_Cart_create to be optimized. And I favor saving space at the cost of time when it comes to text/data/bss (wherever static arrays go). > > And in how many cases does prime generation take longer than the context id allreduce, etc.? > > Jeff > > Sent from my iPhone > >> On Dec 19, 2013, at 6:58 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >> >> >> Increasing the number of indices to 4792 is the faster approach, if we are looking for a quick solution. It’s also safer, since the code is a bit tricky to get right. >> >> — Pavan >> >>> On Dec 20, 2013, at 8:50 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >>> >>> >>> The mixed approach sounds good (it’s fast for “small systems”). >>> >>> But for completeness I should point out a couple of things: >>> >>> 1. If we want to do this statically, the maximum number of MPI processes in an application is MAX_INT. That requires 4792 indices (19KB). So doing a "full set" of primes is not too space consuming. >>> >>> 2. If we want to do this at runtime (presumably at INIT time), my super-inefficient sequential script with gcc and no optimizations took 40 milliseconds to run. So it should not be too bad time wise either. >>> >>> — Pavan >>> >>>> On Dec 20, 2013, at 2:24 AM, Antonio J. Peña <apenya(a)anl.gov> wrote: >>>> >>>> >>>> What about a mixed approach? If we've received no complaints so far that means that the current LUT is fine in most cases. We could fall into runtime computations if it's larger that that value. That will introduce a forced check, but the overhead will be lower than always computing, while the memory consumption would keep the same and the values not limited. >>>> >>>> Toni >>>> >>>> >>>>> On 12/19/2013 11:13 AM, Jeff Hammond wrote: >>>>> I started working on a patch, but I'm not sure if it is the right >>>>> thing to just encode a longer static array of primes. >>>>> >>>>> Any suggestions on how to proceed? Should I generate the array of >>>>> primes on-the-fly (once per MPI instance) up to the max possible >>>>> amount? >>>>> >>>>> Jeff >>>>> >>>>> >>>>> ---------- Forwarded message ---------- >>>>> From: Jeff Hammond <jeff.science(a)gmail.com> >>>>> Date: Thu, Dec 19, 2013 at 10:55 AM >>>>> Subject: Re: [mpich-discuss] MPI_Dims_create() fails for nproc >1000000 >>>>> To: discuss(a)mpich.org >>>>> >>>>> >>>>> Can you provide a short test that reproduces the issue you are seeing? >>>>> I will add a Trac ticket for it. >>>>> >>>>> Thanks, >>>>> >>>>> Jeff >>>>> >>>>>> On Thu, Dec 19, 2013 at 9:49 AM, Andreas Schäfer <gentryx(a)gmx.de> wrote: >>>>>> Hi, >>>>>> >>>>>> I noticed that MPICH's MPI_Dims_create() fails with a floating point >>>>>> exception, if the number of processes is larger than 10^6. Apparently >>>>>> the reason is that the prime factors are hard coded into the code and >>>>>> are limited to 1000 (or sqrt(10^6)). Open MPI's implementation has no >>>>>> such limitation. >>>>>> >>>>>> Best >>>>>> -Andreas >>>>>> >>>>>> >>>>>> -- >>>>>> ========================================================== >>>>>> Andreas Schäfer >>>>>> HPC and Grid Computing >>>>>> Chair of Computer Science 3 >>>>>> Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany >>>>>> +49 9131 85-27910 >>>>>> PGP/GPG key via keyserver >>>>>> http://www.libgeodecomp.org >>>>>> ========================================================== >>>>>> >>>>>> (\___/) >>>>>> (+'.'+) >>>>>> (")_(") >>>>>> This is Bunny. Copy and paste Bunny into your >>>>>> signature to help him gain world domination! >>>>>> >>>>>> _______________________________________________ >>>>>> discuss mailing list discuss(a)mpich.org >>>>>> To manage subscription options or unsubscribe: >>>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> >>>>> >>>>> -- >>>>> Jeff Hammond >>>>> jeff.science(a)gmail.com >>>> >>>> >>>> -- >>>> Antonio J. Peña >>>> Postdoctoral Appointee >>>> Mathematics and Computer Science Division >>>> Argonne National Laboratory >>>> 9700 South Cass Avenue, Bldg. 240, Of. 3148 >>>> Argonne, IL 60439-4847 >>>> apenya(a)mcs.anl.gov >>>> www.mcs.anl.gov/~apenya >>> >>> -- >>> Pavan Balaji >>> http://www.mcs.anl.gov/~balaji >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-devel] MPI_Dims_create() fails for nproc >1000000
by Jeff Hammond 19 Dec '13

19 Dec '13

Indeed, I also wrote a C program to generate primes and found that it was fast enough to justify generating the list on-the-fly for every call. No one expects MPI_Cart_create to be optimized. And I favor saving space at the cost of time when it comes to text/data/bss (wherever static arrays go). And in how many cases does prime generation take longer than the context id allreduce, etc.? Jeff Sent from my iPhone > On Dec 19, 2013, at 6:58 PM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > > Increasing the number of indices to 4792 is the faster approach, if we are looking for a quick solution. It’s also safer, since the code is a bit tricky to get right. > > — Pavan > >> On Dec 20, 2013, at 8:50 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: >> >> >> The mixed approach sounds good (it’s fast for “small systems”). >> >> But for completeness I should point out a couple of things: >> >> 1. If we want to do this statically, the maximum number of MPI processes in an application is MAX_INT. That requires 4792 indices (19KB). So doing a "full set" of primes is not too space consuming. >> >> 2. If we want to do this at runtime (presumably at INIT time), my super-inefficient sequential script with gcc and no optimizations took 40 milliseconds to run. So it should not be too bad time wise either. >> >> — Pavan >> >>> On Dec 20, 2013, at 2:24 AM, Antonio J. Peña <apenya(a)anl.gov> wrote: >>> >>> >>> What about a mixed approach? If we've received no complaints so far that means that the current LUT is fine in most cases. We could fall into runtime computations if it's larger that that value. That will introduce a forced check, but the overhead will be lower than always computing, while the memory consumption would keep the same and the values not limited. >>> >>> Toni >>> >>> >>>> On 12/19/2013 11:13 AM, Jeff Hammond wrote: >>>> I started working on a patch, but I'm not sure if it is the right >>>> thing to just encode a longer static array of primes. >>>> >>>> Any suggestions on how to proceed? Should I generate the array of >>>> primes on-the-fly (once per MPI instance) up to the max possible >>>> amount? >>>> >>>> Jeff >>>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: Jeff Hammond <jeff.science(a)gmail.com> >>>> Date: Thu, Dec 19, 2013 at 10:55 AM >>>> Subject: Re: [mpich-discuss] MPI_Dims_create() fails for nproc >1000000 >>>> To: discuss(a)mpich.org >>>> >>>> >>>> Can you provide a short test that reproduces the issue you are seeing? >>>> I will add a Trac ticket for it. >>>> >>>> Thanks, >>>> >>>> Jeff >>>> >>>>> On Thu, Dec 19, 2013 at 9:49 AM, Andreas Schäfer <gentryx(a)gmx.de> wrote: >>>>> Hi, >>>>> >>>>> I noticed that MPICH's MPI_Dims_create() fails with a floating point >>>>> exception, if the number of processes is larger than 10^6. Apparently >>>>> the reason is that the prime factors are hard coded into the code and >>>>> are limited to 1000 (or sqrt(10^6)). Open MPI's implementation has no >>>>> such limitation. >>>>> >>>>> Best >>>>> -Andreas >>>>> >>>>> >>>>> -- >>>>> ========================================================== >>>>> Andreas Schäfer >>>>> HPC and Grid Computing >>>>> Chair of Computer Science 3 >>>>> Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany >>>>> +49 9131 85-27910 >>>>> PGP/GPG key via keyserver >>>>> http://www.libgeodecomp.org >>>>> ========================================================== >>>>> >>>>> (\___/) >>>>> (+'.'+) >>>>> (")_(") >>>>> This is Bunny. Copy and paste Bunny into your >>>>> signature to help him gain world domination! >>>>> >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> >>>> >>>> -- >>>> Jeff Hammond >>>> jeff.science(a)gmail.com >>> >>> >>> -- >>> Antonio J. Peña >>> Postdoctoral Appointee >>> Mathematics and Computer Science Division >>> Argonne National Laboratory >>> 9700 South Cass Avenue, Bldg. 240, Of. 3148 >>> Argonne, IL 60439-4847 >>> apenya(a)mcs.anl.gov >>> www.mcs.anl.gov/~apenya >> >> -- >> Pavan Balaji >> http://www.mcs.anl.gov/~balaji > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-devel] MPI_Dims_create() fails for nproc >1000000
by Pavan Balaji 19 Dec '13

19 Dec '13

Increasing the number of indices to 4792 is the faster approach, if we are looking for a quick solution. It’s also safer, since the code is a bit tricky to get right. — Pavan On Dec 20, 2013, at 8:50 AM, Pavan Balaji <balaji(a)mcs.anl.gov> wrote: > > The mixed approach sounds good (it’s fast for “small systems”). > > But for completeness I should point out a couple of things: > > 1. If we want to do this statically, the maximum number of MPI processes in an application is MAX_INT. That requires 4792 indices (19KB). So doing a "full set" of primes is not too space consuming. > > 2. If we want to do this at runtime (presumably at INIT time), my super-inefficient sequential script with gcc and no optimizations took 40 milliseconds to run. So it should not be too bad time wise either. > > — Pavan > > On Dec 20, 2013, at 2:24 AM, Antonio J. Peña <apenya(a)anl.gov> wrote: > >> >> What about a mixed approach? If we've received no complaints so far that means that the current LUT is fine in most cases. We could fall into runtime computations if it's larger that that value. That will introduce a forced check, but the overhead will be lower than always computing, while the memory consumption would keep the same and the values not limited. >> >> Toni >> >> >> On 12/19/2013 11:13 AM, Jeff Hammond wrote: >>> I started working on a patch, but I'm not sure if it is the right >>> thing to just encode a longer static array of primes. >>> >>> Any suggestions on how to proceed? Should I generate the array of >>> primes on-the-fly (once per MPI instance) up to the max possible >>> amount? >>> >>> Jeff >>> >>> >>> ---------- Forwarded message ---------- >>> From: Jeff Hammond <jeff.science(a)gmail.com> >>> Date: Thu, Dec 19, 2013 at 10:55 AM >>> Subject: Re: [mpich-discuss] MPI_Dims_create() fails for nproc >1000000 >>> To: discuss(a)mpich.org >>> >>> >>> Can you provide a short test that reproduces the issue you are seeing? >>> I will add a Trac ticket for it. >>> >>> Thanks, >>> >>> Jeff >>> >>> On Thu, Dec 19, 2013 at 9:49 AM, Andreas Schäfer <gentryx(a)gmx.de> wrote: >>>> Hi, >>>> >>>> I noticed that MPICH's MPI_Dims_create() fails with a floating point >>>> exception, if the number of processes is larger than 10^6. Apparently >>>> the reason is that the prime factors are hard coded into the code and >>>> are limited to 1000 (or sqrt(10^6)). Open MPI's implementation has no >>>> such limitation. >>>> >>>> Best >>>> -Andreas >>>> >>>> >>>> -- >>>> ========================================================== >>>> Andreas Schäfer >>>> HPC and Grid Computing >>>> Chair of Computer Science 3 >>>> Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany >>>> +49 9131 85-27910 >>>> PGP/GPG key via keyserver >>>> http://www.libgeodecomp.org >>>> ========================================================== >>>> >>>> (\___/) >>>> (+'.'+) >>>> (")_(") >>>> This is Bunny. Copy and paste Bunny into your >>>> signature to help him gain world domination! >>>> >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> >>> >>> -- >>> Jeff Hammond >>> jeff.science(a)gmail.com >>> >>> >> >> >> -- >> Antonio J. Peña >> Postdoctoral Appointee >> Mathematics and Computer Science Division >> Argonne National Laboratory >> 9700 South Cass Avenue, Bldg. 240, Of. 3148 >> Argonne, IL 60439-4847 >> apenya(a)mcs.anl.gov >> www.mcs.anl.gov/~apenya > > -- > Pavan Balaji > http://www.mcs.anl.gov/~balaji > -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-devel] MPI_Dims_create() fails for nproc >1000000
by Pavan Balaji 19 Dec '13

19 Dec '13

The mixed approach sounds good (it’s fast for “small systems”). But for completeness I should point out a couple of things: 1. If we want to do this statically, the maximum number of MPI processes in an application is MAX_INT. That requires 4792 indices (19KB). So doing a "full set" of primes is not too space consuming. 2. If we want to do this at runtime (presumably at INIT time), my super-inefficient sequential script with gcc and no optimizations took 40 milliseconds to run. So it should not be too bad time wise either. — Pavan On Dec 20, 2013, at 2:24 AM, Antonio J. Peña <apenya(a)anl.gov> wrote: > > What about a mixed approach? If we've received no complaints so far that means that the current LUT is fine in most cases. We could fall into runtime computations if it's larger that that value. That will introduce a forced check, but the overhead will be lower than always computing, while the memory consumption would keep the same and the values not limited. > > Toni > > > On 12/19/2013 11:13 AM, Jeff Hammond wrote: >> I started working on a patch, but I'm not sure if it is the right >> thing to just encode a longer static array of primes. >> >> Any suggestions on how to proceed? Should I generate the array of >> primes on-the-fly (once per MPI instance) up to the max possible >> amount? >> >> Jeff >> >> >> ---------- Forwarded message ---------- >> From: Jeff Hammond <jeff.science(a)gmail.com> >> Date: Thu, Dec 19, 2013 at 10:55 AM >> Subject: Re: [mpich-discuss] MPI_Dims_create() fails for nproc >1000000 >> To: discuss(a)mpich.org >> >> >> Can you provide a short test that reproduces the issue you are seeing? >> I will add a Trac ticket for it. >> >> Thanks, >> >> Jeff >> >> On Thu, Dec 19, 2013 at 9:49 AM, Andreas Schäfer <gentryx(a)gmx.de> wrote: >>> Hi, >>> >>> I noticed that MPICH's MPI_Dims_create() fails with a floating point >>> exception, if the number of processes is larger than 10^6. Apparently >>> the reason is that the prime factors are hard coded into the code and >>> are limited to 1000 (or sqrt(10^6)). Open MPI's implementation has no >>> such limitation. >>> >>> Best >>> -Andreas >>> >>> >>> -- >>> ========================================================== >>> Andreas Schäfer >>> HPC and Grid Computing >>> Chair of Computer Science 3 >>> Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany >>> +49 9131 85-27910 >>> PGP/GPG key via keyserver >>> http://www.libgeodecomp.org >>> ========================================================== >>> >>> (\___/) >>> (+'.'+) >>> (")_(") >>> This is Bunny. Copy and paste Bunny into your >>> signature to help him gain world domination! >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> >> > > > -- > Antonio J. Peña > Postdoctoral Appointee > Mathematics and Computer Science Division > Argonne National Laboratory > 9700 South Cass Avenue, Bldg. 240, Of. 3148 > Argonne, IL 60439-4847 > apenya(a)mcs.anl.gov > www.mcs.anl.gov/~apenya -- Pavan Balaji http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-devel] MPI_Dims_create() fails for nproc >1000000
by William Gropp 19 Dec '13

19 Dec '13

I like this approach - it preserves the working code and the most common usage cases. Bill William Gropp Director, Parallel Computing Institute Deputy Director for Research Institute for Advanced Computing Applications and Technologies Thomas M. Siebel Chair in Computer Science University of Illinois Urbana-Champaign On Dec 19, 2013, at 12:24 PM, Antonio J. Peña wrote: > > What about a mixed approach? If we've received no complaints so far that means that the current LUT is fine in most cases. We could fall into runtime computations if it's larger that that value. That will introduce a forced check, but the overhead will be lower than always computing, while the memory consumption would keep the same and the values not limited. > > Toni > > > On 12/19/2013 11:13 AM, Jeff Hammond wrote: >> I started working on a patch, but I'm not sure if it is the right >> thing to just encode a longer static array of primes. >> >> Any suggestions on how to proceed? Should I generate the array of >> primes on-the-fly (once per MPI instance) up to the max possible >> amount? >> >> Jeff >> >> >> ---------- Forwarded message ---------- >> From: Jeff Hammond <jeff.science(a)gmail.com> >> Date: Thu, Dec 19, 2013 at 10:55 AM >> Subject: Re: [mpich-discuss] MPI_Dims_create() fails for nproc >1000000 >> To: discuss(a)mpich.org >> >> >> Can you provide a short test that reproduces the issue you are seeing? >> I will add a Trac ticket for it. >> >> Thanks, >> >> Jeff >> >> On Thu, Dec 19, 2013 at 9:49 AM, Andreas Schäfer <gentryx(a)gmx.de> wrote: >>> Hi, >>> >>> I noticed that MPICH's MPI_Dims_create() fails with a floating point >>> exception, if the number of processes is larger than 10^6. Apparently >>> the reason is that the prime factors are hard coded into the code and >>> are limited to 1000 (or sqrt(10^6)). Open MPI's implementation has no >>> such limitation. >>> >>> Best >>> -Andreas >>> >>> >>> -- >>> ========================================================== >>> Andreas Schäfer >>> HPC and Grid Computing >>> Chair of Computer Science 3 >>> Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany >>> +49 9131 85-27910 >>> PGP/GPG key via keyserver >>> http://www.libgeodecomp.org >>> ========================================================== >>> >>> (\___/) >>> (+'.'+) >>> (")_(") >>> This is Bunny. Copy and paste Bunny into your >>> signature to help him gain world domination! >>> >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss >> >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> >> > > > -- > Antonio J. Peña > Postdoctoral Appointee > Mathematics and Computer Science Division > Argonne National Laboratory > 9700 South Cass Avenue, Bldg. 240, Of. 3148 > Argonne, IL 60439-4847 > apenya(a)mcs.anl.gov > www.mcs.anl.gov/~apenya >

1 0

Re: [mpich-devel] MPI_Dims_create() fails for nproc >1000000
by "Antonio J. Peña" 19 Dec '13

19 Dec '13

What about a mixed approach? If we've received no complaints so far that means that the current LUT is fine in most cases. We could fall into runtime computations if it's larger that that value. That will introduce a forced check, but the overhead will be lower than always computing, while the memory consumption would keep the same and the values not limited. Toni On 12/19/2013 11:13 AM, Jeff Hammond wrote: > I started working on a patch, but I'm not sure if it is the right > thing to just encode a longer static array of primes. > > Any suggestions on how to proceed? Should I generate the array of > primes on-the-fly (once per MPI instance) up to the max possible > amount? > > Jeff > > > ---------- Forwarded message ---------- > From: Jeff Hammond <jeff.science(a)gmail.com> > Date: Thu, Dec 19, 2013 at 10:55 AM > Subject: Re: [mpich-discuss] MPI_Dims_create() fails for nproc >1000000 > To: discuss(a)mpich.org > > > Can you provide a short test that reproduces the issue you are seeing? > I will add a Trac ticket for it. > > Thanks, > > Jeff > > On Thu, Dec 19, 2013 at 9:49 AM, Andreas Schäfer <gentryx(a)gmx.de> wrote: >> Hi, >> >> I noticed that MPICH's MPI_Dims_create() fails with a floating point >> exception, if the number of processes is larger than 10^6. Apparently >> the reason is that the prime factors are hard coded into the code and >> are limited to 1000 (or sqrt(10^6)). Open MPI's implementation has no >> such limitation. >> >> Best >> -Andreas >> >> >> -- >> ========================================================== >> Andreas Schäfer >> HPC and Grid Computing >> Chair of Computer Science 3 >> Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany >> +49 9131 85-27910 >> PGP/GPG key via keyserver >> http://www.libgeodecomp.org >> ========================================================== >> >> (\___/) >> (+'.'+) >> (")_(") >> This is Bunny. Copy and paste Bunny into your >> signature to help him gain world domination! >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss > > > -- > Jeff Hammond > jeff.science(a)gmail.com > > -- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya

1 0

Re: [mpich-devel] MPI_Dims_create() fails for nproc >1000000
by Jeff Hammond 19 Dec '13

19 Dec '13

I started working on a patch, but I'm not sure if it is the right thing to just encode a longer static array of primes. Any suggestions on how to proceed? Should I generate the array of primes on-the-fly (once per MPI instance) up to the max possible amount? Jeff ---------- Forwarded message ---------- From: Jeff Hammond <jeff.science(a)gmail.com> Date: Thu, Dec 19, 2013 at 10:55 AM Subject: Re: [mpich-discuss] MPI_Dims_create() fails for nproc >1000000 To: discuss(a)mpich.org Can you provide a short test that reproduces the issue you are seeing? I will add a Trac ticket for it. Thanks, Jeff On Thu, Dec 19, 2013 at 9:49 AM, Andreas Schäfer <gentryx(a)gmx.de> wrote: > Hi, > > I noticed that MPICH's MPI_Dims_create() fails with a floating point > exception, if the number of processes is larger than 10^6. Apparently > the reason is that the prime factors are hard coded into the code and > are limited to 1000 (or sqrt(10^6)). Open MPI's implementation has no > such limitation. > > Best > -Andreas > > > -- > ========================================================== > Andreas Schäfer > HPC and Grid Computing > Chair of Computer Science 3 > Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany > +49 9131 85-27910 > PGP/GPG key via keyserver > http://www.libgeodecomp.org > ========================================================== > > (\___/) > (+'.'+) > (")_(") > This is Bunny. Copy and paste Bunny into your > signature to help him gain world domination! > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Jeff Hammond jeff.science(a)gmail.com -- Jeff Hammond jeff.science(a)gmail.com

1 0

Re: [mpich-devel] MPI_Alltoall routine
by "Antonio J. Peña" 10 Dec '13

10 Dec '13

Hello Antoine, What you are seeing is an alltoall overriden by the netmod layer. This is allowed so that netmods can provide collectives tuned for their particular network target. If collfns->Alltoall is NULL, i.e., the netmod didn't provide one, MPICH will execute the default code provided in the MPI layer. The "right" way for you to implement it depends in whether you are implementing a generic collective algorithm or one targeting a specific network. Best, Antonio On 12/10/2013 11:00 AM, Antoine Rougier wrote: > Hi, > > I want to test my own Alltoall implementation. To do that, I would like > to modify MPIR_Alltoall_intra in src/mpi/coll/alltoall.c. > Before that, I have checked (using MPIU_DBG_MSG) that my compiled > version take the "good way": > MPI_Alltoall->MPIR_Alltoall_impl->MPIR_Alltoall->MPIR_Alltoall_intra->"the > good case", but it doesn't take this way. Instead of, in > MPIR_Alltoall_impl, it calls comm_ptr->collfns->Alltoall. > > So I have two questions: > - Is it the good way to add/modify some alltoall implementations ? > - How to execute the good alltoall implementation ? > > Best regards, > -- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 apenya(a)mcs.anl.gov www.mcs.anl.gov/~apenya

1 0

MPI_Alltoall routine
by Antoine Rougier 10 Dec '13

10 Dec '13

Hi, I want to test my own Alltoall implementation. To do that, I would like to modify MPIR_Alltoall_intra in src/mpi/coll/alltoall.c. Before that, I have checked (using MPIU_DBG_MSG) that my compiled version take the "good way": MPI_Alltoall->MPIR_Alltoall_impl->MPIR_Alltoall->MPIR_Alltoall_intra->"the good case", but it doesn't take this way. Instead of, in MPIR_Alltoall_impl, it calls comm_ptr->collfns->Alltoall. So I have two questions: - Is it the good way to add/modify some alltoall implementations ? - How to execute the good alltoall implementation ? Best regards, -- Antoine Rougier

1 0