Actually, I was favoring just increasing the static index set to 4792 indices instead of generating on the fly. This was mostly because it’s the smallest distance change without introducing any new bugs. 19KB is not too bad, since it’s one allocation per process irrespective of how many communicators you create. — Pavan On Dec 20, 2013, at 9:23 AM, Jeff Hammond <[email protected]> wrote:

Indeed, I also wrote a C program to generate primes and found that it was fast enough to justify generating the list on-the-fly for every call.

No one expects MPI_Cart_create to be optimized. And I favor saving space at the cost of time when it comes to text/data/bss (wherever static arrays go).

And in how many cases does prime generation take longer than the context id allreduce, etc.?

Jeff

Sent from my iPhone

...

On Dec 19, 2013, at 6:58 PM, Pavan Balaji <[email protected]> wrote:

Increasing the number of indices to 4792 is the faster approach, if we are looking for a quick solution. It’s also safer, since the code is a bit tricky to get right.

— Pavan

...

On Dec 20, 2013, at 8:50 AM, Pavan Balaji <[email protected]> wrote:

The mixed approach sounds good (it’s fast for “small systems”).

But for completeness I should point out a couple of things:

1. If we want to do this statically, the maximum number of MPI processes in an application is MAX_INT. That requires 4792 indices (19KB). So doing a "full set" of primes is not too space consuming.

2. If we want to do this at runtime (presumably at INIT time), my super-inefficient sequential script with gcc and no optimizations took 40 milliseconds to run. So it should not be too bad time wise either.

— Pavan

...

On Dec 20, 2013, at 2:24 AM, Antonio J. Peña <[email protected]> wrote:

What about a mixed approach? If we've received no complaints so far that means that the current LUT is fine in most cases. We could fall into runtime computations if it's larger that that value. That will introduce a forced check, but the overhead will be lower than always computing, while the memory consumption would keep the same and the values not limited.

Toni

...

On 12/19/2013 11:13 AM, Jeff Hammond wrote: I started working on a patch, but I'm not sure if it is the right thing to just encode a longer static array of primes.

Any suggestions on how to proceed? Should I generate the array of primes on-the-fly (once per MPI instance) up to the max possible amount?

Jeff

---------- Forwarded message ---------- From: Jeff Hammond <[email protected]> Date: Thu, Dec 19, 2013 at 10:55 AM Subject: Re: [mpich-discuss] MPI_Dims_create() fails for nproc >1000000 To: [email protected]

Can you provide a short test that reproduces the issue you are seeing? I will add a Trac ticket for it.

Thanks,

Jeff

...

On Thu, Dec 19, 2013 at 9:49 AM, Andreas Schäfer <[email protected]> wrote: Hi,

I noticed that MPICH's MPI_Dims_create() fails with a floating point exception, if the number of processes is larger than 10^6. Apparently the reason is that the prime factors are hard coded into the code and are limited to 1000 (or sqrt(10^6)). Open MPI's implementation has no such limitation.

Best -Andreas

-- ========================================================== Andreas Schäfer HPC and Grid Computing Chair of Computer Science 3 Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany +49 9131 85-27910 PGP/GPG key via keyserver http://www.libgeodecomp.org ==========================================================

(\___/) (+'.'+) (")_(") This is Bunny. Copy and paste Bunny into your signature to help him gain world domination!

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

-- Jeff Hammond [email protected]

-- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 [email protected] www.mcs.anl.gov/~apenya

-- Pavan Balaji http://www.mcs.anl.gov/~balaji

-- Pavan Balaji http://www.mcs.anl.gov/~balaji

-- Pavan Balaji http://www.mcs.anl.gov/~balaji