October 2014 - mpiwg-rma - mailman3.cels.anl.gov

sync on vector of windows?
by Jeff Hammond 05 Oct '14

05 Oct '14

I created https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/459 in response to my long-standing issue with this in ARMCI-MPI and the additional issues encountered by Min (copied) in another project. Please let me know what you think about this. Thanks, Jeff -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [mpiwg-rma] Memory model and source code portability
by Dave Goodell (dgoodell) 01 Oct '14

01 Oct '14

On Oct 1, 2014, at 11:10 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: >> This is the same chicken little story that McClaren told us a while >> back. No C compiler is going to do what you suggested. It's perverse >> and not within the realm of possibility. > > It was my message that Compiler don't do this, because > they implement C11 memory model. > I other words, Boehms problems are historical for C. > When I understood Boehm correctly, then it was a problem, > i.e. not "perverse" and within "the realm of possibility". To be precise, *some* compilers have begun to support the C11 memory model very recently (i.e., in the last year or two). Just as there are many MPI implementations out there with incomplete MPI-3 support, there are compilers out there with varying levels of C11 memory model support. Prior to the availability of the C11 memory model, most compilers avoided the problems in the Boehm paper in various ways, some requiring non-portable attributes/pragmas. -Dave

1 0

Re: [mpiwg-rma] Memory model and source code portability
by Rolf Rabenseifner 01 Oct '14

01 Oct '14

> This is the same chicken little story that McClaren told us a while > back. No C compiler is going to do what you suggested. It's perverse > and not within the realm of possibility. It was my message that Compiler don't do this, because they implement C11 memory model. I other words, Boehms problems are historical for C. When I understood Boehm correctly, then it was a problem, i.e. not "perverse" and within "the realm of possibility". > Async communication isn't broken and we don't need to fix it. We have > millions of lines of nonblocking MPI code running correctly every > day on RDMA networks. The sky is not falling. It was Bill's idea that we should check Boehm's paper and this idea was good. The answer is: resolved by C11. And by the real compilers because they needed to be without conflicts to pthreads. Rolf ----- Original Message ----- > From: "Jeff Hammond" <jeff.science(a)gmail.com> > To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> > Sent: Wednesday, October 1, 2014 5:47:49 PM > Subject: Re: [mpiwg-rma] Memory model and source code portability > > This is the same chicken little story that McClaren told us a while > back. No C compiler is going to do what you suggested. It's perverse > and not within the realm of possibility. > > And async thread isn't required for this theoretical problem. RDMA > would do the same thing. Or interrupts on BG or anything else > consistent with async progress or overlap of comm, both of which the > user wants. > > Async communication isn't broken and we don't need to fix it. We have > millions of lines of nonblocking MPI code running correctly every > day on RDMA networks. The sky is not falling. > > Jeff > > Sent from my iPhone > > On Oct 1, 2014, at 1:23 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> > wrote: > > >> If you agree with Boehm’s article in the strongest form, then you > >> can > >> maintain source code portability by avoiding the use of features > >> that are not well defined in the source language - i.e., the use > >> of > >> shared memory. > > > > Bill, please read my example. It is not using shared memory. > > If we agree with Boehm's article, then it has direct implication > > to normal nonblocking pt-to-pt as soon as the MPI library > > uses internally a progress thread. This should be allowed. > > Or in other words, not only Fortran is a special case. > > With C, we Need not to talk to the C standardization body. > > Similar to Fortran TS29113, the C standardization body > > already did what MPI needs. > > > > As soon as we say, MPI has a C binding, and allow nonblocking > > pt-to-pt > > and allow parallel progress engines, we should say that this > > requires the C11 memory model, i.e., the user must not modify > > ist optimization in a way that the C11 Memory model is no longer > > guaranteed. > > > > MPI provides a C binding, C does not provide an MPI binding. > > Therefore, it is our job. > > Otherwise MPI does not provide source code portability. > > > > Rolf > > > > > > > > ----- Original Message ----- > >> From: "William Gropp" <wgropp(a)illinois.edu> > >> To: "MPI WG Remote Memory Access working group" > >> <mpiwg-rma(a)lists.mpi-forum.org> > >> Sent: Tuesday, September 30, 2014 6:43:39 PM > >> Subject: Re: [mpiwg-rma] Memory model and source code portability > >> > >> > >> Not at all. It just isn’t our place to talk about other > >> standards. > >> Fortran was a slightly different case because of the discussions > >> between members of the MPI Forum and the Fortran standards > >> committee. We don’t have that option with C, and I don’t think we > >> ever will, since MPI programs are not an important part of the use > >> of C. > >> > >> > >> If you agree with Boehm’s article in the strongest form, then you > >> can > >> maintain source code portability by avoiding the use of features > >> that are not well defined in the source language - i.e., the use > >> of > >> shared memory. The current straw vote addresses something at (a) > >> is > >> within the definitions under control of the MPI Forum and (b) > >> allows > >> users that are willing to do what virtually everyone currently > >> using > >> threads and shared memory does - rely on the compiler and > >> processing > >> environment to implement something other than the standard > >> language. > >> But that latter case does not belong in a standard document. > >> > >> > >> Bill > >> > >> > >> > >> On Sep 30, 2014, at 11:23 AM, Rolf Rabenseifner < > >> rabenseifner(a)hlrs.de > wrote: > >> > >> > >> For me it looks like that you give up with the important goal > >> that the MPI standard should provide source code portability. > >> > >> _______________________________________________ > >> mpiwg-rma mailing list > >> mpiwg-rma(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > > > -- > > Dr. Rolf Rabenseifner . . . . . . . . . .. email > > rabenseifner(a)hlrs.de > > High Performance Computing Center (HLRS) . phone > > ++49(0)711/685-65530 > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / > > 685-65832 > > Head of Dpmt Parallel Computing . . . > > www.hlrs.de/people/rabenseifner > > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room > > 1.307) > > _______________________________________________ > > mpiwg-rma mailing list > > mpiwg-rma(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)

1 0

Re: [mpiwg-rma] Memory model and source code portability
by Jeff Hammond 01 Oct '14

01 Oct '14

This is the same chicken little story that McClaren told us a while back. No C compiler is going to do what you suggested. It's perverse and not within the realm of possibility. And async thread isn't required for this theoretical problem. RDMA would do the same thing. Or interrupts on BG or anything else consistent with async progress or overlap of comm, both of which the user wants. Async communication isn't broken and we don't need to fix it. We have millions of lines of nonblocking MPI code running correctly every day on RDMA networks. The sky is not falling. Jeff Sent from my iPhone On Oct 1, 2014, at 1:23 AM, Rolf Rabenseifner <rabenseifner(a)hlrs.de> wrote: >> If you agree with Boehm’s article in the strongest form, then you can >> maintain source code portability by avoiding the use of features >> that are not well defined in the source language - i.e., the use of >> shared memory. > > Bill, please read my example. It is not using shared memory. > If we agree with Boehm's article, then it has direct implication > to normal nonblocking pt-to-pt as soon as the MPI library > uses internally a progress thread. This should be allowed. > Or in other words, not only Fortran is a special case. > With C, we Need not to talk to the C standardization body. > Similar to Fortran TS29113, the C standardization body > already did what MPI needs. > > As soon as we say, MPI has a C binding, and allow nonblocking pt-to-pt > and allow parallel progress engines, we should say that this > requires the C11 memory model, i.e., the user must not modify > ist optimization in a way that the C11 Memory model is no longer > guaranteed. > > MPI provides a C binding, C does not provide an MPI binding. > Therefore, it is our job. > Otherwise MPI does not provide source code portability. > > Rolf > > > > ----- Original Message ----- >> From: "William Gropp" <wgropp(a)illinois.edu> >> To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> >> Sent: Tuesday, September 30, 2014 6:43:39 PM >> Subject: Re: [mpiwg-rma] Memory model and source code portability >> >> >> Not at all. It just isn’t our place to talk about other standards. >> Fortran was a slightly different case because of the discussions >> between members of the MPI Forum and the Fortran standards >> committee. We don’t have that option with C, and I don’t think we >> ever will, since MPI programs are not an important part of the use >> of C. >> >> >> If you agree with Boehm’s article in the strongest form, then you can >> maintain source code portability by avoiding the use of features >> that are not well defined in the source language - i.e., the use of >> shared memory. The current straw vote addresses something at (a) is >> within the definitions under control of the MPI Forum and (b) allows >> users that are willing to do what virtually everyone currently using >> threads and shared memory does - rely on the compiler and processing >> environment to implement something other than the standard language. >> But that latter case does not belong in a standard document. >> >> >> Bill >> >> >> >> On Sep 30, 2014, at 11:23 AM, Rolf Rabenseifner < >> rabenseifner(a)hlrs.de > wrote: >> >> >> For me it looks like that you give up with the important goal >> that the MPI standard should provide source code portability. >> >> _______________________________________________ >> mpiwg-rma mailing list >> mpiwg-rma(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma > > -- > Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de > High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 > Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner > Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307) > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma

1 0

Re: [mpiwg-rma] Ticket #456 - synchronization in MPI-3.0's shared memory
by Rolf Rabenseifner 01 Oct '14

01 Oct '14

Dear Keith, > Apparently, C11 solves all of these issues (or, at least, that is > what I have been led to believe), and modern C compilers honor the > C11 memory model. Thank you for giving me this second approval. Bill's answer is already on https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/456#comment:13 Best regards Rolf ----- Original Message ----- > From: "Keith D Underwood" <keith.d.underwood(a)intel.com> > To: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > Sent: Wednesday, October 1, 2014 5:40:14 AM > Subject: RE: Ticket #456 - synchronization in MPI-3.0's shared memory > > Apparently, C11 solves all of these issues (or, at least, that is > what I have been led to believe), and modern C compilers honor the > C11 memory model. > > > -----Original Message----- > > From: Rolf Rabenseifner [mailto:[email protected]] > > Sent: Thursday, September 11, 2014 3:13 AM > > To: Underwood, Keith D > > Cc: Bill Long > > Subject: Fwd: Ticket #456 - synchronization in MPI-3.0's shared > > memory > > > > Dear Keith, > > > > welcome back from your sabbatical. I hope you had nice weeks. > > > > I sent Bill Long the following question (see below) and I would > > have the same > > question about the Intel Compilers (C, C++ and Fortran). > > > > This question is independent of the details of a threading library > > and whether > > OS threads or processes are used. > > > > Therefore any answer about pthreads would also fit to #456, i.e., > > the > > synchronization in the errata about MPI-3.0 shared memory. > > > > It would be nice, if you can get the answers to the questions > > below. > > > > Best regards > > Rolf > > > > ----- Forwarded Message ----- > > From: "Rolf Rabenseifner" <rabenseifner(a)hlrs.de> > > To: "Bill Long" <longb(a)cray.com> > > Sent: Tuesday, September 9, 2014 5:58:22 PM > > Subject: Ticket #456 - synchronization in MPI-3.0's shared memory > > > > Dear Bill, > > > > in the Forums discussion on #456 > > https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/456 > > Bill Gropp had pointed to > > Hans-J Boehm's paper from 2004: > > > "Threads Cannot be Implemented as a Library" > > http://www.hpl.hp.com/techreports/2004/HPL-2004-209.pdf > > > > I've the following question to you as Compiler specialist: > > > > Do we still have the problems in Sections 4.2 and 4.3 of Boehm's > > paper? > > > > The problem in Section 4.1 is solved as long as the programmer > > issues > > memory barriers, which will be synchronization library Routine > > calls. > > In C, the opaque library calls should divide conflicting accesses > > in different > > access epochs. > > Same in Fortran with the combination of ASYNCHRONOUS variables and > > opaque library calls. > > > > If I understand this paper correctly, then the compiler writes > > since 2004 are > > aware of applications calling pthreads and therefore, the compilers > > do not > > make optimizations that would produce the described data-races in > > Section > > 4.2 and 4.3. > > And of course also other problems detected so far. > > > > Is my assuption correct? > > > > Best regards > > Rolf > > > > > > > > -- > > Dr. Rolf Rabenseifner . . . . . . . . . .. email > > rabenseifner(a)hlrs.de High > > Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / > > 685-65832 Head of Dpmt > > Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. > > 19, D- > > 70550 Stuttgart, Germany . . . . (Office: Room 1.307) > > > > -- > > Dr. Rolf Rabenseifner . . . . . . . . . .. email > > rabenseifner(a)hlrs.de High > > Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 > > University of Stuttgart . . . . . . . . .. fax ++49(0)711 / > > 685-65832 Head of Dpmt > > Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. > > 19, D- > > 70550 Stuttgart, Germany . . . . (Office: Room 1.307) > -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)

1 0

Re: [mpiwg-rma] Memory model and source code portability
by Rolf Rabenseifner 01 Oct '14

01 Oct '14

> If you agree with Boehm’s article in the strongest form, then you can > maintain source code portability by avoiding the use of features > that are not well defined in the source language - i.e., the use of > shared memory. Bill, please read my example. It is not using shared memory. If we agree with Boehm's article, then it has direct implication to normal nonblocking pt-to-pt as soon as the MPI library uses internally a progress thread. This should be allowed. Or in other words, not only Fortran is a special case. With C, we Need not to talk to the C standardization body. Similar to Fortran TS29113, the C standardization body already did what MPI needs. As soon as we say, MPI has a C binding, and allow nonblocking pt-to-pt and allow parallel progress engines, we should say that this requires the C11 memory model, i.e., the user must not modify ist optimization in a way that the C11 Memory model is no longer guaranteed. MPI provides a C binding, C does not provide an MPI binding. Therefore, it is our job. Otherwise MPI does not provide source code portability. Rolf ----- Original Message ----- > From: "William Gropp" <wgropp(a)illinois.edu> > To: "MPI WG Remote Memory Access working group" <mpiwg-rma(a)lists.mpi-forum.org> > Sent: Tuesday, September 30, 2014 6:43:39 PM > Subject: Re: [mpiwg-rma] Memory model and source code portability > > > Not at all. It just isn’t our place to talk about other standards. > Fortran was a slightly different case because of the discussions > between members of the MPI Forum and the Fortran standards > committee. We don’t have that option with C, and I don’t think we > ever will, since MPI programs are not an important part of the use > of C. > > > If you agree with Boehm’s article in the strongest form, then you can > maintain source code portability by avoiding the use of features > that are not well defined in the source language - i.e., the use of > shared memory. The current straw vote addresses something at (a) is > within the definitions under control of the MPI Forum and (b) allows > users that are willing to do what virtually everyone currently using > threads and shared memory does - rely on the compiler and processing > environment to implement something other than the standard language. > But that latter case does not belong in a standard document. > > > Bill > > > > On Sep 30, 2014, at 11:23 AM, Rolf Rabenseifner < > rabenseifner(a)hlrs.de > wrote: > > > For me it looks like that you give up with the important goal > that the MPI standard should provide source code portability. > > _______________________________________________ > mpiwg-rma mailing list > mpiwg-rma(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-rma -- Dr. Rolf Rabenseifner . . . . . . . . . .. email rabenseifner(a)hlrs.de High Performance Computing Center (HLRS) . phone ++49(0)711/685-65530 University of Stuttgart . . . . . . . . .. fax ++49(0)711 / 685-65832 Head of Dpmt Parallel Computing . . . www.hlrs.de/people/rabenseifner Nobelstr. 19, D-70550 Stuttgart, Germany . . . . (Office: Room 1.307)

1 0