Hello Christian, thanks for the review of the first draft. All comments that I deleted are fixed, but I'd like to open the discussion of others up.
btw. all line numbers within the MPI standard are not really line numbers (they have equal distances and as soon as there is another font size - like for the interface definitions - then you end up anywhere...) It's like "look at page X at _centimetre_ Y". yes, I sent a note about this to the MPI-2.1 mailinglist a while ago, but saw no reaction.
p 49, l 46: it is not enough to state "receive buffer must not be read" it should be forbidden to access the receive buffer in any way (imagine an implementation that uses the buffer to forwards sth.) Right, I changed this to be similar to MPI-2.1 p52 line 4-7. Now it reads "must not be accessed".
p 50, l 7: we should discuss MPI_REQUEST_FREE (again) - I talked to some colleagues and we see application scenarios where this might be useful (e.g., if only a subgroup participates in an irregular collective); because this is a local operation, we also don't see any problems by generally allowing MPI_REQUEST_FREE for nonblocking collective operations I personally don't like request free because it makes thread-safe implementations really hard. Also, I don't see a good portable way to check/ensure that the operation completed on the receiver side (we had this discussion before on the MPI-2.1 ML I think). Do you know one?
p 50, l 19: maybe we should be even more precise and add after "environments." something like "It is therefore erroneous to start a set of nbc in different logical orders on the participating ranks." we didn't really define "different logical order" anywhere. This is why we decided to go by examples.
The sentence in l 20 ("The behavior...") can and should be deleted. "Nbc operations and bc operations [+are totally unrelated and ] do not match each other." addition not made ("totally unrelated") - I think we shouldn't say this sinde they perform very similar operations and can be implemented on top of each other.
several locations: "memory movement" should be replaced by "data movement" (but "movement" sounds bad anyway because you say "after completion" - maybe "data placement" would be more appropriate) the MPI standard says for example "The ``in place'' operations are provided to reduce unnecessary memory motion". I agree that it sounds strange, but it seems to be the used like this in MPI terms. But I am open to change it.
p 61, l 31.8 & p 62, l 27.5: "after it completion" -> "after completion"? I don't see a mistake there. It says "after it completed" in the text.
btw. there seems to be a potential naming problem for beginners: "Exclusive Scan -> Exscan" && "Inclusive Scan -> Iscan"??? Hmm, I wouldn't think so, but we can discuss this.
p 69, l 41: "in SPMD style programs" seems superflous to me but "double buffering" could also be added as an example ... it's not superflous. One can also implement task-parallel or other programming paradigms with MPI. I added double-bufferng (even though this is just a weaker form of pipelining).
Example 5.33 seems to be very dangerous for at least two reasons: 2) this doesn't make it a "global" allreduce because (depending how sbuf and rbuf are initialized) all seem to calculate different results (0+0+1+2 vs. 0+1+1+2 vs. 1+2+0+2) exactly - that's what they want to do. Everybody wants to reduce in two communicators. This tiny example is probably not very useful, but easy to understand. Think of nearest neighbor reductions.
This example is a good idea but needs some clarifications...
What do we want to have (either or): a) the same ordering over both blocking and nonblocking collectives (suggested by Example 5.29) b) blocking and nonblocking collectives are completely independent (suggested by "reserved tag-space" p 50, l 15) Upon a decision, the related text needs some clarification.
we decided to go with a). The advice to implementors talks only about non-blocking point-to-point when it mentions the reserved tag space.
All in all a big step towards the standardization. However, there seem to be quite a couple of mistakes in the copy-and-paste parts and I'm quite sure that I didn't catch all in my first go... so please be especially careful in those places. yes, this is the first draft to be circulated only in the workgroup and far away from an official reading. I attached an updated document to the wiki page [1] and put another convenience copy here [2].
Thank you very much for your review! All the Best, Torsten [1]: https://svn.mpi-forum.org/trac/mpi-forum-web/wiki/NBColl [2]: http://www.unixer.de/sec/nbc-proposal-11-13-2008.pdf -- bash$ :(){ :|:&};: --------------------- http://www.unixer.de/ ----- Torsten Hoefler | Postdoctoral Researcher Open Systems Lab | Indiana University 150 S. Woodlawn Ave. | Bloomington, IN, 474045, USA Lindley Hall Room 135 | +01 (812) 855-3608