October 2015 - mpiwg-tools - mailman3.cels.anl.gov

SC meeting canceled: Nov 19
by Mohror, Kathryn 29 Oct '15

29 Oct '15

Hi all, I'm going to go ahead and cancel the Nov 19 meeting which occurs during the week of SC since I think many of us will be busy during that time. We will still have our Nov 5 meeting -- so talk to you soon! Kathryn _________________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov, http://scalability.llnl.gov/ Scalability Team @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0

Re: [mpiwg-tools] Meeting reminder Oct 22, 8:00 AM PDT
by Marc-Andre Hermanns 23 Oct '15

23 Oct '15

Hi all, the notes of yesterdays meeting are up at: https://github.com/mpiwg-tools/tools-issues/wiki/Notes-2015-10-22 Cheers, Marc-Andre On 21.10.15 22:27, Marc-Andre Hermanns wrote: > Dear all, > > This is your friendly reminder of our regular working group meeting > tomorrow at 8:00 AM PDT. The link to join the webex meeting is on our > wiki: > > https://github.com/mpiwg-tools/tools-issues/wiki/Meetings > > This meeting will be a bit performance-tool centric, with the > following topics to discuss: > > - MPIR: Status of Issue #3 (being_debugged) and the corresponding pull > request > - MPI_T: Jeff may present an additional use case on multi-device > multi-array variables > - QMPI: Matt Legendre of LLNL will join to discuss sysadmin side of > our recent library/symbol discussions > > Kathryn and I thought the one following this one to be more debugging > centric again. > > Cheers, > Marc-Andre > > > > _______________________________________________ > mpiwg-tools mailing list > mpiwg-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools > -- Marc-Andre Hermanns Jülich Aachen Research Alliance, High Performance Computing (JARA-HPC) Jülich Supercomputing Centre (JSC) Schinkelstrasse 2 52062 Aachen Germany Phone: +49 2461 61 2509 | +49 241 80 24381 Fax: +49 2461 80 6 99753 www.jara.org/jara-hpc email: hermanns(a)jara.rwth-aachen.de

1 0

Re: [mpiwg-tools] Mini example of single tool working with multiple MPI implementations
by Jean-Baptiste BESNARD 22 Oct '15

22 Oct '15

Hi Matthew, All, Indeed, it is a very good idea, it should be possible to chain calls using such opaque data-structure. Just some random ideas here, complementary to what has already been introduced: I consider that we want to make the tool independent from the underlying MPI to allow "drop-in" tools. I was thinking during the webex of/MPI_Type_get_envelope/ and /MPI_Type_get_contents/ which provide such abstraction for data-types (p.121 of 3.1) wouldn't it be possible to do it for all MPI functions, packing on the stack before chaining ? /* Internal runtime representation passed as an opaque handler should be a common denominator between function footprints (not the case here !) */ typedef struct { char * function_name; int ints[16]; void * pointers[8]; MPI_Comm comms[2]; MPI_Datatype types[10]; MPI_Status statuses[10]; MPI_Status requests[10]; ... /* Some pointers to handle calls with arrays (Waitall, Testall, ... no need to copy just reference here getters will know where to look) */ MPI_Status * statusses_overflow; MPI_Status * requests_overflow; /* QMPI Call Chain CTX */ int depth_in_the_chain; int (*handler_chain[10])( void *); }QMPI_Context; /param_t/ could point to such context which would be filled prior to the call and passed through the handlers until reaching the actual MPI implementation which would be able to reference packed elements (directly). /* The actual call at the bottom of the chain (with implementation side knowledge) */ __INTERNAL__MPI_Send(ctx->pointers[0], ctx->ints[0], ctx->types[0], ctx->ints[1], ctx->ints[2], ctx->comms[0]) Could be used like this in the tool, as in the example, all wrappers would have the same footprint just a "void *" making bindings very simple: static void mpisend_wrapper(param_t param) { printf("START mpisend tool wrapper. Param is %s\n", (param == MPI_PARAM_VAL) ? "correct" : "wrong"); QMPI_Wrapper_chain(param); printf("DONE mpisend tool wrapper. Param is %s\n", (param == MPI_PARAM_VAL) ? "correct" : "wrong"); } Stacking would allow a tool to inspect the result of a call for functions outputting elements, as in PMPI. Also the context could be used to resolve the next function to call (QMPI chain, incrementing an integer and calling). With the same infrastructure, it is possible to imagine that the tool submits "generic" calls, filling a QMPI_Context (this is why I've put a /name/ field). For example if you do a profile and want to reduce performance data at the end, you could do it without linking to MPI, you just need the qmpi.h header (too much ? sort of "UMPI - Universal MPI"): /* Allocate runtime side storage (you don't know MPI handle sizes !)*/ QMPI_Context_init( &ctx, int sizes, int sizes, ... ); /* Function to call */ QMPI_Context_set_name( &ctx, "MPI_Reduce"); /* Sendbuf */ QMPI_Context_set_pointer( &ctx, 0 /*index*/, sendbuf); /* Recvbuf */ QMPI_Context_set_pointer( &ctx, 1 /*index*/, recvbuf); /* Count */ QMPI_Context_set_int( &ctx, 0 /*index*/, count); /* Datatype */ QMPI_Context_set_type( &ctx, 0 /* Index */, QMPI_CHAR ); /* OP */ QMPI_Context_set_op( &ctx, 0 /* Index */, QMPI_SUM ); /* Root */ QMPI_Context_set_int( &ctx, 1 /*index*/, 0); /* Comm */ QMPI_Context_set_comm( &ctx, 0 /* Index */, QMPI_COMM_WORLD ); QMPI_Submit( &ctx ); /* There would be a GIANT switch behind this ! */ A small subset of data-types, Ops and Comms could be exposed as standardized QMPI_* defines (for example a dup of COMM_WORLD per plugin QMPI_COMM_PLUGIN). However, if the opaque handle is used, one would need to be able to inspect /MPI_Status/, /MPI_Comm/,/MPI_Co//unt/ and /MPI_Datatype/ in an abstract manner: /* Exposed interface relative to a context as you cannot expose the size of the Comm */ int QMPI_Comm_size( QMPI_Context * ctx, int comm_index, int * size ) { /* In the MPI implem */ MPI_Comm_size( ctx->coms[ comm_index], size ); } ... where do we stop ? No problem for integers, you just need to know what is where (standardized as for get_contents), idem for void * you just need something like this on tool side (but only C types this way): int QMPI_Context_get_envelope( QMPI_Context * ctx , int * int_count, int * addr_count, int * datatype_count, int * comm_count... ); int QMPI_Context_get_contents( QMPI_Context * ctx , int * ints, void * addr, ... ); If you are interested, I'll try do draft a first implementation later in the week to see the performance overhead. Thank you very much for the interesting discussions examples and ideas, Jean-Baptiste. Le 22/10/2015 19:34, Matthew LeGendre a ï¿½crit : > > At the end of the call, Jeff asked about a simple example that > illustrates building a tool with QMPI that works with multiple MPI > implementations. I've attached a tarball that illustrates what I was > picturing. > > This example builds a pair of libraries, libfakeopenmpi.so and > libfakempich.so, which each export "mpisend" functions with different > signatures as well as a pair of fake apps, app_openmpi and app_mpich, > that use each library. It also builds a single libtool.so, which > contains wrapper functions for each mpisend and dynamically chooses > which one is used. > > Note that the tool library still has files built against each MPI > implementation, but produces a single library that works with either > implementation. > > On the phone I speculated that you'd need the callback registration > functions to be ABI compatible across all MPI implementations, but > this didn't turn up being necessary. You can get away with only the > MPI identification function having a standard ABI. In this example I > called the function get_mpi_implementation(). > > It also turned up being very convenient if the MPI header exported the > return value of get_mpi_implementation() though a #define (which I > called MPI_IMPL). This allowed for an easy runtime test of whether > the a tool implementation was compiled for the running MPI. > > Also, On the phone I speculated that getting per-compiler tools would > be difficult. You actually could solve part of the problem by > including a compiler name and version in the MPI_IMPL #define--most > compilers have preprocessor macros that exports that info. Though > this still wouldn't solve any tool incompatibilities with runtime > libraries like libstdc++.so. > > -Matt > > > > > _______________________________________________ > mpiwg-tools mailing list > mpiwg-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools

1 0

Mini example of single tool working with multiple MPI implementations
by Matthew LeGendre 22 Oct '15

22 Oct '15

At the end of the call, Jeff asked about a simple example that illustrates building a tool with QMPI that works with multiple MPI implementations. I've attached a tarball that illustrates what I was picturing. This example builds a pair of libraries, libfakeopenmpi.so and libfakempich.so, which each export "mpisend" functions with different signatures as well as a pair of fake apps, app_openmpi and app_mpich, that use each library. It also builds a single libtool.so, which contains wrapper functions for each mpisend and dynamically chooses which one is used. Note that the tool library still has files built against each MPI implementation, but produces a single library that works with either implementation. On the phone I speculated that you'd need the callback registration functions to be ABI compatible across all MPI implementations, but this didn't turn up being necessary. You can get away with only the MPI identification function having a standard ABI. In this example I called the function get_mpi_implementation(). It also turned up being very convenient if the MPI header exported the return value of get_mpi_implementation() though a #define (which I called MPI_IMPL). This allowed for an easy runtime test of whether the a tool implementation was compiled for the running MPI. Also, On the phone I speculated that getting per-compiler tools would be difficult. You actually could solve part of the problem by including a compiler name and version in the MPI_IMPL #define--most compilers have preprocessor macros that exports that info. Though this still wouldn't solve any tool incompatibilities with runtime libraries like libstdc++.so. -Matt

1 0

MPIR_being_debugger
by Jeff Squyres (jsquyres) 22 Oct '15

22 Oct '15

Per discussion on the webex today, I merged the tools pull request into the integration branch. I amended the MPI Forum issue with a new PDF and text: https://github.com/mpi-forum/mpi-issues/issues/5 And created an MPI Forum pull request (based on the same branch as the tools WG branch that we all agreed to today): https://github.com/mpi-forum/mpir/pull/1 -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Meeting reminder Oct 22, 8:00 AM PDT
by Marc-Andre Hermanns 21 Oct '15

21 Oct '15

Dear all, This is your friendly reminder of our regular working group meeting tomorrow at 8:00 AM PDT. The link to join the webex meeting is on our wiki: https://github.com/mpiwg-tools/tools-issues/wiki/Meetings This meeting will be a bit performance-tool centric, with the following topics to discuss: - MPIR: Status of Issue #3 (being_debugged) and the corresponding pull request - MPI_T: Jeff may present an additional use case on multi-device multi-array variables - QMPI: Matt Legendre of LLNL will join to discuss sysadmin side of our recent library/symbol discussions Kathryn and I thought the one following this one to be more debugging centric again. Cheers, Marc-Andre -- Marc-Andre Hermanns Jülich Aachen Research Alliance, High Performance Computing (JARA-HPC) Jülich Supercomputing Centre (JSC) Schinkelstrasse 2 52062 Aachen Germany Phone: +49 2461 61 2509 | +49 241 80 24381 Fax: +49 2461 80 6 99753 www.jara.org/jara-hpc email: hermanns(a)jara.rwth-aachen.de

1 0

Integration git branch
by Jeff Squyres (jsquyres) 15 Oct '15

15 Oct '15

Per discussions on this list, I created an integration git branch in the mpir repo (since I have an outstanding PR). The repo now has 2 main branches: - mpir-1.x: this is basically a mirror of mpi-forum/mpir/mpir-1.x - mpir-1.x-integration: the branch against which we file PRs It's like "option 2" in this wiki page: https://github.com/mpi-forum/mpi-issues/wiki/Working-Group-Guidelines#workf… (note, that set of wiki pages isn't final final final yet, meaning that we haven't mailed it out to the greater Forum yet, but it's close. The Tools WG git repo discussions were quite helpful in shaping up those wiki pages) -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpiwg-tools] Changes from Bordeaux to MPIR
by Jeff Squyres (jsquyres) 09 Oct '15

09 Oct '15

+1 on Todd's idea. > On Oct 9, 2015, at 10:35 AM, Todd Gamblin <tgamblin(a)llnl.gov> wrote: > > On 10/9/15, 6:57 AM, "mpiwg-tools on behalf of Bland, Wesley" > <mpiwg-tools-bounces(a)lists.mpi-forum.org on behalf of > wesley.bland(a)intel.com> wrote: > >> I¹m not sure this solves part two of his question though. Once you accept >> the PR into the working group¹s copy of the MPIR doc, you now have an >> mpir-1.x branch that¹s out of sync with the main mpi-forum/mpir document. >> This means that you open a PR for >> mpiwg-tools/mpir/mpir-1.x:mpi-forum/mpir/mpir-1.x. This is fine until you >> have multiple tickets going at once and want to PR each of them >> individually. >> >> I was actually thinking that we¹d use the model that Marc-Andre >> suggested. Have working groups work on their ideas in a branch on their >> repositories, then PR from the WG repository to the mpi-forum repository. >> I know it¹s not exactly the GitHub model, but I don¹t know that the >> GitHub model is set up for the two-level system that we¹re working with. > > I like Wesley's idea. It's common for teams working closely together to > use a naming scheme for topic branches within a single repository, and to > do PR's *within* that repo. So this could work, and it would preserve > topic branches for later piece-by-piece pushes into the mainline repo. > > It sounds like we want to use PR's for approval of tools proposals, then > use PR's to submit them again to the mainline MPI repo. Could you use a > local integration branch in the tools repo for this, e.g. consider the > following branches: > > mpi-forum: > master > > mpi-tools-wg: > master > integration > topic/mpi-plugins > topic/proposal-2 > topic/proposal-3 > > Within the tools repo someone could easily submit a topic for review with > a PR, e.g.: > > PR: topic/mpi-plugins -> integration > > That could be used to discuss and review the proposal. Once the topic is > merged to integration, it can then be submitted upstream to master because > It's still nicely packaged in a branch: > > PR: mpi-tools-wg:topic/mpi-plugins -> mpi-forum:master > > Then if it gets merged into the mainline, you just sync your fork's master > with the upstream master (common thing to do on github). And you don't > have the problem of submitting a bunch of merged stuff into the upstream. > > The integration branch and all the topic branches should be based on > master. The integration branch itself can be transient, i.e. the tools WG > lead can rebuild it if some PR into upstream master is not accepted. > Anyone can check out the integration branch to see the current state of > all locally accepted tools proposals, but you never submit a PR from > mpi-tools-wg:integration to mpi-forum:master. mpi-tools-wg:master is > always in sync with the upstream master. You keep the original topic > branches around and use them as building blocks for PRs, until they're > merged to upstream master. > > This is not unlike the way Kitware maintains the CMake repo, or the > branchy workflow described in the git manpages. > > You can implement protections on some branches like this: > > https://github.com/blog/2051-protected-branches-and-required-status-checks > > So that people besides the WG lead can't just clobber integration or > master. > > Thoughts? > > -Todd > > > > >> >>> >>>> 2) As commented on the pull request at: >>>> >>>> https://github.com/mpiwg-tools/mpir/pull/1 >>>> >>>> we discussed some wording that goes beyond what this current pull >>>> requests reflects. Should we update the pull request, or merge it and >>>> create a new branch for further updates? >>> >>> Yeah, sorry about that -- I had to miss yesterday's call. >>> >>> We have two options: >>> >>> 1. Anh can file an entirely new PR that obsoletes mine, and we can close >>> mine. >>> >>> 2. Anh can file a pull request against >>> jsquyres:pr/mpir-being-debugged-fixes. I can accept his changes, which >>> will then merge his changes into my branch, which will automatically >>> update the existing PR. >>> >>> Either is fine. >>> >>> (#2 may sound mind blowing to those of you to whom this is a new >>> concept, but it's actually quite common -- i.e., you're effectively >>> filing a PR against a PR, and it works exactly as you would expect it to) >>> >>>> 3) In the call yesterday, we were all Git novices, thus I am not sure >>>> whether we agreed on the right workflow within the working group. Our >>>> idea was to create a branch on the working-group repo of the document, >>>> and commit to there, so multiple people can access the text from >>>> there, and we can also create the pull request to the mpi-forum from >>>> there. >>> >>> No, let's not do that. >>> >>>> Would one still need "personal forks" of the documents? >>> >>> Yes. This is a much better practice. >>> >>>> I could >>>> imagine to pull from personal forks into the created branch instead of >>>> working on the branch directly. Is that better practice? >>> >>> Yes. This is pretty much The Github Way: individual people create forks >>> and then file pull requests against the upstream repo (e.g., in our >>> case, the mpiwg-tools/mpir repo, and then ultimately the mpi-forum/mpir >>> repo). >>> >>> There is a bit of a new ground for us to figure out, though: since WGs >>> now have their own "private" working areas to create their own text, >>> etc., when exactly do we file a PR against our own WG repo and when do >>> we file a PR against the mpi-forum repo? (I think Marc-Andre asked the >>> same question) I don't have an immediate answer for this. >>> >>>> 4) If 3 makes sense, how can I merge Jeffs pull request not into the >>>> main branch of mpiwg-tools:mpir but into something like >>>> mpiwg-tools:issue_6_being_debugged? >>> >>> No. Once a PR has been created, the source and target repo:branches are >>> fixed. If you want to change the source/target, you have to close that >>> PR and open a new one. >>> >>> -- >>> Jeff Squyres >>> jsquyres(a)cisco.com >>> For corporate legal information go to: >>> http://www.cisco.com/web/about/doing_business/legal/cri/ >>> >> _______________________________________________ >> mpiwg-tools mailing list >> mpiwg-tools(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools > > -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpiwg-tools] Changes from Bordeaux to MPIR
by Todd Gamblin 09 Oct '15

09 Oct '15

On 10/9/15, 7:42 AM, "mpiwg-tools on behalf of Bland, Wesley" <mpiwg-tools-bounces(a)lists.mpi-forum.org on behalf of wesley.bland(a)intel.com> wrote: > +1. > > My thinking was that the mpi-1.x (and the similar branches for the side > documents) would be “special” in that no one should commit to them, but they > would just be mirrors of the main mpi-forum branches. If you’d like to do > something like an integration branch within your repository, that is also > fine. > > For the FTWG, my plan was to just do everything inside a branch in the FTWG > repository and not have internal pull requests unless someone has something > they’re working on outside of the working group and want to bring it in for > everyone else to see. > > > > > On 10/9/15, 9:35 AM, "mpiwg-tools on behalf of Todd Gamblin" > <mpiwg-tools-bounces(a)lists.mpi-forum.org on behalf of tgamblin(a)llnl.gov> > wrote: > >> On 10/9/15, 6:57 AM, "mpiwg-tools on behalf of Bland, Wesley" >> <mpiwg-tools-bounces(a)lists.mpi-forum.org on behalf of >> wesley.bland(a)intel.com> wrote: >> >>> I¹m not sure this solves part two of his question though. Once you accept >>> the PR into the working group¹s copy of the MPIR doc, you now have an >>> mpir-1.x branch that¹s out of sync with the main mpi-forum/mpir document. >>> This means that you open a PR for >>> mpiwg-tools/mpir/mpir-1.x:mpi-forum/mpir/mpir-1.x. This is fine until you >>> have multiple tickets going at once and want to PR each of them >>> individually. >>> >>> I was actually thinking that we¹d use the model that Marc-Andre >>> suggested. Have working groups work on their ideas in a branch on their >>> repositories, then PR from the WG repository to the mpi-forum repository. >>> I know it¹s not exactly the GitHub model, but I don¹t know that the >>> GitHub model is set up for the two-level system that we¹re working with. >> >> I like Wesley's idea. It's common for teams working closely together to >> use a naming scheme for topic branches within a single repository, and to >> do PR's *within* that repo. So this could work, and it would preserve >> topic branches for later piece-by-piece pushes into the mainline repo. >> >> It sounds like we want to use PR's for approval of tools proposals, then >> use PR's to submit them again to the mainline MPI repo. Could you use a >> local integration branch in the tools repo for this, e.g. consider the >> following branches: >> >> mpi-forum: >> master >> >> mpi-tools-wg: >> master >> integration >> topic/mpi-plugins >> topic/proposal-2 >> topic/proposal-3 >> >> Within the tools repo someone could easily submit a topic for review with >> a PR, e.g.: >> PR: topic/mpi-plugins -> integration >> >> That could be used to discuss and review the proposal. Once the topic is >> merged to integration, it can then be submitted upstream to master because >> It's still nicely packaged in a branch: >> >> PR: mpi-tools-wg:topic/mpi-plugins -> mpi-forum:master >> >> Then if it gets merged into the mainline, you just sync your fork's master >> with the upstream master (common thing to do on github). And you don't >> have the problem of submitting a bunch of merged stuff into the upstream. >> >> The integration branch and all the topic branches should be based on >> master. The integration branch itself can be transient, i.e. the tools WG >> lead can rebuild it if some PR into upstream master is not accepted. >> Anyone can check out the integration branch to see the current state of >> all locally accepted tools proposals, but you never submit a PR from >> mpi-tools-wg:integration to mpi-forum:master. mpi-tools-wg:master is >> always in sync with the upstream master. You keep the original topic >> branches around and use them as building blocks for PRs, until they're >> merged to upstream master. >> >> This is not unlike the way Kitware maintains the CMake repo, or the >> branchy workflow described in the git manpages. >> >> You can implement protections on some branches like this: >> >> https://github.com/blog/2051-protected-branches-and-required-status-checks >> >> So that people besides the WG lead can't just clobber integration or >> master. >> >> Thoughts? >> >> -Todd >> >> >> >> >>> >>>> >>>>> 2) As commented on the pull request at: >>>>> >>>>> https://github.com/mpiwg-tools/mpir/pull/1 >>>>> >>>>> we discussed some wording that goes beyond what this current pull >>>>> requests reflects. Should we update the pull request, or merge it and >>>>> create a new branch for further updates? >>>> >>>> Yeah, sorry about that -- I had to miss yesterday's call. >>>> >>>> We have two options: >>>> >>>> 1. Anh can file an entirely new PR that obsoletes mine, and we can close >>>> mine. >>>> >>>> 2. Anh can file a pull request against >>>> jsquyres:pr/mpir-being-debugged-fixes. I can accept his changes, which >>>> will then merge his changes into my branch, which will automatically >>>> update the existing PR. >>>> >>>> Either is fine. >>>> >>>> (#2 may sound mind blowing to those of you to whom this is a new >>>> concept, but it's actually quite common -- i.e., you're effectively >>>> filing a PR against a PR, and it works exactly as you would expect it to) >>>> >>>>> 3) In the call yesterday, we were all Git novices, thus I am not sure >>>>> whether we agreed on the right workflow within the working group. Our >>>>> idea was to create a branch on the working-group repo of the document, >>>>> and commit to there, so multiple people can access the text from >>>>> there, and we can also create the pull request to the mpi-forum from >>>>> there. >>>> >>>> No, let's not do that. >>>> >>>>> Would one still need "personal forks" of the documents? >>>> >>>> Yes. This is a much better practice. >>>> >>>>> I could >>>>> imagine to pull from personal forks into the created branch instead of >>>>> working on the branch directly. Is that better practice? >>>> >>>> Yes. This is pretty much The Github Way: individual people create forks >>>> and then file pull requests against the upstream repo (e.g., in our >>>> case, the mpiwg-tools/mpir repo, and then ultimately the mpi-forum/mpir >>>> repo). >>>> >>>> There is a bit of a new ground for us to figure out, though: since WGs >>>> now have their own "private" working areas to create their own text, >>>> etc., when exactly do we file a PR against our own WG repo and when do >>>> we file a PR against the mpi-forum repo? (I think Marc-Andre asked the >>>> same question) I don't have an immediate answer for this. >>>> >>>>> 4) If 3 makes sense, how can I merge Jeffs pull request not into the >>>>> main branch of mpiwg-tools:mpir but into something like >>>>> mpiwg-tools:issue_6_being_debugged? >>>> >>>> No. Once a PR has been created, the source and target repo:branches are >>>> fixed. If you want to change the source/target, you have to close that >>>> PR and open a new one. >>>> >>>> -- >>>> Jeff Squyres >>>> jsquyres(a)cisco.com >>>> For corporate legal information go to: >>>> http://www.cisco.com/web/about/doing_business/legal/cri/ >>>> >>> _______________________________________________ >>> mpiwg-tools mailing list >>> mpiwg-tools(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools >> >> >> _______________________________________________ >> mpiwg-tools mailing list >> mpiwg-tools(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools > _______________________________________________ > mpiwg-tools mailing list > mpiwg-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools

1 0

Re: [mpiwg-tools] Changes from Bordeaux to MPIR
by Bland, Wesley 09 Oct '15

09 Oct '15

+1. My thinking was that the mpi-1.x (and the similar branches for the side documents) would be “special” in that no one should commit to them, but they would just be mirrors of the main mpi-forum branches. If you’d like to do something like an integration branch within your repository, that is also fine. For the FTWG, my plan was to just do everything inside a branch in the FTWG repository and not have internal pull requests unless someone has something they’re working on outside of the working group and want to bring it in for everyone else to see. On 10/9/15, 9:35 AM, "mpiwg-tools on behalf of Todd Gamblin" <mpiwg-tools-bounces(a)lists.mpi-forum.org on behalf of tgamblin(a)llnl.gov> wrote: >On 10/9/15, 6:57 AM, "mpiwg-tools on behalf of Bland, Wesley" ><mpiwg-tools-bounces(a)lists.mpi-forum.org on behalf of >wesley.bland(a)intel.com> wrote: > >>I¹m not sure this solves part two of his question though. Once you accept >>the PR into the working group¹s copy of the MPIR doc, you now have an >>mpir-1.x branch that¹s out of sync with the main mpi-forum/mpir document. >>This means that you open a PR for >>mpiwg-tools/mpir/mpir-1.x:mpi-forum/mpir/mpir-1.x. This is fine until you >>have multiple tickets going at once and want to PR each of them >>individually. >> >>I was actually thinking that we¹d use the model that Marc-Andre >>suggested. Have working groups work on their ideas in a branch on their >>repositories, then PR from the WG repository to the mpi-forum repository. >>I know it¹s not exactly the GitHub model, but I don¹t know that the >>GitHub model is set up for the two-level system that we¹re working with. > >I like Wesley's idea. It's common for teams working closely together to >use a naming scheme for topic branches within a single repository, and to >do PR's *within* that repo. So this could work, and it would preserve >topic branches for later piece-by-piece pushes into the mainline repo. > >It sounds like we want to use PR's for approval of tools proposals, then >use PR's to submit them again to the mainline MPI repo. Could you use a >local integration branch in the tools repo for this, e.g. consider the >following branches: > >mpi-forum: > master > >mpi-tools-wg: > master > integration > topic/mpi-plugins > topic/proposal-2 > topic/proposal-3 > >Within the tools repo someone could easily submit a topic for review with >a PR, e.g.: > > PR: topic/mpi-plugins -> integration > >That could be used to discuss and review the proposal. Once the topic is >merged to integration, it can then be submitted upstream to master because >It's still nicely packaged in a branch: > > PR: mpi-tools-wg:topic/mpi-plugins -> mpi-forum:master > >Then if it gets merged into the mainline, you just sync your fork's master >with the upstream master (common thing to do on github). And you don't >have the problem of submitting a bunch of merged stuff into the upstream. > >The integration branch and all the topic branches should be based on >master. The integration branch itself can be transient, i.e. the tools WG >lead can rebuild it if some PR into upstream master is not accepted. >Anyone can check out the integration branch to see the current state of >all locally accepted tools proposals, but you never submit a PR from >mpi-tools-wg:integration to mpi-forum:master. mpi-tools-wg:master is >always in sync with the upstream master. You keep the original topic >branches around and use them as building blocks for PRs, until they're >merged to upstream master. > >This is not unlike the way Kitware maintains the CMake repo, or the >branchy workflow described in the git manpages. > >You can implement protections on some branches like this: > > https://github.com/blog/2051-protected-branches-and-required-status-checks > >So that people besides the WG lead can't just clobber integration or >master. > >Thoughts? > >-Todd > > > > >> >>> >>>> 2) As commented on the pull request at: >>>> >>>> https://github.com/mpiwg-tools/mpir/pull/1 >>>> >>>> we discussed some wording that goes beyond what this current pull >>>> requests reflects. Should we update the pull request, or merge it and >>>> create a new branch for further updates? >>> >>>Yeah, sorry about that -- I had to miss yesterday's call. >>> >>>We have two options: >>> >>>1. Anh can file an entirely new PR that obsoletes mine, and we can close >>>mine. >>> >>>2. Anh can file a pull request against >>>jsquyres:pr/mpir-being-debugged-fixes. I can accept his changes, which >>>will then merge his changes into my branch, which will automatically >>>update the existing PR. >>> >>>Either is fine. >>> >>>(#2 may sound mind blowing to those of you to whom this is a new >>>concept, but it's actually quite common -- i.e., you're effectively >>>filing a PR against a PR, and it works exactly as you would expect it to) >>> >>>> 3) In the call yesterday, we were all Git novices, thus I am not sure >>>> whether we agreed on the right workflow within the working group. Our >>>> idea was to create a branch on the working-group repo of the document, >>>> and commit to there, so multiple people can access the text from >>>> there, and we can also create the pull request to the mpi-forum from >>>> there. >>> >>>No, let's not do that. >>> >>>> Would one still need "personal forks" of the documents? >>> >>>Yes. This is a much better practice. >>> >>>> I could >>>> imagine to pull from personal forks into the created branch instead of >>>> working on the branch directly. Is that better practice? >>> >>>Yes. This is pretty much The Github Way: individual people create forks >>>and then file pull requests against the upstream repo (e.g., in our >>>case, the mpiwg-tools/mpir repo, and then ultimately the mpi-forum/mpir >>>repo). >>> >>>There is a bit of a new ground for us to figure out, though: since WGs >>>now have their own "private" working areas to create their own text, >>>etc., when exactly do we file a PR against our own WG repo and when do >>>we file a PR against the mpi-forum repo? (I think Marc-Andre asked the >>>same question) I don't have an immediate answer for this. >>> >>>> 4) If 3 makes sense, how can I merge Jeffs pull request not into the >>>> main branch of mpiwg-tools:mpir but into something like >>>> mpiwg-tools:issue_6_being_debugged? >>> >>>No. Once a PR has been created, the source and target repo:branches are >>>fixed. If you want to change the source/target, you have to close that >>>PR and open a new one. >>> >>>-- >>>Jeff Squyres >>>jsquyres(a)cisco.com >>>For corporate legal information go to: >>>http://www.cisco.com/web/about/doing_business/legal/cri/ >>> >>_______________________________________________ >>mpiwg-tools mailing list >>mpiwg-tools(a)lists.mpi-forum.org >>http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools > > >_______________________________________________ >mpiwg-tools mailing list >mpiwg-tools(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-tools

1 0