May 2013 - mpiwg-tools - mailman3.cels.anl.gov

Re: [Mpi3-tools] latest draft of mqs
by Anh Vo 22 May '13

22 May '13

Comments inline From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Kathryn Mohror Sent: Wednesday, May 22, 2013 1:44 PM To: mpi3-tools(a)lists.mpi-forum.org Subject: Re: [Mpi3-tools] latest draft of mqs Hi Ahn, comments inline: Page 2, Comment JD4 John's comments about lots of hyphenated identifiers at end of line. Some of it are artifacts of converting PDF -> Word, but I think this is a common (and widely accepted) problems of latex documents. So we will probably have to live with it unless there're some magic way to make it nicer. I searched the standard (with PDF viewer search) and didn't see any broken hyphenated identifiers at the ends of lines. I noted that in the standard type and function names are like so: MPI\_COMM\_WORLD while in MPIR and MQS they are like so: MPI\_\-COMM\_\-WORLD. If you remove the \- will it do the right thing? I am not a Latex guru so I don't know. [anhvo]: Yes, I think I mainly followed the practice of the latex in MPIR and put the \- there, it basically tells Latex it can break off the lines there. Not sure which one is more desirable? Page 9 (Page 14 in Word), Comment JD18 John recommended the formatted table similar to what the MPIR document had. For example: Char MPIR_dll_name[] Definition is required Definition is contained within the address space of the MPI process Variable is written by the MPI process, and read by the tool I thought I did ask about this in the conf call at some point (or maybe it was an email, I can't remember the exact time and place) and we agreed that we would not be using that format because many people in the forum thought it was weird looking. Well, my opinion is that it should be consistent with what is in MPIR since it's supposed to be part of the same document. I don't remember the 'weird looking' comment, but I wouldn't be surprised if someone made it. [anhvo] I think Jeff mentioned this in the scanned PDF he sent back to me. And I agreed with Jeff so I removed it. I might be in the minority but I don't necessarily agree that we should have similar format with the MPIR Document. They're both side documents but they're quite different. In MPIR there are many variables so it's quite important to mention where they should stay and who is writing, reading it. In MQS we separate callbacks by debugger and DLL so it's quite clear who should define what Page 9 (Page 14 in Word), Comment JD19: I think we agreed that we would not include the mpimsgq_dll_locations in this version and leave all extensions/improvements to document V1.1 or V2.0 Right, I agree that we're only trying to document what the state of things is. However, he says the at OpenMPI implements this. Is that so Jeff? And does a debugger support it? If so, I'd think we want to mention it. [anhvo]: I'll let Jeff comment on this. Page 15 (Page 21 in Word),Comment JD26 Should we include semicolon at the end of the typedef definition? When I first created the document, I also debated a bit between this. I ended up following the style of the MPI standard where it lists the C declaration without any ; I'm fine either way, but would like to hear others' opinions on this. I think the standard does use ; . At least I didn't see any -- did I miss what you mean? [anhvo]: The standard does not put semicolon after a C definition. Randomly picking: page 38, C definition of MPI_BSend: int MPI_Bsend(...) (no semicolon at the end) Thanks for your hard work Ahn! Kathryn I'm leaving for lunch now. More comments after I'm back. In the meantime, if folks can comments on what I've sent so far, that'd be great!! Thanks Anh -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Anh Vo Sent: Wednesday, May 22, 2013 8:10 AM To: mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> Subject: Re: [Mpi3-tools] latest draft of mqs The definitions John provide look great for me. Unless there are other opinions, I will use those. --Anh -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Ahn, Dong H. Sent: Wednesday, May 22, 2013 7:54 AM To: mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> Subject: Re: [Mpi3-tools] latest draft of mqs Hi John, This works for me and should clarify a few things in the current draft. Best, Dong -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of John DelSignore Sent: Wednesday, May 22, 2013 9:55 AM To: mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> Subject: Re: [Mpi3-tools] latest draft of mqs I prefer to not muddy up the "standard definitions" (or maybe "common-sense definitions") of commonly used term, such as "image file", to make them conform to the MQD interface. Instead, I like to define the MQD typedefs in terms of the standard definitions. So, WRT to the document, I think we should include the following definitions: * An "image file" is an executable or shared library file, that may contain symbol definitions needed by the MQD interface. * An "MPI process" consists of an "address space" and a collection of execution contexts (threads or lightweight processes). * An "address space" is a region of memory that consists of executable code and data, and is partially composed of a collection of image files. The collection of image files may change at any point during the execution of the MPI process, and image files may be relocated at runtime within the address space at the point they are loaded into memory. Then we can say: * An "mqs_image" is an abstract concept that represents the collection of image files loaded into the address space of the MPI process at any given time, and is debugger implementation defined. In static execution environments, where shared libraries are not supported, an mqs_image can represent an executable image file. However, in dynamic execution environments, where shared libraries, dynamically loaded shared libraries, and runtime relocation of shared libraries are supported, an mqs_image should represent the collection of image files loaded into the address space of the MPI process at any given point in time; in this situation, mqs_image may in fact represent the MPI process itself. Cheers, John D. Anh Vo wrote: Thanks Kathryn for starting up the discussion. I'll start with my existing definition of mqs_image in the document, which definitely is not adequate. Existing definition: An image is an executable file that was loaded into memory when a process is started. For SIMD-style programs, all MPI processes have the same image. For MIMD-style programs, MPI processes might have different images. Dong's comments: The definition of "Image" isn't all that rigorous here. I think this creates some confusion somewhere down this document. Is an image really an executable file? Which typically means the base executable or does this refer to both the based executable and DSOs being loaded into the process address space? Though it sounds hair-splitting, even the SPMD remark, the statement is only true if the image is defined to be the base executable. John's comments: I think we should be more general with the term "image" to mean executable or shared library. The reason is that we do not want to require symbols to reside in the executable. We should allow symbols to reside in shared libraries. Also, I think we need to define the "address space" of a process, and an "image list" as a collection of image files that have been loaded in the address space, and possibly relocated at runtime. I think I agree with both comments. Image should not refer to a single executable image (a.out or a.exe), but rather the base executable and the shared libraries that were loaded into the process address space. So how about this new definition: An image represents the executable (e.g., a.out) and the collection of dynamically loaded libraries that were loaded into the process address space. --Anh *From:* mpi3-tools-bounces(a)lists.mpi-forum.org<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] *On Behalf Of *Kathryn Mohror *Sent:* Tuesday, May 21, 2013 6:27 PM *To:* mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> *Subject:* Re: [Mpi3-tools] latest draft of mqs Hi John, Thank you for the extensive feedback and help with the document! I'm sure you will get the 'nod' you referred to on page 1 :-) For the definition of mqs_image, I agree that it needs work. Dong Ahn also mentioned that it needed to be more rigorously defined. However if possible, I think it would be beneficial if we could at least start the conversation before next week so that we have a better chance of having it nailed down in advance of the next meeting. So, that said -- does anyone want to take a first stab at a definition to get us started? Kathryn On May 21, 2013, at 2:18 PM, John DelSignore <John.DelSignore(a)roguewave.com<mailto:[email protected]> <mailto:[email protected]>> wrote: Hi, Attached are my edits and comments on the MPI Message Queue document. Anh provided me with a Word document generated from the LaTex, so the formatting is funky in some places. I enabled change tracking in Word before making my edits, and I added many comments to the document in places where I think we have more work to do. Unfortunately, I think the changes might be somewhat extensive. The volume of edits and comments aside, I think Anh did a great job! The first draft of the document closely reflects the 10+ year-old description of the interface pretty well. Unfortunately, the original interface has some problem that we need to resolve, most critically, we have to resolve how to describe the concept of "mqs_image". We can talk about this during the next concall. Cheers, John D. Anh Vo wrote: Hi all, As mentioned on the tool WG call today, I'm re-sending the latest draft. Feedbacks are appreciated. We'll send this one out to the wider audience on Tuesday 5/21 to meet the requirement for first reading in June --Anh <MPI-MQD-JVD-2013-05-17.doc>_______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> <mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools ______________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov<mailto:[email protected]> <mailto:[email protected]>, http://people.llnl.gov/mohror1 CASC @ Lawrence Livermore National Laboratory, Livermore, CA, USA _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools <msgq.pdf>_______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools ______________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov<mailto:[email protected]>, http://people.llnl.gov/mohror1 CASC @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0

Re: [Mpi3-tools] latest draft of mqs
by Jeff Squyres (jsquyres) 22 May '13

22 May '13

On May 22, 2013, at 4:44 PM, Kathryn Mohror <kathryn(a)llnl.gov> wrote: > I searched the standard (with PDF viewer search) and didn't see any broken hyphenated identifiers at the ends of lines. I noted that in the standard type and function names are like so: MPI\_COMM\_WORLD while in MPIR and MQS they are like so: MPI\_\-COMM\_\-WORLD. If you remove the \- will it do the right thing? I am not a Latex guru so I don't know. \- tells Latex that it's ok to hyphenate there if it wants to. So you might see "MPI_-" at the end of a line, and "COMM_WORLD" at the beginning of the next. I'm personally ok with that kind of hyphenation -- I always put in the \-'s between words of MPI constants, a la MPI\_\-COMM\_\-WORLD. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [Mpi3-tools] latest draft of mqs
by Kathryn Mohror 22 May '13

22 May '13

Hi Ahn, comments inline: > > Page 1, Comment JD3 > Existing text: "the MPI Forum published a document which formally standardized the MPIR Process Acquisition Interface" > John's comments that we should not use the word standardized, but rather should say that "the MPI Forum published a document which formally described..." > My impression was that the MPIR was standardized (i.e., it went through the voting process), but the quality of the document was not on-par with the rest of the standard so it was not included in the official document. But I wasn't there when that happened so I would like more background and context here. John is right. We documented the state of the practice, but did not standardize it per se. I think of it as 'blessed by the forum.' > Page 2, Comment JD4 > John's comments about lots of hyphenated identifiers at end of line. Some of it are artifacts of converting PDF -> Word, but I think this is a common (and widely accepted) problems of latex documents. So we will probably have to live with it unless there're some magic way to make it nicer. I searched the standard (with PDF viewer search) and didn't see any broken hyphenated identifiers at the ends of lines. I noted that in the standard type and function names are like so: MPI\_COMM\_WORLD while in MPIR and MQS they are like so: MPI\_\-COMM\_\-WORLD. If you remove the \- will it do the right thing? I am not a Latex guru so I don't know. > Page 9 (Page 14 in Word), Comment JD18 > John recommended the formatted table similar to what the MPIR document had. For example: > > Char MPIR_dll_name[] > Definition is required > Definition is contained within the address space of the MPI process > Variable is written by the MPI process, and read by the tool > > I thought I did ask about this in the conf call at some point (or maybe it was an email, I can't remember the exact time and place) and we agreed that we would not be using that format because many people in the forum thought it was weird looking. Well, my opinion is that it should be consistent with what is in MPIR since it's supposed to be part of the same document. I don't remember the 'weird looking' comment, but I wouldn't be surprised if someone made it. > Page 9 (Page 14 in Word), Comment JD19: > I think we agreed that we would not include the mpimsgq_dll_locations in this version and leave all extensions/improvements to document V1.1 or V2.0 Right, I agree that we're only trying to document what the state of things is. However, he says the at OpenMPI implements this. Is that so Jeff? And does a debugger support it? If so, I'd think we want to mention it. > Page 15 (Page 21 in Word),Comment JD26 > Should we include semicolon at the end of the typedef definition? When I first created the document, I also debated a bit between this. I ended up following the style of the MPI standard where it lists the C declaration without any ; I'm fine either way, but would like to hear others' opinions on this. I think the standard does use ; . At least I didn't see any -- did I miss what you mean? Thanks for your hard work Ahn! Kathryn > > I'm leaving for lunch now. More comments after I'm back. In the meantime, if folks can comments on what I've sent so far, that'd be great!! > > Thanks > Anh > > -----Original Message----- > From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Anh Vo > Sent: Wednesday, May 22, 2013 8:10 AM > To: mpi3-tools(a)lists.mpi-forum.org > Subject: Re: [Mpi3-tools] latest draft of mqs > > The definitions John provide look great for me. Unless there are other opinions, I will use those. > > --Anh > > -----Original Message----- > From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Ahn, Dong H. > Sent: Wednesday, May 22, 2013 7:54 AM > To: mpi3-tools(a)lists.mpi-forum.org > Subject: Re: [Mpi3-tools] latest draft of mqs > > Hi John, > > This works for me and should clarify a few things in the current draft. > > Best, > Dong > >> -----Original Message----- >> From: mpi3-tools-bounces(a)lists.mpi-forum.org >> [mailto:[email protected]] On Behalf Of John >> DelSignore >> Sent: Wednesday, May 22, 2013 9:55 AM >> To: mpi3-tools(a)lists.mpi-forum.org >> Subject: Re: [Mpi3-tools] latest draft of mqs >> >> I prefer to not muddy up the "standard definitions" (or maybe >> "common-sense definitions") of commonly used term, such as "image >> file", to make them conform to the MQD interface. Instead, I like to define the MQD typedefs in terms of the standard definitions. So, WRT to the document, I think we should include the following definitions: >> >> * An "image file" is an executable or shared library file, that may contain symbol definitions needed by the MQD interface. >> >> * An "MPI process" consists of an "address space" and a collection of execution contexts (threads or lightweight processes). >> >> * An "address space" is a region of memory that consists of executable >> code and data, and is partially composed of a collection of image >> files. The collection of image files may change at any point during the execution of the MPI process, and image files may be relocated at runtime within the address space at the point they are loaded into memory. >> >> Then we can say: >> >> * An "mqs_image" is an abstract concept that represents the collection >> of image files loaded into the address space of the MPI process at any >> given time, and is debugger implementation defined. In static >> execution environments, where shared libraries are not supported, an >> mqs_image can represent an executable image file. However, in dynamic >> execution environments, where shared libraries, dynamically loaded shared libraries, and runtime relocation of shared libraries are supported, an mqs_image should represent the collection of image files loaded into the address space of the MPI process at any given point in time; in this situation, mqs_image may in fact represent the MPI process itself. >> >> Cheers, John D. >> >> >> Anh Vo wrote: >>> Thanks Kathryn for starting up the discussion. >>> >>> >>> >>> I'll start with my existing definition of mqs_image in the document, >>> which definitely is not adequate. >>> >>> >>> >>> Existing definition: >>> >>> An image is an executable file that was loaded into memory when >>> >>> a process is started. For SIMD-style programs, all MPI processes >>> have >>> >>> the same image. For MIMD-style programs, MPI processes might have >>> >>> different images. >>> >>> >>> >>> Dong's comments: >>> >>> The definition of "Image" isn't all that rigorous here. I think this >>> creates some confusion somewhere down this document. Is an image >>> really an executable file? Which typically means the base >>> executable or does this refer to both the based executable and DSOs >>> being loaded into the process address space? Though it sounds >>> hair-splitting, even the SPMD remark, the statement is only true if >>> the image is defined to be the base executable. >>> >>> >>> >>> John's comments: >>> >>> I think we should be more general with the term "image" to mean >>> executable or shared library. The reason is that we do not want to >>> require symbols to reside in the executable. We should allow symbols >>> to reside in shared libraries. >>> >>> Also, I think we need to define the "address space" of a process, >>> and an "image list" as a collection of image files that have been >>> loaded in the address space, and possibly relocated at runtime. >>> >>> >>> >>> I think I agree with both comments. Image should not refer to a >>> single executable image (a.out or a.exe), but rather the base >>> executable and the shared libraries that were loaded into the process address space. >>> >>> >>> >>> So how about this new definition: >>> >>> An image represents the executable (e.g., a.out) and the collection >>> of dynamically loaded libraries that were loaded into the process >>> address space. >>> >>> >>> >>> --Anh >>> >>> *From:* mpi3-tools-bounces(a)lists.mpi-forum.org >>> [mailto:[email protected]] *On Behalf Of >>> *Kathryn Mohror >>> *Sent:* Tuesday, May 21, 2013 6:27 PM >>> *To:* mpi3-tools(a)lists.mpi-forum.org >>> *Subject:* Re: [Mpi3-tools] latest draft of mqs >>> >>> >>> >>> Hi John, >>> >>> >>> >>> Thank you for the extensive feedback and help with the document! I'm >>> sure you will get the 'nod' you referred to on page 1 :-) >>> >>> >>> >>> For the definition of mqs_image, I agree that it needs work. Dong >>> Ahn also mentioned that it needed to be more rigorously defined. >>> However if possible, I think it would be beneficial if we could at >>> least start the conversation before next week so that we have a >>> better chance of having it nailed down in advance of the next meeting. >>> >>> >>> >>> So, that said -- does anyone want to take a first stab at a >>> definition to get us started? >>> >>> >>> >>> Kathryn >>> >>> >>> >>> On May 21, 2013, at 2:18 PM, John DelSignore >>> <John.DelSignore(a)roguewave.com >>> <mailto:[email protected]>> >>> wrote: >>> >>> >>> >>> Hi, >>> >>> Attached are my edits and comments on the MPI Message Queue >>> document. Anh provided me with a Word document generated from the >>> LaTex, so the formatting is funky in some places. I enabled change >>> tracking in Word before making my edits, and I added many comments >>> to the document in places where I think we have more work to do. >>> Unfortunately, I think the changes might be somewhat extensive. >>> >>> The volume of edits and comments aside, I think Anh did a great job! >>> The first draft of the document closely reflects the 10+ year-old >>> description of the interface pretty well. Unfortunately, the >>> original interface has some problem that we need to resolve, most >>> critically, we have to resolve how to describe the concept of >>> "mqs_image". We can talk about this during the next concall. >>> >>> Cheers, John D. >>> >>> >>> Anh Vo wrote: >>> >>> Hi all, >>> >>> As mentioned on the tool WG call today, I'm re-sending the >>> latest draft. >>> Feedbacks are appreciated. We'll send this one out to the wider >>> audience >>> on Tuesday 5/21 to meet the requirement for first reading in >>> June >>> >>> >>> >>> --Anh >>> >>> <MPI-MQD-JVD-2013-05-17.doc>_______________________________________________ >>> Mpi3-tools mailing list >>> Mpi3-tools(a)lists.mpi-forum.org <mailto:[email protected]> >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools >>> >>> >>> >>> ______________________________________________________________ >>> Kathryn Mohror, kathryn(a)llnl.gov >>> <mailto:[email protected]>, http://people.llnl.gov/mohror1 CASC @ >>> Lawrence Livermore National Laboratory, Livermore, CA, USA >>> >>> >>> >>> >>> >>> >>> >> _______________________________________________ >> Mpi3-tools mailing list >> Mpi3-tools(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > > > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > > > <msgq.pdf>_______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools ______________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov, http://people.llnl.gov/mohror1 CASC @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0

Re: [Mpi3-tools] latest draft of mqs
by Anh Vo 22 May '13

22 May '13

Continue from where I left off Page 15 (Word page 22) I believe the typedef for function mqs_put_image_info should mark image as OUT (which is what I have in the existing document now), but one of John's edit changed it to IN This callback should be provided by the debugger so I'm not 100% clear, but based on the example tvtest.c, I see put_image_info being defined as follows: static void dbgrI_put_image_info( mqs_image *image, mqs_image_info *info ) { image->i_info = info; } This means image needs to be marked as OUT because we're modifying image. Even though we're only modifying a field of "image", we're still modifying it. Now, it could also be INOUT, if we happen to read the image, but without a debugger's source code, I cannot tell for sure. --Anh -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Anh Vo Sent: Wednesday, May 22, 2013 12:31 PM To: mpi3-tools(a)lists.mpi-forum.org Subject: Re: [Mpi3-tools] latest draft of mqs Hi all, I've addressed almost all of John's edits and comments (Many thanks to John!). There are a few items that I would like to get some feedbacks and consensus: Page number refers to the PDF document (although I try to include the Word page number and comment number for easy reference) Page 1, Comment JD3 Existing text: "the MPI Forum published a document which formally standardized the MPIR Process Acquisition Interface" John's comments that we should not use the word standardized, but rather should say that "the MPI Forum published a document which formally described..." My impression was that the MPIR was standardized (i.e., it went through the voting process), but the quality of the document was not on-par with the rest of the standard so it was not included in the official document. But I wasn't there when that happened so I would like more background and context here. Page 2, Comment JD4 John's comments about lots of hyphenated identifiers at end of line. Some of it are artifacts of converting PDF -> Word, but I think this is a common (and widely accepted) problems of latex documents. So we will probably have to live with it unless there're some magic way to make it nicer. Page 9 (Page 14 in Word), Comment JD18 John recommended the formatted table similar to what the MPIR document had. For example: Char MPIR_dll_name[] Definition is required Definition is contained within the address space of the MPI process Variable is written by the MPI process, and read by the tool I thought I did ask about this in the conf call at some point (or maybe it was an email, I can't remember the exact time and place) and we agreed that we would not be using that format because many people in the forum thought it was weird looking. Page 9 (Page 14 in Word), Comment JD19: I think we agreed that we would not include the mpimsgq_dll_locations in this version and leave all extensions/improvements to document V1.1 or V2.0 Page 15 (Page 21 in Word),Comment JD26 Should we include semicolon at the end of the typedef definition? When I first created the document, I also debated a bit between this. I ended up following the style of the MPI standard where it lists the C declaration without any ; I'm fine either way, but would like to hear others' opinions on this. I'm leaving for lunch now. More comments after I'm back. In the meantime, if folks can comments on what I've sent so far, that'd be great!! Thanks Anh -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Anh Vo Sent: Wednesday, May 22, 2013 8:10 AM To: mpi3-tools(a)lists.mpi-forum.org Subject: Re: [Mpi3-tools] latest draft of mqs The definitions John provide look great for me. Unless there are other opinions, I will use those. --Anh -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Ahn, Dong H. Sent: Wednesday, May 22, 2013 7:54 AM To: mpi3-tools(a)lists.mpi-forum.org Subject: Re: [Mpi3-tools] latest draft of mqs Hi John, This works for me and should clarify a few things in the current draft. Best, Dong > -----Original Message----- > From: mpi3-tools-bounces(a)lists.mpi-forum.org > [mailto:[email protected]] On Behalf Of John > DelSignore > Sent: Wednesday, May 22, 2013 9:55 AM > To: mpi3-tools(a)lists.mpi-forum.org > Subject: Re: [Mpi3-tools] latest draft of mqs > > I prefer to not muddy up the "standard definitions" (or maybe > "common-sense definitions") of commonly used term, such as "image > file", to make them conform to the MQD interface. Instead, I like to define the MQD typedefs in terms of the standard definitions. So, WRT to the document, I think we should include the following definitions: > > * An "image file" is an executable or shared library file, that may contain symbol definitions needed by the MQD interface. > > * An "MPI process" consists of an "address space" and a collection of execution contexts (threads or lightweight processes). > > * An "address space" is a region of memory that consists of executable > code and data, and is partially composed of a collection of image > files. The collection of image files may change at any point during the execution of the MPI process, and image files may be relocated at runtime within the address space at the point they are loaded into memory. > > Then we can say: > > * An "mqs_image" is an abstract concept that represents the collection > of image files loaded into the address space of the MPI process at any > given time, and is debugger implementation defined. In static > execution environments, where shared libraries are not supported, an > mqs_image can represent an executable image file. However, in dynamic > execution environments, where shared libraries, dynamically loaded shared libraries, and runtime relocation of shared libraries are supported, an mqs_image should represent the collection of image files loaded into the address space of the MPI process at any given point in time; in this situation, mqs_image may in fact represent the MPI process itself. > > Cheers, John D. > > > Anh Vo wrote: > > Thanks Kathryn for starting up the discussion. > > > > > > > > I'll start with my existing definition of mqs_image in the document, > > which definitely is not adequate. > > > > > > > > Existing definition: > > > > An image is an executable file that was loaded into memory when > > > > a process is started. For SIMD-style programs, all MPI processes > > have > > > > the same image. For MIMD-style programs, MPI processes might have > > > > different images. > > > > > > > > Dong's comments: > > > > The definition of "Image" isn't all that rigorous here. I think this > > creates some confusion somewhere down this document. Is an image > > really an executable file? Which typically means the base > > executable or does this refer to both the based executable and DSOs > > being loaded into the process address space? Though it sounds > > hair-splitting, even the SPMD remark, the statement is only true if > > the image is defined to be the base executable. > > > > > > > > John's comments: > > > > I think we should be more general with the term "image" to mean > > executable or shared library. The reason is that we do not want to > > require symbols to reside in the executable. We should allow symbols > > to reside in shared libraries. > > > > Also, I think we need to define the "address space" of a process, > > and an "image list" as a collection of image files that have been > > loaded in the address space, and possibly relocated at runtime. > > > > > > > > I think I agree with both comments. Image should not refer to a > > single executable image (a.out or a.exe), but rather the base > > executable and the shared libraries that were loaded into the process address space. > > > > > > > > So how about this new definition: > > > > An image represents the executable (e.g., a.out) and the collection > > of dynamically loaded libraries that were loaded into the process > > address space. > > > > > > > > --Anh > > > > *From:* mpi3-tools-bounces(a)lists.mpi-forum.org > > [mailto:[email protected]] *On Behalf Of > > *Kathryn Mohror > > *Sent:* Tuesday, May 21, 2013 6:27 PM > > *To:* mpi3-tools(a)lists.mpi-forum.org > > *Subject:* Re: [Mpi3-tools] latest draft of mqs > > > > > > > > Hi John, > > > > > > > > Thank you for the extensive feedback and help with the document! I'm > > sure you will get the 'nod' you referred to on page 1 :-) > > > > > > > > For the definition of mqs_image, I agree that it needs work. Dong > > Ahn also mentioned that it needed to be more rigorously defined. > > However if possible, I think it would be beneficial if we could at > > least start the conversation before next week so that we have a > > better chance of having it nailed down in advance of the next meeting. > > > > > > > > So, that said -- does anyone want to take a first stab at a > > definition to get us started? > > > > > > > > Kathryn > > > > > > > > On May 21, 2013, at 2:18 PM, John DelSignore > > <John.DelSignore(a)roguewave.com > > <mailto:[email protected]>> > > wrote: > > > > > > > > Hi, > > > > Attached are my edits and comments on the MPI Message Queue > > document. Anh provided me with a Word document generated from the > > LaTex, so the formatting is funky in some places. I enabled change > > tracking in Word before making my edits, and I added many comments > > to the document in places where I think we have more work to do. > > Unfortunately, I think the changes might be somewhat extensive. > > > > The volume of edits and comments aside, I think Anh did a great job! > > The first draft of the document closely reflects the 10+ year-old > > description of the interface pretty well. Unfortunately, the > > original interface has some problem that we need to resolve, most > > critically, we have to resolve how to describe the concept of > > "mqs_image". We can talk about this during the next concall. > > > > Cheers, John D. > > > > > > Anh Vo wrote: > > > > Hi all, > > > > As mentioned on the tool WG call today, I'm re-sending the > > latest draft. > > Feedbacks are appreciated. We'll send this one out to the wider > > audience > > on Tuesday 5/21 to meet the requirement for first reading in > > June > > > > > > > > --Anh > > > > <MPI-MQD-JVD-2013-05-17.doc>_______________________________________________ > > Mpi3-tools mailing list > > Mpi3-tools(a)lists.mpi-forum.org <mailto:[email protected]> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > > > > > > > > ______________________________________________________________ > > Kathryn Mohror, kathryn(a)llnl.gov > > <mailto:[email protected]>, http://people.llnl.gov/mohror1 CASC @ > > Lawrence Livermore National Laboratory, Livermore, CA, USA > > > > > > > > > > > > > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools

1 0

Re: [Mpi3-tools] latest draft of mqs
by Anh Vo 22 May '13

22 May '13

Hi all, I've addressed almost all of John's edits and comments (Many thanks to John!). There are a few items that I would like to get some feedbacks and consensus: Page number refers to the PDF document (although I try to include the Word page number and comment number for easy reference) Page 1, Comment JD3 Existing text: "the MPI Forum published a document which formally standardized the MPIR Process Acquisition Interface" John's comments that we should not use the word standardized, but rather should say that "the MPI Forum published a document which formally described..." My impression was that the MPIR was standardized (i.e., it went through the voting process), but the quality of the document was not on-par with the rest of the standard so it was not included in the official document. But I wasn't there when that happened so I would like more background and context here. Page 2, Comment JD4 John's comments about lots of hyphenated identifiers at end of line. Some of it are artifacts of converting PDF -> Word, but I think this is a common (and widely accepted) problems of latex documents. So we will probably have to live with it unless there're some magic way to make it nicer. Page 9 (Page 14 in Word), Comment JD18 John recommended the formatted table similar to what the MPIR document had. For example: Char MPIR_dll_name[] Definition is required Definition is contained within the address space of the MPI process Variable is written by the MPI process, and read by the tool I thought I did ask about this in the conf call at some point (or maybe it was an email, I can't remember the exact time and place) and we agreed that we would not be using that format because many people in the forum thought it was weird looking. Page 9 (Page 14 in Word), Comment JD19: I think we agreed that we would not include the mpimsgq_dll_locations in this version and leave all extensions/improvements to document V1.1 or V2.0 Page 15 (Page 21 in Word),Comment JD26 Should we include semicolon at the end of the typedef definition? When I first created the document, I also debated a bit between this. I ended up following the style of the MPI standard where it lists the C declaration without any ; I'm fine either way, but would like to hear others' opinions on this. I'm leaving for lunch now. More comments after I'm back. In the meantime, if folks can comments on what I've sent so far, that'd be great!! Thanks Anh -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Anh Vo Sent: Wednesday, May 22, 2013 8:10 AM To: mpi3-tools(a)lists.mpi-forum.org Subject: Re: [Mpi3-tools] latest draft of mqs The definitions John provide look great for me. Unless there are other opinions, I will use those. --Anh -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Ahn, Dong H. Sent: Wednesday, May 22, 2013 7:54 AM To: mpi3-tools(a)lists.mpi-forum.org Subject: Re: [Mpi3-tools] latest draft of mqs Hi John, This works for me and should clarify a few things in the current draft. Best, Dong > -----Original Message----- > From: mpi3-tools-bounces(a)lists.mpi-forum.org > [mailto:[email protected]] On Behalf Of John > DelSignore > Sent: Wednesday, May 22, 2013 9:55 AM > To: mpi3-tools(a)lists.mpi-forum.org > Subject: Re: [Mpi3-tools] latest draft of mqs > > I prefer to not muddy up the "standard definitions" (or maybe > "common-sense definitions") of commonly used term, such as "image > file", to make them conform to the MQD interface. Instead, I like to define the MQD typedefs in terms of the standard definitions. So, WRT to the document, I think we should include the following definitions: > > * An "image file" is an executable or shared library file, that may contain symbol definitions needed by the MQD interface. > > * An "MPI process" consists of an "address space" and a collection of execution contexts (threads or lightweight processes). > > * An "address space" is a region of memory that consists of executable > code and data, and is partially composed of a collection of image > files. The collection of image files may change at any point during the execution of the MPI process, and image files may be relocated at runtime within the address space at the point they are loaded into memory. > > Then we can say: > > * An "mqs_image" is an abstract concept that represents the collection > of image files loaded into the address space of the MPI process at any > given time, and is debugger implementation defined. In static > execution environments, where shared libraries are not supported, an > mqs_image can represent an executable image file. However, in dynamic > execution environments, where shared libraries, dynamically loaded shared libraries, and runtime relocation of shared libraries are supported, an mqs_image should represent the collection of image files loaded into the address space of the MPI process at any given point in time; in this situation, mqs_image may in fact represent the MPI process itself. > > Cheers, John D. > > > Anh Vo wrote: > > Thanks Kathryn for starting up the discussion. > > > > > > > > I'll start with my existing definition of mqs_image in the document, > > which definitely is not adequate. > > > > > > > > Existing definition: > > > > An image is an executable file that was loaded into memory when > > > > a process is started. For SIMD-style programs, all MPI processes > > have > > > > the same image. For MIMD-style programs, MPI processes might have > > > > different images. > > > > > > > > Dong's comments: > > > > The definition of "Image" isn't all that rigorous here. I think this > > creates some confusion somewhere down this document. Is an image > > really an executable file? Which typically means the base > > executable or does this refer to both the based executable and DSOs > > being loaded into the process address space? Though it sounds > > hair-splitting, even the SPMD remark, the statement is only true if > > the image is defined to be the base executable. > > > > > > > > John's comments: > > > > I think we should be more general with the term "image" to mean > > executable or shared library. The reason is that we do not want to > > require symbols to reside in the executable. We should allow symbols > > to reside in shared libraries. > > > > Also, I think we need to define the "address space" of a process, > > and an "image list" as a collection of image files that have been > > loaded in the address space, and possibly relocated at runtime. > > > > > > > > I think I agree with both comments. Image should not refer to a > > single executable image (a.out or a.exe), but rather the base > > executable and the shared libraries that were loaded into the process address space. > > > > > > > > So how about this new definition: > > > > An image represents the executable (e.g., a.out) and the collection > > of dynamically loaded libraries that were loaded into the process > > address space. > > > > > > > > --Anh > > > > *From:* mpi3-tools-bounces(a)lists.mpi-forum.org > > [mailto:[email protected]] *On Behalf Of > > *Kathryn Mohror > > *Sent:* Tuesday, May 21, 2013 6:27 PM > > *To:* mpi3-tools(a)lists.mpi-forum.org > > *Subject:* Re: [Mpi3-tools] latest draft of mqs > > > > > > > > Hi John, > > > > > > > > Thank you for the extensive feedback and help with the document! I'm > > sure you will get the 'nod' you referred to on page 1 :-) > > > > > > > > For the definition of mqs_image, I agree that it needs work. Dong > > Ahn also mentioned that it needed to be more rigorously defined. > > However if possible, I think it would be beneficial if we could at > > least start the conversation before next week so that we have a > > better chance of having it nailed down in advance of the next meeting. > > > > > > > > So, that said -- does anyone want to take a first stab at a > > definition to get us started? > > > > > > > > Kathryn > > > > > > > > On May 21, 2013, at 2:18 PM, John DelSignore > > <John.DelSignore(a)roguewave.com > > <mailto:[email protected]>> > > wrote: > > > > > > > > Hi, > > > > Attached are my edits and comments on the MPI Message Queue > > document. Anh provided me with a Word document generated from the > > LaTex, so the formatting is funky in some places. I enabled change > > tracking in Word before making my edits, and I added many comments > > to the document in places where I think we have more work to do. > > Unfortunately, I think the changes might be somewhat extensive. > > > > The volume of edits and comments aside, I think Anh did a great job! > > The first draft of the document closely reflects the 10+ year-old > > description of the interface pretty well. Unfortunately, the > > original interface has some problem that we need to resolve, most > > critically, we have to resolve how to describe the concept of > > "mqs_image". We can talk about this during the next concall. > > > > Cheers, John D. > > > > > > Anh Vo wrote: > > > > Hi all, > > > > As mentioned on the tool WG call today, I'm re-sending the > > latest draft. > > Feedbacks are appreciated. We'll send this one out to the wider > > audience > > on Tuesday 5/21 to meet the requirement for first reading in > > June > > > > > > > > --Anh > > > > <MPI-MQD-JVD-2013-05-17.doc>_______________________________________________ > > Mpi3-tools mailing list > > Mpi3-tools(a)lists.mpi-forum.org <mailto:[email protected]> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > > > > > > > > ______________________________________________________________ > > Kathryn Mohror, kathryn(a)llnl.gov > > <mailto:[email protected]>, http://people.llnl.gov/mohror1 CASC @ > > Lawrence Livermore National Laboratory, Livermore, CA, USA > > > > > > > > > > > > > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools

1 0

Re: [Mpi3-tools] latest draft of mqs
by Anh Vo 22 May '13

22 May '13

The definitions John provide look great for me. Unless there are other opinions, I will use those. --Anh -----Original Message----- From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Ahn, Dong H. Sent: Wednesday, May 22, 2013 7:54 AM To: mpi3-tools(a)lists.mpi-forum.org Subject: Re: [Mpi3-tools] latest draft of mqs Hi John, This works for me and should clarify a few things in the current draft. Best, Dong > -----Original Message----- > From: mpi3-tools-bounces(a)lists.mpi-forum.org > [mailto:[email protected]] On Behalf Of John > DelSignore > Sent: Wednesday, May 22, 2013 9:55 AM > To: mpi3-tools(a)lists.mpi-forum.org > Subject: Re: [Mpi3-tools] latest draft of mqs > > I prefer to not muddy up the "standard definitions" (or maybe > "common-sense definitions") of commonly used term, such as "image > file", to make them conform to the MQD interface. Instead, I like to define the MQD typedefs in terms of the standard definitions. So, WRT to the document, I think we should include the following definitions: > > * An "image file" is an executable or shared library file, that may contain symbol definitions needed by the MQD interface. > > * An "MPI process" consists of an "address space" and a collection of execution contexts (threads or lightweight processes). > > * An "address space" is a region of memory that consists of executable > code and data, and is partially composed of a collection of image > files. The collection of image files may change at any point during the execution of the MPI process, and image files may be relocated at runtime within the address space at the point they are loaded into memory. > > Then we can say: > > * An "mqs_image" is an abstract concept that represents the collection > of image files loaded into the address space of the MPI process at any > given time, and is debugger implementation defined. In static > execution environments, where shared libraries are not supported, an > mqs_image can represent an executable image file. However, in dynamic > execution environments, where shared libraries, dynamically loaded shared libraries, and runtime relocation of shared libraries are supported, an mqs_image should represent the collection of image files loaded into the address space of the MPI process at any given point in time; in this situation, mqs_image may in fact represent the MPI process itself. > > Cheers, John D. > > > Anh Vo wrote: > > Thanks Kathryn for starting up the discussion. > > > > > > > > I'll start with my existing definition of mqs_image in the document, > > which definitely is not adequate. > > > > > > > > Existing definition: > > > > An image is an executable file that was loaded into memory when > > > > a process is started. For SIMD-style programs, all MPI processes > > have > > > > the same image. For MIMD-style programs, MPI processes might have > > > > different images. > > > > > > > > Dong's comments: > > > > The definition of "Image" isn't all that rigorous here. I think this > > creates some confusion somewhere down this document. Is an image > > really an executable file? Which typically means the base > > executable or does this refer to both the based executable and DSOs > > being loaded into the process address space? Though it sounds > > hair-splitting, even the SPMD remark, the statement is only true if > > the image is defined to be the base executable. > > > > > > > > John's comments: > > > > I think we should be more general with the term "image" to mean > > executable or shared library. The reason is that we do not want to > > require symbols to reside in the executable. We should allow symbols > > to reside in shared libraries. > > > > Also, I think we need to define the "address space" of a process, > > and an "image list" as a collection of image files that have been > > loaded in the address space, and possibly relocated at runtime. > > > > > > > > I think I agree with both comments. Image should not refer to a > > single executable image (a.out or a.exe), but rather the base > > executable and the shared libraries that were loaded into the process address space. > > > > > > > > So how about this new definition: > > > > An image represents the executable (e.g., a.out) and the collection > > of dynamically loaded libraries that were loaded into the process > > address space. > > > > > > > > --Anh > > > > *From:* mpi3-tools-bounces(a)lists.mpi-forum.org > > [mailto:[email protected]] *On Behalf Of > > *Kathryn Mohror > > *Sent:* Tuesday, May 21, 2013 6:27 PM > > *To:* mpi3-tools(a)lists.mpi-forum.org > > *Subject:* Re: [Mpi3-tools] latest draft of mqs > > > > > > > > Hi John, > > > > > > > > Thank you for the extensive feedback and help with the document! I'm > > sure you will get the 'nod' you referred to on page 1 :-) > > > > > > > > For the definition of mqs_image, I agree that it needs work. Dong > > Ahn also mentioned that it needed to be more rigorously defined. > > However if possible, I think it would be beneficial if we could at > > least start the conversation before next week so that we have a > > better chance of having it nailed down in advance of the next meeting. > > > > > > > > So, that said -- does anyone want to take a first stab at a > > definition to get us started? > > > > > > > > Kathryn > > > > > > > > On May 21, 2013, at 2:18 PM, John DelSignore > > <John.DelSignore(a)roguewave.com > > <mailto:[email protected]>> > > wrote: > > > > > > > > Hi, > > > > Attached are my edits and comments on the MPI Message Queue > > document. Anh provided me with a Word document generated from the > > LaTex, so the formatting is funky in some places. I enabled change > > tracking in Word before making my edits, and I added many comments > > to the document in places where I think we have more work to do. > > Unfortunately, I think the changes might be somewhat extensive. > > > > The volume of edits and comments aside, I think Anh did a great job! > > The first draft of the document closely reflects the 10+ year-old > > description of the interface pretty well. Unfortunately, the > > original interface has some problem that we need to resolve, most > > critically, we have to resolve how to describe the concept of > > "mqs_image". We can talk about this during the next concall. > > > > Cheers, John D. > > > > > > Anh Vo wrote: > > > > Hi all, > > > > As mentioned on the tool WG call today, I'm re-sending the > > latest draft. > > Feedbacks are appreciated. We'll send this one out to the wider > > audience > > on Tuesday 5/21 to meet the requirement for first reading in > > June > > > > > > > > --Anh > > > > <MPI-MQD-JVD-2013-05-17.doc>_______________________________________________ > > Mpi3-tools mailing list > > Mpi3-tools(a)lists.mpi-forum.org <mailto:[email protected]> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > > > > > > > > ______________________________________________________________ > > Kathryn Mohror, kathryn(a)llnl.gov > > <mailto:[email protected]>, http://people.llnl.gov/mohror1 CASC @ > > Lawrence Livermore National Laboratory, Livermore, CA, USA > > > > > > > > > > > > > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools _______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools

1 0

Re: [Mpi3-tools] latest draft of mqs
by Ahn, Dong H. 22 May '13

22 May '13

Hi John, This works for me and should clarify a few things in the current draft. Best, Dong > -----Original Message----- > From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of John DelSignore > Sent: Wednesday, May 22, 2013 9:55 AM > To: mpi3-tools(a)lists.mpi-forum.org > Subject: Re: [Mpi3-tools] latest draft of mqs > > I prefer to not muddy up the "standard definitions" (or maybe "common-sense definitions") of commonly used term, such as "image > file", to make them conform to the MQD interface. Instead, I like to define the MQD typedefs in terms of the standard definitions. So, > WRT to the document, I think we should include the following definitions: > > * An "image file" is an executable or shared library file, that may contain symbol definitions needed by the MQD interface. > > * An "MPI process" consists of an "address space" and a collection of execution contexts (threads or lightweight processes). > > * An "address space" is a region of memory that consists of executable code and data, and is partially composed of a collection of image > files. The collection of image files may change at any point during the execution of the MPI process, and image files may be relocated at > runtime within the address space at the point they are loaded into memory. > > Then we can say: > > * An "mqs_image" is an abstract concept that represents the collection of image files loaded into the address space of the MPI process > at any given time, and is debugger implementation defined. In static execution environments, where shared libraries are not supported, > an mqs_image can represent an executable image file. However, in dynamic execution environments, where shared libraries, > dynamically loaded shared libraries, and runtime relocation of shared libraries are supported, an mqs_image should represent the > collection of image files loaded into the address space of the MPI process at any given point in time; in this situation, mqs_image may in > fact represent the MPI process itself. > > Cheers, John D. > > > Anh Vo wrote: > > Thanks Kathryn for starting up the discussion. > > > > > > > > I'll start with my existing definition of mqs_image in the document, > > which definitely is not adequate. > > > > > > > > Existing definition: > > > > An image is an executable file that was loaded into memory when > > > > a process is started. For SIMD-style programs, all MPI processes have > > > > the same image. For MIMD-style programs, MPI processes might have > > > > different images. > > > > > > > > Dong's comments: > > > > The definition of "Image" isn't all that rigorous here. I think this > > creates some confusion somewhere down this document. Is an image > > really an executable file? Which typically means the base executable > > or does this refer to both the based executable and DSOs being loaded > > into the process address space? Though it sounds hair-splitting, even > > the SPMD remark, the statement is only true if the image is defined to > > be the base executable. > > > > > > > > John's comments: > > > > I think we should be more general with the term "image" to mean > > executable or shared library. The reason is that we do not want to > > require symbols to reside in the executable. We should allow symbols > > to reside in shared libraries. > > > > Also, I think we need to define the "address space" of a process, and > > an "image list" as a collection of image files that have been loaded > > in the address space, and possibly relocated at runtime. > > > > > > > > I think I agree with both comments. Image should not refer to a single > > executable image (a.out or a.exe), but rather the base executable and > > the shared libraries that were loaded into the process address space. > > > > > > > > So how about this new definition: > > > > An image represents the executable (e.g., a.out) and the collection of > > dynamically loaded libraries that were loaded into the process address > > space. > > > > > > > > --Anh > > > > *From:* mpi3-tools-bounces(a)lists.mpi-forum.org > > [mailto:[email protected]] *On Behalf Of *Kathryn > > Mohror > > *Sent:* Tuesday, May 21, 2013 6:27 PM > > *To:* mpi3-tools(a)lists.mpi-forum.org > > *Subject:* Re: [Mpi3-tools] latest draft of mqs > > > > > > > > Hi John, > > > > > > > > Thank you for the extensive feedback and help with the document! I'm > > sure you will get the 'nod' you referred to on page 1 :-) > > > > > > > > For the definition of mqs_image, I agree that it needs work. Dong Ahn > > also mentioned that it needed to be more rigorously defined. However > > if possible, I think it would be beneficial if we could at least start > > the conversation before next week so that we have a better chance of > > having it nailed down in advance of the next meeting. > > > > > > > > So, that said -- does anyone want to take a first stab at a definition > > to get us started? > > > > > > > > Kathryn > > > > > > > > On May 21, 2013, at 2:18 PM, John DelSignore > > <John.DelSignore(a)roguewave.com <mailto:[email protected]>> > > wrote: > > > > > > > > Hi, > > > > Attached are my edits and comments on the MPI Message Queue > > document. Anh provided me with a Word document generated from the > > LaTex, so the formatting is funky in some places. I enabled change > > tracking in Word before making my edits, and I added many comments > > to the document in places where I think we have more work to do. > > Unfortunately, I think the changes might be somewhat extensive. > > > > The volume of edits and comments aside, I think Anh did a great job! > > The first draft of the document closely reflects the 10+ year-old > > description of the interface pretty well. Unfortunately, the > > original interface has some problem that we need to resolve, most > > critically, we have to resolve how to describe the concept of > > "mqs_image". We can talk about this during the next concall. > > > > Cheers, John D. > > > > > > Anh Vo wrote: > > > > Hi all, > > > > As mentioned on the tool WG call today, I'm re-sending the > > latest draft. > > Feedbacks are appreciated. We'll send this one out to the wider > > audience > > on Tuesday 5/21 to meet the requirement for first reading in > > June > > > > > > > > --Anh > > > > <MPI-MQD-JVD-2013-05-17.doc>_______________________________________________ > > Mpi3-tools mailing list > > Mpi3-tools(a)lists.mpi-forum.org <mailto:[email protected]> > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > > > > > > > > ______________________________________________________________ > > Kathryn Mohror, kathryn(a)llnl.gov > > <mailto:[email protected]>, http://people.llnl.gov/mohror1 CASC @ > > Lawrence Livermore National Laboratory, Livermore, CA, USA > > > > > > > > > > > > > > > _______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools

1 0

Re: [Mpi3-tools] latest draft of mqs
by John DelSignore 22 May '13

22 May '13

I prefer to not muddy up the "standard definitions" (or maybe "common-sense definitions") of commonly used term, such as "image file", to make them conform to the MQD interface. Instead, I like to define the MQD typedefs in terms of the standard definitions. So, WRT to the document, I think we should include the following definitions: * An "image file" is an executable or shared library file, that may contain symbol definitions needed by the MQD interface. * An "MPI process" consists of an "address space" and a collection of execution contexts (threads or lightweight processes). * An "address space" is a region of memory that consists of executable code and data, and is partially composed of a collection of image files. The collection of image files may change at any point during the execution of the MPI process, and image files may be relocated at runtime within the address space at the point they are loaded into memory. Then we can say: * An "mqs_image" is an abstract concept that represents the collection of image files loaded into the address space of the MPI process at any given time, and is debugger implementation defined. In static execution environments, where shared libraries are not supported, an mqs_image can represent an executable image file. However, in dynamic execution environments, where shared libraries, dynamically loaded shared libraries, and runtime relocation of shared libraries are supported, an mqs_image should represent the collection of image files loaded into the address space of the MPI process at any given point in time; in this situation, mqs_image may in fact represent the MPI process itself. Cheers, John D. Anh Vo wrote: > Thanks Kathryn for starting up the discussion. > > > > Iï¿½ll start with my existing definition of mqs_image in the document, > which definitely is not adequate. > > > > Existing definition: > > An image is an executable file that was loaded into memory when > > a process is started. For SIMD-style programs, all MPI processes have > > the same image. For MIMD-style programs, MPI processes might have > > different images. > > > > Dongï¿½s comments: > > The definition of ï¿½Imageï¿½ isnï¿½t all that rigorous here. I think this > creates some confusion somewhere down this document. Is an image really > an executable file? Which typically means the base executable or does > this refer to both the based executable and DSOs being loaded into the > process address space? Though it sounds hair-splitting, even the SPMD > remark, the statement is only true if the image is defined to be the > base executable. > > > > Johnï¿½s comments: > > I think we should be more general with the term ï¿½imageï¿½ to mean > executable or shared library. The reason is that we do not want to > require symbols to reside in the executable. We should allow symbols to > reside in shared libraries. > > Also, I think we need to define the ï¿½address spaceï¿½ of a process, and an > ï¿½image listï¿½ as a collection of image files that have been loaded in the > address space, and possibly relocated at runtime. > > > > I think I agree with both comments. Image should not refer to a single > executable image (a.out or a.exe), but rather the base executable and > the shared libraries that were loaded into the process address space. > > > > So how about this new definition: > > An image represents the executable (e.g., a.out) and the collection of > dynamically loaded libraries that were loaded into the process address > space. > > > > --Anh > > *From:* mpi3-tools-bounces(a)lists.mpi-forum.org > [mailto:[email protected]] *On Behalf Of *Kathryn > Mohror > *Sent:* Tuesday, May 21, 2013 6:27 PM > *To:* mpi3-tools(a)lists.mpi-forum.org > *Subject:* Re: [Mpi3-tools] latest draft of mqs > > > > Hi John, > > > > Thank you for the extensive feedback and help with the document! I'm > sure you will get the 'nod' you referred to on page 1 :-) > > > > For the definition of mqs_image, I agree that it needs work. Dong Ahn > also mentioned that it needed to be more rigorously defined. However if > possible, I think it would be beneficial if we could at least start the > conversation before next week so that we have a better chance of having > it nailed down in advance of the next meeting. > > > > So, that said -- does anyone want to take a first stab at a definition > to get us started? > > > > Kathryn > > > > On May 21, 2013, at 2:18 PM, John DelSignore > <John.DelSignore(a)roguewave.com <mailto:[email protected]>> > wrote: > > > > Hi, > > Attached are my edits and comments on the MPI Message Queue > document. Anh provided me with a Word document generated from the > LaTex, so the formatting is funky in some places. I enabled change > tracking in Word before making my edits, and I added many comments > to the document in places where I think we have more work to do. > Unfortunately, I think the changes might be somewhat extensive. > > The volume of edits and comments aside, I think Anh did a great job! > The first draft of the document closely reflects the 10+ year-old > description of the interface pretty well. Unfortunately, the > original interface has some problem that we need to resolve, most > critically, we have to resolve how to describe the concept of > "mqs_image". We can talk about this during the next concall. > > Cheers, John D. > > > Anh Vo wrote: > > Hi all, > > As mentioned on the tool WG call today, Iï¿½m re-sending the > latest draft. > Feedbacks are appreciated. Weï¿½ll send this one out to the wider > audience > on Tuesday 5/21 to meet the requirement for first reading in June > > > > --Anh > > <MPI-MQD-JVD-2013-05-17.doc>_______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org <mailto:[email protected]> > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools > > > > ______________________________________________________________ > Kathryn Mohror, kathryn(a)llnl.gov > <mailto:[email protected]>, http://people.llnl.gov/mohror1 > CASC @ Lawrence Livermore National Laboratory, Livermore, CA, USA > > > > > > >

1 0

Re: [Mpi3-tools] latest draft of mqs
by Anh Vo 21 May '13

21 May '13

Thanks Kathryn for starting up the discussion. I'll start with my existing definition of mqs_image in the document, which definitely is not adequate. Existing definition: An image is an executable file that was loaded into memory when a process is started. For SIMD-style programs, all MPI processes have the same image. For MIMD-style programs, MPI processes might have different images. Dong's comments: The definition of "Image" isn't all that rigorous here. I think this creates some confusion somewhere down this document. Is an image really an executable file? Which typically means the base executable or does this refer to both the based executable and DSOs being loaded into the process address space? Though it sounds hair-splitting, even the SPMD remark, the statement is only true if the image is defined to be the base executable. John's comments: I think we should be more general with the term "image" to mean executable or shared library. The reason is that we do not want to require symbols to reside in the executable. We should allow symbols to reside in shared libraries. Also, I think we need to define the "address space" of a process, and an "image list" as a collection of image files that have been loaded in the address space, and possibly relocated at runtime. I think I agree with both comments. Image should not refer to a single executable image (a.out or a.exe), but rather the base executable and the shared libraries that were loaded into the process address space. So how about this new definition: An image represents the executable (e.g., a.out) and the collection of dynamically loaded libraries that were loaded into the process address space. --Anh From: mpi3-tools-bounces(a)lists.mpi-forum.org [mailto:[email protected]] On Behalf Of Kathryn Mohror Sent: Tuesday, May 21, 2013 6:27 PM To: mpi3-tools(a)lists.mpi-forum.org Subject: Re: [Mpi3-tools] latest draft of mqs Hi John, Thank you for the extensive feedback and help with the document! I'm sure you will get the 'nod' you referred to on page 1 :-) For the definition of mqs_image, I agree that it needs work. Dong Ahn also mentioned that it needed to be more rigorously defined. However if possible, I think it would be beneficial if we could at least start the conversation before next week so that we have a better chance of having it nailed down in advance of the next meeting. So, that said -- does anyone want to take a first stab at a definition to get us started? Kathryn On May 21, 2013, at 2:18 PM, John DelSignore <John.DelSignore(a)roguewave.com<mailto:[email protected]>> wrote: Hi, Attached are my edits and comments on the MPI Message Queue document. Anh provided me with a Word document generated from the LaTex, so the formatting is funky in some places. I enabled change tracking in Word before making my edits, and I added many comments to the document in places where I think we have more work to do. Unfortunately, I think the changes might be somewhat extensive. The volume of edits and comments aside, I think Anh did a great job! The first draft of the document closely reflects the 10+ year-old description of the interface pretty well. Unfortunately, the original interface has some problem that we need to resolve, most critically, we have to resolve how to describe the concept of "mqs_image". We can talk about this during the next concall. Cheers, John D. Anh Vo wrote: Hi all, As mentioned on the tool WG call today, I'm re-sending the latest draft. Feedbacks are appreciated. We'll send this one out to the wider audience on Tuesday 5/21 to meet the requirement for first reading in June --Anh <MPI-MQD-JVD-2013-05-17.doc>_______________________________________________ Mpi3-tools mailing list Mpi3-tools(a)lists.mpi-forum.org<mailto:[email protected]> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools ______________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov<mailto:[email protected]>, http://people.llnl.gov/mohror1 CASC @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0

Re: [Mpi3-tools] latest draft of mqs
by Kathryn Mohror 21 May '13

21 May '13

Hi John, Thank you for the extensive feedback and help with the document! I'm sure you will get the 'nod' you referred to on page 1 :-) For the definition of mqs_image, I agree that it needs work. Dong Ahn also mentioned that it needed to be more rigorously defined. However if possible, I think it would be beneficial if we could at least start the conversation before next week so that we have a better chance of having it nailed down in advance of the next meeting. So, that said -- does anyone want to take a first stab at a definition to get us started? Kathryn On May 21, 2013, at 2:18 PM, John DelSignore <John.DelSignore(a)roguewave.com> wrote: > Hi, > > Attached are my edits and comments on the MPI Message Queue document. Anh provided me with a Word document generated from the LaTex, so the formatting is funky in some places. I enabled change tracking in Word before making my edits, and I added many comments to the document in places where I think we have more work to do. Unfortunately, I think the changes might be somewhat extensive. > > The volume of edits and comments aside, I think Anh did a great job! The first draft of the document closely reflects the 10+ year-old description of the interface pretty well. Unfortunately, the original interface has some problem that we need to resolve, most critically, we have to resolve how to describe the concept of "mqs_image". We can talk about this during the next concall. > > Cheers, John D. > > > Anh Vo wrote: >> Hi all, >> >> As mentioned on the tool WG call today, I’m re-sending the latest draft. >> Feedbacks are appreciated. We’ll send this one out to the wider audience >> on Tuesday 5/21 to meet the requirement for first reading in June >> >> >> >> --Anh >> > <MPI-MQD-JVD-2013-05-17.doc>_______________________________________________ > Mpi3-tools mailing list > Mpi3-tools(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-tools ______________________________________________________________ Kathryn Mohror, kathryn(a)llnl.gov, http://people.llnl.gov/mohror1 CASC @ Lawrence Livermore National Laboratory, Livermore, CA, USA

1 0