- Mpi3-abi - mailman3.cels.anl.gov

Re: [Mpi3-abi] ABI: for languages?
by unknown＠example.com 12 Sep '08

12 Sep '08

Jeff, > But if there are oodles of PMPI-based tools that want an ABI, I would > encourage them to publicly speak out -- just like anyone who both > wants and will actually use an ABI should publicly speak out. Yes, there are several PMPI-based tools that want an ABI and the compatibility you describe. We, in the TAU project [http://tau.uoregon.edu] will appreciate this very much. Thanks! - Sameer On Fri, 12 Sep 2008, Jeff Squyres wrote: > On Sep 12, 2008, at 5:00 PM, Erez Haba wrote: > > >>>> is there some other perceived gain beyond MPI-based ISV apps > >>>> being able to conveniently ship a single binary? > > > > Yes there is. > > We discussed one example during the MPI conference. There is a great > > benefit for tools, especially for tools that are using the PMPI > > interface. > > Many implementers ship their mpi implementation as a dynamically > > loaded binary; vendors like Microsoft, Intel HP and others. As such > > it is more difficult for tools to plugin between the application and > > the mpi implementation; it is not impossible but still more difficult. > > I guess it depends on the type of tool, no? The parallel debuggers > have gotten along fine without an ABI because they don't hook in at > the MPI API layer. Alternate mechanisms exist for them to find out > the exact value of MPI_COMM_WORLD (for example). Expansions of this > concept are just being introduced into MPI-3.0 for other kinds of > tools, too. > > But if there are oodles of PMPI-based tools that want an ABI, I would > encourage them to publicly speak out -- just like anyone who both > wants and will actually use an ABI should publicly speak out. > > > Having an API allows MPI implementers, ISV's, tools developers and > > end user to verify the application with a tool like ISP (University > > of Utha), without the special need from any of the other parties. > > I think you mean ABI, right? > > -- > Jeff Squyres > Cisco Systems > > _______________________________________________ > mpi3-abi mailing list > mpi3-abi_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-abi >

1 0

Re: [Mpi3-abi] ABI: for languages?
by unknown＠example.com 12 Sep '08

12 Sep '08

On Fri, Sep 12, 2008 at 05:53:24PM -0400, Jeff Squyres wrote: > Can we build a list of these to see how many packages actually *will* do > what you assert? Sure, you're welcome to ask. But predicting the future is hard, and even package maintainers probably aren't very good at it, i.e. someone who's never considered a binary distribution due to MPI issues may well not leap at the possibility today, but might in a year or two. Instead, you might look at the rapid growth of non-MPI scientific programs being distributed as binaries, and project from there. -- greg

1 0

Re: [Mpi3-abi] ABI: for languages?
by unknown＠example.com 12 Sep '08

12 Sep '08

On Sep 12, 2008, at 5:00 PM, Erez Haba wrote: >>>> is there some other perceived gain beyond MPI-based ISV apps >>>> being able to conveniently ship a single binary? > > Yes there is. > We discussed one example during the MPI conference. There is a great > benefit for tools, especially for tools that are using the PMPI > interface. > Many implementers ship their mpi implementation as a dynamically > loaded binary; vendors like Microsoft, Intel HP and others. As such > it is more difficult for tools to plugin between the application and > the mpi implementation; it is not impossible but still more difficult. I guess it depends on the type of tool, no? The parallel debuggers have gotten along fine without an ABI because they don't hook in at the MPI API layer. Alternate mechanisms exist for them to find out the exact value of MPI_COMM_WORLD (for example). Expansions of this concept are just being introduced into MPI-3.0 for other kinds of tools, too. But if there are oodles of PMPI-based tools that want an ABI, I would encourage them to publicly speak out -- just like anyone who both wants and will actually use an ABI should publicly speak out. > Having an API allows MPI implementers, ISV's, tools developers and > end user to verify the application with a tool like ISP (University > of Utha), without the special need from any of the other parties. I think you mean ABI, right? -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi3-abi] ABI: for languages?
by unknown＠example.com 12 Sep '08

12 Sep '08

Can we build a list of these to see how many packages actually *will* do what you assert? (e.g., as opposed to software maintainers being entranced by the idea of possibly doing it) I would strongly prefer to hear actual software maintainers publicly saying "yes, I will do this" instead of having someone else speak for them. Not to be too untrusting, but there was a stellar example at the Dublin ABI meeting where person A said "trust me; I can speak for group XYZ", but later, group XYZ said exactly the opposite of what person A had previously asserted for them. On Sep 12, 2008, at 5:37 PM, Greg Lindahl wrote: > On Fri, Sep 12, 2008 at 01:17:45PM -0400, Jeff Squyres wrote: > >> If there are not -- if the majority are in the "we QA with >> MPI's X, Y, and Z, and thou shallt not run with any other", then an >> ABI >> is not worthwhile. > > We knew before we started that many ISVs had this opionion. But if > they ever change their minds in the future, the ABI is useful to > them. And when they go to test, the ABI saves them a recompile. > > The early adopters of the ABI will not be ISVs. It will be > non-commercial people who want to ship pre-compiled application > binaries, and who don't do much testing of individual MPIs or > platforms anyway. There are many free non-MPI scientific programs > that distribute precompiled binaries. MPI app authors would like to do > the same. > > -- greg > > _______________________________________________ > mpi3-abi mailing list > mpi3-abi_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-abi -- Jeff Squyres Cisco Systems

1 0

Re: [Mpi3-abi] ABI: for languages?
by unknown＠example.com 12 Sep '08

12 Sep '08

On Fri, Sep 12, 2008 at 01:17:45PM -0400, Jeff Squyres wrote: > If there are not -- if the majority are in the "we QA with > MPI's X, Y, and Z, and thou shallt not run with any other", then an ABI > is not worthwhile. We knew before we started that many ISVs had this opionion. But if they ever change their minds in the future, the ABI is useful to them. And when they go to test, the ABI saves them a recompile. The early adopters of the ABI will not be ISVs. It will be non-commercial people who want to ship pre-compiled application binaries, and who don't do much testing of individual MPIs or platforms anyway. There are many free non-MPI scientific programs that distribute precompiled binaries. MPI app authors would like to do the same. -- greg

1 0

Re: [Mpi3-abi] ABI: for languages?
by unknown＠example.com 12 Sep '08

12 Sep '08

>>>is there some other perceived gain beyond MPI-based ISV apps being able to conveniently ship a single binary? Yes there is. We discussed one example during the MPI conference. There is a great benefit for tools, especially for tools that are using the PMPI interface. Many implementers ship their mpi implementation as a dynamically loaded binary; vendors like Microsoft, Intel HP and others. As such it is more difficult for tools to plugin between the application and the mpi implementation; it is not impossible but still more difficult. Having an API allows MPI implementers, ISV's, tools developers and end user to verify the application with a tool like ISP (University of Utha), without the special need from any of the other parties. Thanks, .Erez -----Original Message----- From: mpi3-abi-bounces_at_[hidden] [mailto:mpi3-abi-bounces_at_[hidden]] On Behalf Of Jeff Squyres Sent: Friday, September 12, 2008 10:18 AM To: MPI 3.0 ABI working group Subject: [Mpi3-abi] ABI: for languages? Edric from the Mathworks was kind enough to give me some of his time yesterday; we had a nice chat about the MPI ABI and why the Mathworks wants it. I think I understand his position *much* better now. It was somewhat of an epiphany for me. Let me explain... (Edric: please feel free to correct anything that I say below!) In short, Matlab is a very different kind of product than some of other MPI-based ISV applications out there (reminder: I'm coming from a background of a [limited] survey of ISVs who told me that they *don't* want an MPI ABI -- they only want to run with the MPI's that they specifically choose and QA). With other MPI-based ISV apps, they have a certified application that is QA checked, etc. They have a turnkey solution -- setup your input file, click "go" in the GUI and you get the computed answer. If you have a cluster behind the GUI, you magically get the answer faster. The user doesn't really know/care that MPI is being used behind the scenes. MPI is not a primary focus for the ISV developers -- indeed, they only see it as [yet another] tool to get the job done faster. They do want to use high speed network interconnects, so the multi- protocol MPI implementations do quite well for them. But the point here is that the user doesn't have religion about what MPI is used -- they just want the answer "faster". So the ISV's that I spoke to just QA a single solution stack and then sell that. Matlab, has two key differences from this model: 1. Matlab has an overall philosophy of being able to swap in and out different parts of Matlab to effect the same functionality. For example, you can swap out various computational cores with different implementations to see if they perform better on a particular customer's platform. In the same philosophy, they want to be able to swap out the MPI implementation. Matlab ships with MPICH2 (which they support), but they allow users to swap out to use other (MPICH2-ABI- compatible) MPIs, with the understanding that they are now running in an unsupported (but not discouraged) manner. In short, Matlab ships with lots of plugin modules for all kinds of functionality, but they actively allow and encourage users to swap out those plugins for alternate implementations. MPI, to them, falls under this same philosophy. --> It is worth mentioning that Matlab already dlopen's the MPI .so library and dlsym's to find all the MPI function symbols. The values that they use for constants and sizes for datatypes are simply what are prescribed by MPICH2. They also have templated scripts for setting up the MPI run-time environment and invoking it. So if you don't use MPICH2, the user/administrator can edit these Matlab scripts to setup daemons or whatever is required by that MPI implementation. 2. Matlab has a scripting language that (at the moment) has a few simple API calls that call back to the C MPI API. As such, you can view the Matlab as a high-level scripting language interpreter with MPI bindings. User-level scripts can invoke these API calls and call the back-end MPI implementation. In this way, the Mathworks doesn't care which MPI is on the back-end -- they only want to provide a conduit to whatever MPI implementation is on the back end. In some sense, the run-time characteristics of that MPI implementation are not really their problem -- the user-level Matlab script needs to do whatever it needs to do to be portable between MPI implementations (if it wishes -- similar to other portable MPI libraries/applications, like SCALAPACK, etc.). --- Matlab has some pre-built MPI-based algorithms, but at least for me, point #2 above is the more important one: you can look at Matlab as a [commercial] higher-level scripting language that provides access to the underlying C MPI API. This is a fundamentally different QA issue than other turnkey ISV MPI-based applications. Indeed, the Mathworks largely doesn't care about many of the criticisms of the MPI ABI because they simply aren't relevant to Matlab's desired end goal. So where does this leave us? I now understand the Mathworks' position. But the question is -- how many others are in this position? If there are a lot, then an ABI is a good idea. If there are not -- if the majority are in the "we QA with MPI's X, Y, and Z, and thou shallt not run with any other", then an ABI is not worthwhile. So forgive me, I may be asking a question which has already been answered (my memory is horrid): do we have a concrete list of ISVs, languages, or other software packages that are definitively (and publicly) stating "Yes, I want and will use an MPI ABI in my software"? Can we build this list on the wiki? Perhaps it would also be interesting to build a list of those who do not want an MPI ABI (not counting MPI implementors ;-) ), just for comparison's sake...? ** And I mean a "yes" answer that is stronger than the typical knee- jerk "yes, vendor, give me everything I ask for and more!" reaction. I'm looking for software maintainers who have thought through the issues and understand the tradeoffs and limitations of an MPI ABI. E.g., the Boost C++ MPI bindings guys don't care too much about an MPI ABI because they have C++ compiler ABI issues of their own. In short -- I want to do a cost/benefit analysis. Is it worth our time (and money) to do all this work? Will customers pay for it? Or is there some other perceived gain beyond MPI-based ISV apps being able to conveniently ship a single binary? (that's an honest question, not intended to be provocative) If you're still reading this, thanks for your time. :-) -- Jeff Squyres Cisco Systems _______________________________________________ mpi3-abi mailing list mpi3-abi_at_[hidden] http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-abi

1 0

ABI: for languages?
by unknown＠example.com 12 Sep '08

12 Sep '08

Edric from the Mathworks was kind enough to give me some of his time yesterday; we had a nice chat about the MPI ABI and why the Mathworks wants it. I think I understand his position *much* better now. It was somewhat of an epiphany for me. Let me explain... (Edric: please feel free to correct anything that I say below!) In short, Matlab is a very different kind of product than some of other MPI-based ISV applications out there (reminder: I'm coming from a background of a [limited] survey of ISVs who told me that they *don't* want an MPI ABI -- they only want to run with the MPI's that they specifically choose and QA). With other MPI-based ISV apps, they have a certified application that is QA checked, etc. They have a turnkey solution -- setup your input file, click "go" in the GUI and you get the computed answer. If you have a cluster behind the GUI, you magically get the answer faster. The user doesn't really know/care that MPI is being used behind the scenes. MPI is not a primary focus for the ISV developers -- indeed, they only see it as [yet another] tool to get the job done faster. They do want to use high speed network interconnects, so the multi- protocol MPI implementations do quite well for them. But the point here is that the user doesn't have religion about what MPI is used -- they just want the answer "faster". So the ISV's that I spoke to just QA a single solution stack and then sell that. Matlab, has two key differences from this model: 1. Matlab has an overall philosophy of being able to swap in and out different parts of Matlab to effect the same functionality. For example, you can swap out various computational cores with different implementations to see if they perform better on a particular customer's platform. In the same philosophy, they want to be able to swap out the MPI implementation. Matlab ships with MPICH2 (which they support), but they allow users to swap out to use other (MPICH2-ABI- compatible) MPIs, with the understanding that they are now running in an unsupported (but not discouraged) manner. In short, Matlab ships with lots of plugin modules for all kinds of functionality, but they actively allow and encourage users to swap out those plugins for alternate implementations. MPI, to them, falls under this same philosophy. --> It is worth mentioning that Matlab already dlopen's the MPI .so library and dlsym's to find all the MPI function symbols. The values that they use for constants and sizes for datatypes are simply what are prescribed by MPICH2. They also have templated scripts for setting up the MPI run-time environment and invoking it. So if you don't use MPICH2, the user/administrator can edit these Matlab scripts to setup daemons or whatever is required by that MPI implementation. 2. Matlab has a scripting language that (at the moment) has a few simple API calls that call back to the C MPI API. As such, you can view the Matlab as a high-level scripting language interpreter with MPI bindings. User-level scripts can invoke these API calls and call the back-end MPI implementation. In this way, the Mathworks doesn't care which MPI is on the back-end -- they only want to provide a conduit to whatever MPI implementation is on the back end. In some sense, the run-time characteristics of that MPI implementation are not really their problem -- the user-level Matlab script needs to do whatever it needs to do to be portable between MPI implementations (if it wishes -- similar to other portable MPI libraries/applications, like SCALAPACK, etc.). --- Matlab has some pre-built MPI-based algorithms, but at least for me, point #2 above is the more important one: you can look at Matlab as a [commercial] higher-level scripting language that provides access to the underlying C MPI API. This is a fundamentally different QA issue than other turnkey ISV MPI-based applications. Indeed, the Mathworks largely doesn't care about many of the criticisms of the MPI ABI because they simply aren't relevant to Matlab's desired end goal. So where does this leave us? I now understand the Mathworks' position. But the question is -- how many others are in this position? If there are a lot, then an ABI is a good idea. If there are not -- if the majority are in the "we QA with MPI's X, Y, and Z, and thou shallt not run with any other", then an ABI is not worthwhile. So forgive me, I may be asking a question which has already been answered (my memory is horrid): do we have a concrete list of ISVs, languages, or other software packages that are definitively (and publicly) stating "Yes, I want and will use an MPI ABI in my software"? Can we build this list on the wiki? Perhaps it would also be interesting to build a list of those who do not want an MPI ABI (not counting MPI implementors ;-) ), just for comparison's sake...? ** And I mean a "yes" answer that is stronger than the typical knee- jerk "yes, vendor, give me everything I ask for and more!" reaction. I'm looking for software maintainers who have thought through the issues and understand the tradeoffs and limitations of an MPI ABI. E.g., the Boost C++ MPI bindings guys don't care too much about an MPI ABI because they have C++ compiler ABI issues of their own. In short -- I want to do a cost/benefit analysis. Is it worth our time (and money) to do all this work? Will customers pay for it? Or is there some other perceived gain beyond MPI-based ISV apps being able to conveniently ship a single binary? (that's an honest question, not intended to be provocative) If you're still reading this, thanks for your time. :-) -- Jeff Squyres Cisco Systems

1 0

Open MPI and ABI
by unknown＠example.com 12 Sep '08

12 Sep '08

I told the group in Dublin that I would check with the Open MPI community to see who wanted to work on the Linux side of the 6- function ABI proof of concept with our colleagues at Intel MPI. Unfortunately, it seems like no one is interested or has the resources to commit. :-( I got feedback through the LANL Open MPI rep that LANL won't be committing resources to this effort, either, but perhaps Jeff Brown can double check this (I know there are many more groups at LANL than just the team with the Open MPI developer). -- Jeff Squyres Cisco Systems

1 0

ABI WG
by unknown＠example.com 28 Aug '08

28 Aug '08

folks, I'd like to get some momentum going on the ABI front. Attached is the latest spreadsheet detailing the mpi.h differences between major implementations of MPI, and the beginnings of a common ABI column. If you can, please take a look and be prepared to provide your input at the working group meeting next week in Dublin. We have proposed values for the "compile time value" items (open for discussion). I think it's time to tackle the "init time constants" where the implementations differ by a fair amount. ee you in Ireland! Jeff *

1 0

Re: [Mpi3-abi] July forum meeting
by unknown＠example.com 08 Jul '08

08 Jul '08

References: <B07C99647880FF4E989E9004EC3A6C0303C42D34_at_[hidden]> <CCC67F1566B8C244A0EC577C841A7AB10319A5CD22_at_[hidden]> <6.2.3.4.2.20080416095806.02d139f8_at_[hidden]> <6.2.3.4.2.20080421133743.02d31f30_at_[hidden]> <4810AEEC.5090502_at_[hidden]> <6.2.3.4.2.20080424101301.02f784d8_at_[hidden]> <6B68D01C00C9994A8E150183E62A119E72BD94BF0B_at_NA-EXMSG-C105.redmond.corp.micros> <6.2.3.4.2.20080424114556.02fa6e18_at_[hidden]> <6B68D01C00C9994A8E150183E62A119E72BD94C031_at_NA-EXMSG-C105.redmond.corp.micros> <6.2.3.4.2.20080424124023.02fba368_at_[hidden]> <20080424184647.GB23338_at_[hidden]> <6B68D01C00C9994A8E150183E62A119E72BD94C0AE_at_NA-EXMSG-C105.redmond.corp.micros> <5ECAB1304A8B5B4CB3F9D6C01E4E21A201467216_at_[hidden]> <3993C75C-10F0-438B-A3FA-3AA41FEF64F0_at_[hidden]> <5ECAB1304A8B5B4CB3F9D6C01E4E21A20146736F_at_[hidden]> <25CDC614-ADBB-4C43-A3D7-DB5D02D94ACE_at_[hidden]> <6B68D01C00C9994A8E150183E62A119E72B EF8E01E_at_[hidden]> <20080708173312.DE1631F8002_at_[hidden]> X-Mailer: Apple Mail (2.926) Return-Path: jsquyres_at_[hidden] X-OriginalArrivalTime: 08 Jul 2008 18:21:58.0493 (UTC) FILETIME=[823AE0D0:01C8E127] The ABI subcommittee meeting got edged out (time-wise) for other discussions that the Forum felt needed more interactive time. There are only 2 people interested in discussing the ABI in Menlo Park (Erez and myself), so it didn't seem worthwhile to have a subcommittee meeting. Sorry. :-( On Jul 8, 2008, at 1:33 PM, Jeff Brown wrote: > all, > > What transpired at the meeting ABI-wise? > > Do we have an updated spreadsheet? > > thanks > > Jeff > > > _______________________________________________ > mpi3-abi mailing list > mpi3-abi_at_[hidden] > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpi3-abi -- Jeff Squyres Cisco Systems

1 0