On Apr 2, 2013, at 4:34 PM CDT, Larry Baker <baker(a)usgs.gov> wrote:
> Jeff referred me to the MPI 3.0 spec (http://mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf) for data sizes. But, even it is contradictory. For example, in section 3.2.2, it says an MPI_INTEGER is a Fortran (default KIND) INTEGER. Fortran does not specify the size of a (default KIND) INTEGER. However, Table 13.2 says it is 4 bytes. It could just as easily be 8 bytes in a 64-bit execution environment. In that case, Fortran requires that the size of all the default KIND numeric types in Table 13.2 would double in size. But, not necessarily pointers; pointers in Fortran do not have a specified size (storage unit).
Table 13.2 lists sizes for a specific kind of encoding of MPI datatypes known as "external32". You should not use it as a reference for the actual size of the MPI datatype, which must instead always match the actual language type in the corresponding language binding.
> The MPI 3.0 spec says if you send an MPI_INTEGER, you have to receive an MPI_INTEGER. I didn't exhaustively search the MPI 3.0 spec, but you might also have to make sure the default KIND INTEGER is the same for the sender and receiver. This might be beyond the scope of MPI, and I do not know how exhaustive the run-time tests can be to detect violations of the rules. (A message would have to carry type metadata for the payload data in addition to the payload data itself.) Section 17 had lots of information about Fortran and C data sizes, including automatically conversions. I didn't read enough to know if the spec covers the case of exchanging default KIND INTEGERs between two executables with different sizes. Maybe it is true that MPI mandates something like lexical scoping rules, such that, for example, all members of an MPI application must assign the same semantics to their fundamental data types -- the default KIND INTEGER, etc., in the case of Fortran.
Most users should ignore the rules regarding conversions on heterogeneous systems. They don't make a lot of sense even if you're an MPI expert, and:
1) almost no MPI implementations properly support this behavior
2) almost no user codes are written correctly to deal with it, even if #1 were true
-Dave
Tatiana,
Dave or Jeff are probably better capable to address your specific application. I meant only to raise the issue that, typically, in a 64-bit execution environment, an INTEGER is 32 bits while an address is 64 bits. Thus, a C void * is a 64 bit "unsigned long" while a Fortran INTEGER would likely be a 32 bit C "int". Your change from a void * type to an INTEGER type would likely only work for a 32 bit application -- sort of. Since Fortran does not have unsigned numeric variables, whatever INTEGER KIND gets used to store an address must be used strictly as a container for an opaque value. I saw mention of an MPI_AINT, which looks to me like the Fortran MPI name for a C void *.
Jeff referred me to the MPI 3.0 spec (http://mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf) for data sizes. But, even it is contradictory. For example, in section 3.2.2, it says an MPI_INTEGER is a Fortran (default KIND) INTEGER. Fortran does not specify the size of a (default KIND) INTEGER. However, Table 13.2 says it is 4 bytes. It could just as easily be 8 bytes in a 64-bit execution environment. In that case, Fortran requires that the size of all the default KIND numeric types in Table 13.2 would double in size. But, not necessarily pointers; pointers in Fortran do not have a specified size (storage unit).
The MPI 3.0 spec says if you send an MPI_INTEGER, you have to receive an MPI_INTEGER. I didn't exhaustively search the MPI 3.0 spec, but you might also have to make sure the default KIND INTEGER is the same for the sender and receiver. This might be beyond the scope of MPI, and I do not know how exhaustive the run-time tests can be to detect violations of the rules. (A message would have to carry type metadata for the payload data in addition to the payload data itself.) Section 17 had lots of information about Fortran and C data sizes, including automatically conversions. I didn't read enough to know if the spec covers the case of exchanging default KIND INTEGERs between two executables with different sizes. Maybe it is true that MPI mandates something like lexical scoping rules, such that, for example, all members of an MPI application must assign the same semantics to their fundamental data types -- the default KIND INTEGER, etc., in the case of Fortran.
Jeff warned that using 64 bit INTEGERs with Fortran is very dangerous. I agree. I also think it is dangerous to confuse pointer data types with integer data types.
I apologize if I am off base.
Larry Baker
US Geological Survey
650-329-5608
baker(a)usgs.gov
On 1 Apr 2013, at 7:01 PM, Tatiana V. Martsinkevich wrote:
> hello,
>
> Dave, thank you for the brief explanation and links. I still didn't managed to make fortran programs recognize MPIX_Pattern type so as a temporary solution we just pass an integer type variable.
>
> Larry, do you mean that we might have problems when passing integer pointer to a function that takes void* as an argument? We've changed the interfaces too so we operate with integers and integer pointers everywhere.
>
> best regards,
> Tatiana
>
>
> 2013/3/29 Dave Goodell <goodell(a)mcs.anl.gov>
> The "@FOO@" snippets in a file named "BAR.in" are replaced by "configure" (by "config.status", technically) when it is run. The replacement string is taken from a shell environment variable in the configure script by the same name. Valid substitution values are designated by "AC_SUBST" macro invocations in "configure.ac".
>
> There are a lot of moving parts here which would take a lot of time to explain in detail. You will probably help yourself out by learning a bit about the autotools:
>
> http://www.gnu.org/software/autoconf/manual/autoconf.html
> http://www.gnu.org/software/automake/manual/automake.html
> http://www.flameeyes.eu/autotools-mythbuster/
>
> Grep will also help you answer many of these questions.
>
> -Dave
>
> On Mar 29, 2013, at 3:11 PM CDT, Tatiana V. Martsinkevich <tanya.manekineko(a)gmail.com> wrote:
>
> > Hello,
> >
> > Thanks again for your help Dave.
> >
> > So we went the other way around and changed the type of MPI_Pattern from void* to integer and now fortran interfaces are being generated correctly. However, now the question is how to register the new type MPI_Pattern so that we could use it in fortran programs. This also doesn't seem to be trivial...
> >
> > In buildface there are this kind of code sections:
> >
> > # Datatypes
> > # These are determined and set at configure time
> > foreach $key (COMPLEX, DOUBLE_COMPLEX, LOGICAL, REAL, DOUBLE_PRECISION, INTEGER, '2INTEGER', '2DOUBLE_PRECISION', '2REAL', CHARACTER) {
> > print MPIFFD " INTEGER MPI_$key\n";
> > print MPIFFD " PARAMETER (MPI_$key=\@MPI_$key\@)\n";
> > }
> >
> > this looks like where I should start digging but apparently, simply adding PATTERN to the key set doesn't work:
> >
> > Included at src/binding/f90/mpi_constants.f90:6:
> >
> > PARAMETER (MPI_PATTERN=@MPI_PATTERN@)
> > 1
> > Error: Expected expression at (1) in PARAMETER statement
> >
> > Also, could someone please explain what does this notion - @MPI_COMPLEX@ - in mpi.h.in mean
> >
> > /* Fortran types */
> > #define MPI_COMPLEX ((MPI_Datatype)@MPI_COMPLEX@)
> >
> >
> > regards,
> > Tatiana
> >
> >
> > 2013/3/28 Dave Goodell <goodell(a)mcs.anl.gov>
> > You almost certainly will also need some modifications in "src/binding/f77/buildiface". Whenever I need to modify this script I have to stare at it for a while before I'm sure where the modifications need to go.
> >
> > I don't think we have any existing types which map to pointers under the hood. So there isn't a good example for you to use here. The "MPI_Fint *" changes in particular do not seem correct to me in general. They *might* work in a pinch on a platform with where INTEGER and (void*) are the same size. They might also work on little endian platforms like x86/x86_64, but it will be purely by luck if they do.
> >
> > You may be better writing your own Fortran bindings by hand in the short term.
> >
> > Bill may have something to add on this subject, since he's the primary maintainer for these scripts.
> >
> > -Dave
> >
> > On Mar 28, 2013, at 3:04 PM CDT, Tatiana V. Martsinkevich <tanya.manekineko(a)gmail.com> wrote:
> >
> > > Thanks for reply.
> > >
> > > Yes, unfortunately we need fortran interfaces because most of the
> > > applications that we use for testing are written in fortran.
> > > Will it be enough if I just add a conversion rule to %parmc2f in
> > > ./src/binding/f90/buildface? Something like:
> > >
> > > 'MPI_Pattern' => 'INTEGER(KIND=MPI_ADDRESS_KIND)',
> > > 'MPI_Pattern*' => 'INTEGER(KIND=MPI_ADDRESS_KIND)',
> > >
> > > And for f77 it will be:
> > >
> > > 'MPI_Pattern' => 'MPI_Fint *'
> > > 'MPI_Pattern*' => 'MPI_Fint *'
> > >
> > >
> > >> Also, as a side note, please use "MPIX_" for nonstandard MPI extensions. It makes nonstandard interfaces very obvious to users and will help prevent potential issues with standardization in the future.
> > >
> > > Duly noted.
> > >
> > > Tatiana
> > >
> > > 2013/3/28 Dave Goodell <goodell(a)mcs.anl.gov>:
> > >> On Mar 28, 2013, at 2:16 PM CDT, Tatiana V. Martsinkevich <tanya.manekineko(a)gmail.com> wrote:
> > >>
> > >>> We are implementing some fault tolerance related functionality in
> > >>> MPICH-3.0.2. We define a new MPI type in
> > >>> ~/mpich-3.0.2/src/include/mpi.h.in as:
> > >>>
> > >>> typedef void* MPI_Pattern;
> > >>>
> > >>> then in the same file we define:
> > >>>
> > >>> int MPI_Pattern_declare(char* name, MPI_Pattern* handler);
> > >>>
> > >>> However, when I run autogen.sh and fortran binding script executes it
> > >>> gives out:
> > >>>
> > >>> Pattern_declare: No parm type for MPI_Pattern* (MPI_Pattern*)
> > >>> Use of uninitialized value $fparm in pattern match (m//) at
> > >>> ./buildiface line 1289, <FD> line 1710.
> > >>
> > >> Do you want Fortran interfaces at this time or would you be just as happy to have C-only bindings right now? The former will require hacking on one or more of the "buildiface" scripts in the "src/binding/LANGUAGE" directories. The latter can be accomplished by moving your prototypes and types near the end of mpi.h.in, near the prototypes for "MPIX_Mutex_unlock" and "PMPIX_Mutex_unlock".
> > >>
> > >> Also, as a side note, please use "MPIX_" for nonstandard MPI extensions. It makes nonstandard interfaces very obvious to users and will help prevent potential issues with standardization in the future.
> > >>
> > >> -Dave
> > >>
> >
> >
>
>
hello,
Dave, thank you for the brief explanation and links. I still didn't managed
to make fortran programs recognize MPIX_Pattern type so as a temporary
solution we just pass an integer type variable.
Larry, do you mean that we might have problems when passing integer pointer
to a function that takes void* as an argument? We've changed the interfaces
too so we operate with integers and integer pointers everywhere.
best regards,
Tatiana
2013/3/29 Dave Goodell <goodell(a)mcs.anl.gov>
> The "@FOO@" snippets in a file named "BAR.in" are replaced by "configure"
> (by "config.status", technically) when it is run. The replacement string
> is taken from a shell environment variable in the configure script by the
> same name. Valid substitution values are designated by "AC_SUBST" macro
> invocations in "configure.ac".
>
> There are a lot of moving parts here which would take a lot of time to
> explain in detail. You will probably help yourself out by learning a bit
> about the autotools:
>
> http://www.gnu.org/software/autoconf/manual/autoconf.html
> http://www.gnu.org/software/automake/manual/automake.html
> http://www.flameeyes.eu/autotools-mythbuster/
>
> Grep will also help you answer many of these questions.
>
> -Dave
>
> On Mar 29, 2013, at 3:11 PM CDT, Tatiana V. Martsinkevich <
> tanya.manekineko(a)gmail.com> wrote:
>
> > Hello,
> >
> > Thanks again for your help Dave.
> >
> > So we went the other way around and changed the type of MPI_Pattern from
> void* to integer and now fortran interfaces are being generated correctly.
> However, now the question is how to register the new type MPI_Pattern so
> that we could use it in fortran programs. This also doesn't seem to be
> trivial...
> >
> > In buildface there are this kind of code sections:
> >
> > # Datatypes
> > # These are determined and set at configure time
> > foreach $key (COMPLEX, DOUBLE_COMPLEX, LOGICAL, REAL,
> DOUBLE_PRECISION, INTEGER, '2INTEGER', '2DOUBLE_PRECISION', '2REAL',
> CHARACTER) {
> > print MPIFFD " INTEGER MPI_$key\n";
> > print MPIFFD " PARAMETER (MPI_$key=\@MPI_$key\@)\n";
> > }
> >
> > this looks like where I should start digging but apparently, simply
> adding PATTERN to the key set doesn't work:
> >
> > Included at src/binding/f90/mpi_constants.f90:6:
> >
> > PARAMETER (MPI_PATTERN=@MPI_PATTERN@)
> > 1
> > Error: Expected expression at (1) in PARAMETER statement
> >
> > Also, could someone please explain what does this notion - @MPI_COMPLEX@- in
> mpi.h.in mean
> >
> > /* Fortran types */
> > #define MPI_COMPLEX ((MPI_Datatype)@MPI_COMPLEX@)
> >
> >
> > regards,
> > Tatiana
> >
> >
> > 2013/3/28 Dave Goodell <goodell(a)mcs.anl.gov>
> > You almost certainly will also need some modifications in
> "src/binding/f77/buildiface". Whenever I need to modify this script I have
> to stare at it for a while before I'm sure where the modifications need to
> go.
> >
> > I don't think we have any existing types which map to pointers under the
> hood. So there isn't a good example for you to use here. The "MPI_Fint *"
> changes in particular do not seem correct to me in general. They *might*
> work in a pinch on a platform with where INTEGER and (void*) are the same
> size. They might also work on little endian platforms like x86/x86_64, but
> it will be purely by luck if they do.
> >
> > You may be better writing your own Fortran bindings by hand in the short
> term.
> >
> > Bill may have something to add on this subject, since he's the primary
> maintainer for these scripts.
> >
> > -Dave
> >
> > On Mar 28, 2013, at 3:04 PM CDT, Tatiana V. Martsinkevich <
> tanya.manekineko(a)gmail.com> wrote:
> >
> > > Thanks for reply.
> > >
> > > Yes, unfortunately we need fortran interfaces because most of the
> > > applications that we use for testing are written in fortran.
> > > Will it be enough if I just add a conversion rule to %parmc2f in
> > > ./src/binding/f90/buildface? Something like:
> > >
> > > 'MPI_Pattern' => 'INTEGER(KIND=MPI_ADDRESS_KIND)',
> > > 'MPI_Pattern*' => 'INTEGER(KIND=MPI_ADDRESS_KIND)',
> > >
> > > And for f77 it will be:
> > >
> > > 'MPI_Pattern' => 'MPI_Fint *'
> > > 'MPI_Pattern*' => 'MPI_Fint *'
> > >
> > >
> > >> Also, as a side note, please use "MPIX_" for nonstandard MPI
> extensions. It makes nonstandard interfaces very obvious to users and will
> help prevent potential issues with standardization in the future.
> > >
> > > Duly noted.
> > >
> > > Tatiana
> > >
> > > 2013/3/28 Dave Goodell <goodell(a)mcs.anl.gov>:
> > >> On Mar 28, 2013, at 2:16 PM CDT, Tatiana V. Martsinkevich <
> tanya.manekineko(a)gmail.com> wrote:
> > >>
> > >>> We are implementing some fault tolerance related functionality in
> > >>> MPICH-3.0.2. We define a new MPI type in
> > >>> ~/mpich-3.0.2/src/include/mpi.h.in as:
> > >>>
> > >>> typedef void* MPI_Pattern;
> > >>>
> > >>> then in the same file we define:
> > >>>
> > >>> int MPI_Pattern_declare(char* name, MPI_Pattern* handler);
> > >>>
> > >>> However, when I run autogen.sh and fortran binding script executes it
> > >>> gives out:
> > >>>
> > >>> Pattern_declare: No parm type for MPI_Pattern* (MPI_Pattern*)
> > >>> Use of uninitialized value $fparm in pattern match (m//) at
> > >>> ./buildiface line 1289, <FD> line 1710.
> > >>
> > >> Do you want Fortran interfaces at this time or would you be just as
> happy to have C-only bindings right now? The former will require hacking
> on one or more of the "buildiface" scripts in the "src/binding/LANGUAGE"
> directories. The latter can be accomplished by moving your prototypes and
> types near the end of mpi.h.in, near the prototypes for
> "MPIX_Mutex_unlock" and "PMPIX_Mutex_unlock".
> > >>
> > >> Also, as a side note, please use "MPIX_" for nonstandard MPI
> extensions. It makes nonstandard interfaces very obvious to users and will
> help prevent potential issues with standardization in the future.
> > >>
> > >> -Dave
> > >>
> >
> >
>
>
