- mpiwg-fortran - mailman3.cels.anl.gov

Re: [MPIWG Fortran] Fortran coarrays - failed images
by Bland, Wesley 07 Sep '15

07 Sep '15

Yep. That's the same challenge were facing. Pointing people at that link is a reasonable place to start. That's my old group at UTK where the first ULFM implementation was done and they're still maintaining it. I think they have some examples and use cases there as well. Right now, I think the MPICH version of PMI is probably the best place to start for what we want from launchers. I think PMIx is doing similar things, but I'm less familiar with it. > On Sep 7, 2015, at 10:27 PM, Bill Long <longb(a)cray.com> wrote: > > Hi Wesley, > > Thanks for the detailed reply. It looks like the OpenCoarrays implementors should not have any problem getting the “did someone fail?” and “who failed?” information from MPI. It’s not clear how that will get handled for their GASnet alternative, thought I suspect the UPC clan will be interested in some form of FT on the really large systems. Maybe PMI would be a place to start for them. > > The remaining boogieman is convincing the execution environment to not kill the job. I’ll try to find out what our MPI group is considering as a first step. The http://fault-tolerance.org/ link in the main ticket (OpenMPI implementation) also looks potentially useful. If the various MPI implementations as well as the Fortran implementors ask the ALPS/PBS/SLURM/… people for the same thing we’re more likely to get it. > > Cheers, > Bill > > >> On Sep 7, 2015, at 11:58 AM, Bland, Wesley <wesley.bland(a)intel.com> wrote: >> >> Hi Bill, >> >> It sounds like at least some of what coarrays requires would be provided by the ULFM proposal the FTWG is bringing forward. I’ll drop some comments inline. >> >> >> >> >>> On 9/7/15, 12:45 AM, "mpiwg-fortran on behalf of Bill Long" <mpiwg-fortran-bounces(a)lists.mpi-forum.org on behalf of longb(a)cray.com> wrote: >>> >>> Hi Jeff, >>> >>> The current version of the TS is WG5 document N2074. The failed image feature has not changed much recently, but still better to use the latest version. N2074 is currently out for WG5 review. Assuming it passes, this (minus the line numbers that we like but ISO doesn’t) is the version that will be sent to the ISO editors for publication. >>> >>> I think it would be an excellent idea for the TS/Fortran 2015 and MPI facilities for FT to be able to use common underlying infrastructure if possible. I have not thought about how a program that uses both the Fortran and the MPI facilities would work (or how the MPI spec should be written in that regard), but if the links into components like PMI or SLURM would be the same, that would certainly help. I wrote up a summary of the TS features for the benefit of the MPI FT experts, pasted in below. >>> >>> Cheers, >>> Bill >>> >>> ISO TS 18508 includes features that can be used to help a program >>> react to the failure of an image. It is intended to be a minimal >>> capability. Facilities are included for notification, inquiry, >>> testing, and simple continuation of execution. >>> >>> >>> Background: >>> >>> The parallel programming model using coarrays that is included in >>> Fortran 2008 assumes that the number of executing images remains >>> constant for the entire program. As a consequence, failure of an image >>> (typically from a hardware failure) aborted the whole program. The >>> addition of teams in TS 18508 allows for the possibility of the number >>> of images decreasing following a failure by forming a new team >>> consisting of the active images and continuing execution in that team. >>> While this was not the main motivation for introducing teams, this >>> observation lead to the addition of minimal resilience facilities to >>> the TS. >>> >>> >>> Notification: >>> >>> The image control statements include an optional STAT= specifier will >>> return an error status. The image selector syntax for remote >>> references also allow an optional STAT= specifier. The new collective >>> and atomic subroutines have an optional STAT argument that will also >>> return an error status. An error status of zero indicates success. If >>> the operation involved communication with a failed image, the status >>> returned is equal to the named constant STAT_FAILED_IMAGE that is >>> defined in the intrinsic module ISO_FORTRAN_ENV, and execution >>> continues. If there is no status variable provided and the operation >>> involves communication with a failed image, the program aborts. A >>> negative value of STAT_FAILED_IMAGE indicates that the processor >>> cannot detect processor failure, and a positive value indicates it can. >>> This effectively makes provision of this facility optional. >>> >>> [MPI analog: Remote communication operations in MPI either return a >>> status as a function result (C), or have an optional MPI subroutine >>> argument that returns an error status (Fortran). A named constant >>> MPI_FAILED_RANK could be added to the MPI spec to provide a >>> corresponding capability.] >> >> >> WB: We do propose adding a new error class called MPI_ERR_PROC_FAILED which would tell you that a rank has failed. There are separate functions available to find out exactly which process. >> >>> >>> >>> Inquiry: >>> >>> Two inquiry functions are provided that can return information on the >>> status of images. IMAGE_STATUS( N ) will return STAT_FAILED_IMAGE if >>> image N has failed. This could be used to check on the health of an >>> image just before a loop that involved many accesses to that >>> image. FAILED_IMAGES( ) returns a 1-D array containing the numbers of >>> the failed images. This can be used to compute which images should be >>> omitted from a new team that can be used for continued execution. If >>> there are no failed images, the returned array has size zero. >>> >>> [MPI analog: Adding corresponding functions ( IMAGE -> RANK ) to MPI >>> would seem straightforward.] >> >> >> WB: While there isn’t exactly a function in the proposal to find out whether a communicator or window has a failed rank in it, it’s relatively straightforward to get that functionality with a different function that we propose. MPI_COMM_AGREE will let you know if a rank on the communicator has failed and then you can use the query functions on the communicator to get the group of failed processes. >> >>> >>> >>> Testing: >>> >>> A new statement >>> >>> FAIL IMAGE >>> >>> is added. When executed by image N it causes image N to appear to have >>> failed as seen from the other images. This is included so that >>> programmers can test recovery algorithms without having to wait for an >>> actual failure. An image that executes this statement does not >>> continue execution. >>> >>> [MPI analog: One more function.] >> >> >> WB: You can emulate failures in MPI by just having a process exit without calling finalize or MPI_Abort. That’s erroneous behavior and will start kicking off error handlers in a HQI. >> >>> >>> >>> Continuation: >>> >>> The FORM TEAM statement allows the program to create a new team. In >>> the case of a failed image, the strategy would be to form a new team >>> consisting of the remaining active images. The CHANGE TEAM statement >>> causes a switch in the execution environment to the specified >>> team. Note that whether the program can meaningfully continue depends >>> on the algorithm being implemented and whether the program includes >>> code to switch teams to continue. >>> >>> [MPI analog: Fortran teams could be mapped to MPI communicators. One >>> difference is that Fortran allows you to omit a team specification in >>> most statements involving communication, in which case the "current" >>> team is used. The MPI spec might want to specify that MPI_COMM_WORLD >>> is redefined in the case that "team" is shrunk through rank failure, >>> to reduce the need to modify existing code.] >> >> WB: We decided against changing the makeup of MPI_COMM_WORLD mid-execution to follow the law of least astonishment. What we do provide is a function to shrink an existing communicator to create a new one without the failed processes. Then it’s just a matter of changing your handles to use the new communicator instead of MPI_COMM_WORLD. A well-behaved app should be duping MPI_COMM_WORLD first anyway, right? :) >> >>> >>> >>> Implementation issue: >>> >>> The program launching and management environment (PMI, ALPS, SLURM, >>> ...) need to be modified to include an API that can provide failed >>> image information to the program and keep the remaining images >>> executing, as opposed to the current behavior of aborting all the >>> images. The API also needs a function that the program startup code >>> can call to inform the management environment whether it is employing >>> resilient features. It would certainly be advantageous to have the >>> same API used for both Fortran and MPI. >> >> >> WB: Agreed. We have a similar requirement from the launcher to inform MPI about failed processes. Technically, we don’t require it of the launcher as MPI prefers to remain launcher/runtime neutral, but it would be good if implementations could take advantage of the same systems. I know that for PMI in MPICH, we added a new key that stays updated with a list of failed processes. We don’t yet have an implementation on top of ALPS or SLURM so there isn’t a state of the art there as far as I know. In most instances, everything just aborts so that’s a behavior that we’d like to get changed too. >> >> As for informing the runtime about whether to enable resilience or not, we’ve put in the proposal language to say that the implementation is free to not provide the resilience features as long as it provides the API functions as no-ops. It would just never return MPI_ERR_PROC_FAILED. I expect that implementations that can’t/don’t want to support FT would do this and if people don’t want to give up the little overhead, they could configure it out at compile time (or provide two libraries). >> >>> >>> ——————————End of Summary —————————————— >> >> WB: Thanks for the summary. If you’d like to see a little more detail that Jeff is discussing, it’s in our ticket: #325: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/325 (main ticket #323: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/323) It sounds to me like the main points that are needed for the new standard are there. It’d be good to have another set of eyes on the RMA part in particular though to make sure we’re providing all the needed semantics. I wouldn’t say any of us in the FTWG are RMA or Fortran experts. >> >> Thanks, >> Wesley >> >>> >>> >>> >>>> On Sep 5, 2015, at 7:09 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >>>> >>>> So Fortran 2015 (TS 18508 - section 6 in the attachment) is going to support failed images. >>>> >>>> The OpenCoarrays folks (who are responsible for enabling GCC 5+ Fortran coarray support) have started looking into how to support this feature. They currently use MPI-3 RMA and GASNet as communication runtimes, but the need to support FT will likely push them in a new direction. The options they have mentioned thus far are undesirable. >>>> >>>> It would be great if there were more people who could help look at RMA FT, particularly as it pertains to Fortran 2015. Are any of the Fortran WG folks savvy on how those failures map to MPI concepts and whether or not the MPI RMA FT discussion is going in the right direction? >>>> >>>> Thanks, >>>> >>>> Jeff >>>> >>>> -- >>>> Jeff Hammond >>>> jeff.science(a)gmail.com >>>> http://jeffhammond.github.io/ >>>> <ISO-IECJTC1-SC22-WG5_N2056_Draft_TS_18508_Additional_Paralle.pdf>_______________________________________________ >>>> mpiwg-fortran mailing list >>>> mpiwg-fortran(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >>> >>> Bill Long longb(a)cray.com >>> Fortran Technical Support & voice: 651-605-9024 >>> Bioinformatics Software Development fax: 651-605-9142 >>> Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101 >>> >>> >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > Bill Long longb(a)cray.com > Fortran Technical Support & voice: 651-605-9024 > Bioinformatics Software Development fax: 651-605-9142 > Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101 > > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran

1 0

Re: [MPIWG Fortran] Fortran coarrays - failed images
by Bill Long 07 Sep '15

07 Sep '15

Hi Wesley, Thanks for the detailed reply. It looks like the OpenCoarrays implementors should not have any problem getting the “did someone fail?” and “who failed?” information from MPI. It’s not clear how that will get handled for their GASnet alternative, thought I suspect the UPC clan will be interested in some form of FT on the really large systems. Maybe PMI would be a place to start for them. The remaining boogieman is convincing the execution environment to not kill the job. I’ll try to find out what our MPI group is considering as a first step. The http://fault-tolerance.org/ link in the main ticket (OpenMPI implementation) also looks potentially useful. If the various MPI implementations as well as the Fortran implementors ask the ALPS/PBS/SLURM/… people for the same thing we’re more likely to get it. Cheers, Bill On Sep 7, 2015, at 11:58 AM, Bland, Wesley <wesley.bland(a)intel.com> wrote: > Hi Bill, > > It sounds like at least some of what coarrays requires would be provided by the ULFM proposal the FTWG is bringing forward. I’ll drop some comments inline. > > > > > On 9/7/15, 12:45 AM, "mpiwg-fortran on behalf of Bill Long" <mpiwg-fortran-bounces(a)lists.mpi-forum.org on behalf of longb(a)cray.com> wrote: > >> Hi Jeff, >> >> The current version of the TS is WG5 document N2074. The failed image feature has not changed much recently, but still better to use the latest version. N2074 is currently out for WG5 review. Assuming it passes, this (minus the line numbers that we like but ISO doesn’t) is the version that will be sent to the ISO editors for publication. >> >> I think it would be an excellent idea for the TS/Fortran 2015 and MPI facilities for FT to be able to use common underlying infrastructure if possible. I have not thought about how a program that uses both the Fortran and the MPI facilities would work (or how the MPI spec should be written in that regard), but if the links into components like PMI or SLURM would be the same, that would certainly help. I wrote up a summary of the TS features for the benefit of the MPI FT experts, pasted in below. >> >> Cheers, >> Bill >> >> ISO TS 18508 includes features that can be used to help a program >> react to the failure of an image. It is intended to be a minimal >> capability. Facilities are included for notification, inquiry, >> testing, and simple continuation of execution. >> >> >> Background: >> >> The parallel programming model using coarrays that is included in >> Fortran 2008 assumes that the number of executing images remains >> constant for the entire program. As a consequence, failure of an image >> (typically from a hardware failure) aborted the whole program. The >> addition of teams in TS 18508 allows for the possibility of the number >> of images decreasing following a failure by forming a new team >> consisting of the active images and continuing execution in that team. >> While this was not the main motivation for introducing teams, this >> observation lead to the addition of minimal resilience facilities to >> the TS. >> >> >> Notification: >> >> The image control statements include an optional STAT= specifier will >> return an error status. The image selector syntax for remote >> references also allow an optional STAT= specifier. The new collective >> and atomic subroutines have an optional STAT argument that will also >> return an error status. An error status of zero indicates success. If >> the operation involved communication with a failed image, the status >> returned is equal to the named constant STAT_FAILED_IMAGE that is >> defined in the intrinsic module ISO_FORTRAN_ENV, and execution >> continues. If there is no status variable provided and the operation >> involves communication with a failed image, the program aborts. A >> negative value of STAT_FAILED_IMAGE indicates that the processor >> cannot detect processor failure, and a positive value indicates it can. >> This effectively makes provision of this facility optional. >> >> [MPI analog: Remote communication operations in MPI either return a >> status as a function result (C), or have an optional MPI subroutine >> argument that returns an error status (Fortran). A named constant >> MPI_FAILED_RANK could be added to the MPI spec to provide a >> corresponding capability.] > > > WB: We do propose adding a new error class called MPI_ERR_PROC_FAILED which would tell you that a rank has failed. There are separate functions available to find out exactly which process. > >> >> >> Inquiry: >> >> Two inquiry functions are provided that can return information on the >> status of images. IMAGE_STATUS( N ) will return STAT_FAILED_IMAGE if >> image N has failed. This could be used to check on the health of an >> image just before a loop that involved many accesses to that >> image. FAILED_IMAGES( ) returns a 1-D array containing the numbers of >> the failed images. This can be used to compute which images should be >> omitted from a new team that can be used for continued execution. If >> there are no failed images, the returned array has size zero. >> >> [MPI analog: Adding corresponding functions ( IMAGE -> RANK ) to MPI >> would seem straightforward.] > > > WB: While there isn’t exactly a function in the proposal to find out whether a communicator or window has a failed rank in it, it’s relatively straightforward to get that functionality with a different function that we propose. MPI_COMM_AGREE will let you know if a rank on the communicator has failed and then you can use the query functions on the communicator to get the group of failed processes. > >> >> >> Testing: >> >> A new statement >> >> FAIL IMAGE >> >> is added. When executed by image N it causes image N to appear to have >> failed as seen from the other images. This is included so that >> programmers can test recovery algorithms without having to wait for an >> actual failure. An image that executes this statement does not >> continue execution. >> >> [MPI analog: One more function.] > > > WB: You can emulate failures in MPI by just having a process exit without calling finalize or MPI_Abort. That’s erroneous behavior and will start kicking off error handlers in a HQI. > >> >> >> Continuation: >> >> The FORM TEAM statement allows the program to create a new team. In >> the case of a failed image, the strategy would be to form a new team >> consisting of the remaining active images. The CHANGE TEAM statement >> causes a switch in the execution environment to the specified >> team. Note that whether the program can meaningfully continue depends >> on the algorithm being implemented and whether the program includes >> code to switch teams to continue. >> >> [MPI analog: Fortran teams could be mapped to MPI communicators. One >> difference is that Fortran allows you to omit a team specification in >> most statements involving communication, in which case the "current" >> team is used. The MPI spec might want to specify that MPI_COMM_WORLD >> is redefined in the case that "team" is shrunk through rank failure, >> to reduce the need to modify existing code.] > > WB: We decided against changing the makeup of MPI_COMM_WORLD mid-execution to follow the law of least astonishment. What we do provide is a function to shrink an existing communicator to create a new one without the failed processes. Then it’s just a matter of changing your handles to use the new communicator instead of MPI_COMM_WORLD. A well-behaved app should be duping MPI_COMM_WORLD first anyway, right? :) > >> >> >> Implementation issue: >> >> The program launching and management environment (PMI, ALPS, SLURM, >> ...) need to be modified to include an API that can provide failed >> image information to the program and keep the remaining images >> executing, as opposed to the current behavior of aborting all the >> images. The API also needs a function that the program startup code >> can call to inform the management environment whether it is employing >> resilient features. It would certainly be advantageous to have the >> same API used for both Fortran and MPI. > > > WB: Agreed. We have a similar requirement from the launcher to inform MPI about failed processes. Technically, we don’t require it of the launcher as MPI prefers to remain launcher/runtime neutral, but it would be good if implementations could take advantage of the same systems. I know that for PMI in MPICH, we added a new key that stays updated with a list of failed processes. We don’t yet have an implementation on top of ALPS or SLURM so there isn’t a state of the art there as far as I know. In most instances, everything just aborts so that’s a behavior that we’d like to get changed too. > > As for informing the runtime about whether to enable resilience or not, we’ve put in the proposal language to say that the implementation is free to not provide the resilience features as long as it provides the API functions as no-ops. It would just never return MPI_ERR_PROC_FAILED. I expect that implementations that can’t/don’t want to support FT would do this and if people don’t want to give up the little overhead, they could configure it out at compile time (or provide two libraries). > >> >> ——————————End of Summary —————————————— >> > > WB: Thanks for the summary. If you’d like to see a little more detail that Jeff is discussing, it’s in our ticket: #325: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/325 (main ticket #323: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/323) It sounds to me like the main points that are needed for the new standard are there. It’d be good to have another set of eyes on the RMA part in particular though to make sure we’re providing all the needed semantics. I wouldn’t say any of us in the FTWG are RMA or Fortran experts. > > Thanks, > Wesley > >> >> >> >> On Sep 5, 2015, at 7:09 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: >> >>> So Fortran 2015 (TS 18508 - section 6 in the attachment) is going to support failed images. >>> >>> The OpenCoarrays folks (who are responsible for enabling GCC 5+ Fortran coarray support) have started looking into how to support this feature. They currently use MPI-3 RMA and GASNet as communication runtimes, but the need to support FT will likely push them in a new direction. The options they have mentioned thus far are undesirable. >>> >>> It would be great if there were more people who could help look at RMA FT, particularly as it pertains to Fortran 2015. Are any of the Fortran WG folks savvy on how those failures map to MPI concepts and whether or not the MPI RMA FT discussion is going in the right direction? >>> >>> Thanks, >>> >>> Jeff >>> >>> -- >>> Jeff Hammond >>> jeff.science(a)gmail.com >>> http://jeffhammond.github.io/ >>> <ISO-IECJTC1-SC22-WG5_N2056_Draft_TS_18508_Additional_Paralle.pdf>_______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >> >> Bill Long longb(a)cray.com >> Fortran Technical Support & voice: 651-605-9024 >> Bioinformatics Software Development fax: 651-605-9142 >> Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101 >> >> >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101

1 0

Re: [MPIWG Fortran] Fortran coarrays - failed images
by Bland, Wesley 07 Sep '15

07 Sep '15

Hi Bill, It sounds like at least some of what coarrays requires would be provided by the ULFM proposal the FTWG is bringing forward. I’ll drop some comments inline. On 9/7/15, 12:45 AM, "mpiwg-fortran on behalf of Bill Long" <mpiwg-fortran-bounces(a)lists.mpi-forum.org on behalf of longb(a)cray.com> wrote: >Hi Jeff, > >The current version of the TS is WG5 document N2074. The failed image feature has not changed much recently, but still better to use the latest version. N2074 is currently out for WG5 review. Assuming it passes, this (minus the line numbers that we like but ISO doesn’t) is the version that will be sent to the ISO editors for publication. > >I think it would be an excellent idea for the TS/Fortran 2015 and MPI facilities for FT to be able to use common underlying infrastructure if possible. I have not thought about how a program that uses both the Fortran and the MPI facilities would work (or how the MPI spec should be written in that regard), but if the links into components like PMI or SLURM would be the same, that would certainly help. I wrote up a summary of the TS features for the benefit of the MPI FT experts, pasted in below. > >Cheers, >Bill > >ISO TS 18508 includes features that can be used to help a program >react to the failure of an image. It is intended to be a minimal >capability. Facilities are included for notification, inquiry, >testing, and simple continuation of execution. > > >Background: > >The parallel programming model using coarrays that is included in >Fortran 2008 assumes that the number of executing images remains >constant for the entire program. As a consequence, failure of an image >(typically from a hardware failure) aborted the whole program. The >addition of teams in TS 18508 allows for the possibility of the number >of images decreasing following a failure by forming a new team >consisting of the active images and continuing execution in that team. >While this was not the main motivation for introducing teams, this >observation lead to the addition of minimal resilience facilities to >the TS. > > >Notification: > >The image control statements include an optional STAT= specifier will >return an error status. The image selector syntax for remote >references also allow an optional STAT= specifier. The new collective >and atomic subroutines have an optional STAT argument that will also >return an error status. An error status of zero indicates success. If >the operation involved communication with a failed image, the status >returned is equal to the named constant STAT_FAILED_IMAGE that is >defined in the intrinsic module ISO_FORTRAN_ENV, and execution >continues. If there is no status variable provided and the operation >involves communication with a failed image, the program aborts. A >negative value of STAT_FAILED_IMAGE indicates that the processor >cannot detect processor failure, and a positive value indicates it can. >This effectively makes provision of this facility optional. > >[MPI analog: Remote communication operations in MPI either return a >status as a function result (C), or have an optional MPI subroutine >argument that returns an error status (Fortran). A named constant >MPI_FAILED_RANK could be added to the MPI spec to provide a >corresponding capability.] WB: We do propose adding a new error class called MPI_ERR_PROC_FAILED which would tell you that a rank has failed. There are separate functions available to find out exactly which process. > > >Inquiry: > >Two inquiry functions are provided that can return information on the >status of images. IMAGE_STATUS( N ) will return STAT_FAILED_IMAGE if >image N has failed. This could be used to check on the health of an >image just before a loop that involved many accesses to that >image. FAILED_IMAGES( ) returns a 1-D array containing the numbers of >the failed images. This can be used to compute which images should be >omitted from a new team that can be used for continued execution. If >there are no failed images, the returned array has size zero. > >[MPI analog: Adding corresponding functions ( IMAGE -> RANK ) to MPI >would seem straightforward.] WB: While there isn’t exactly a function in the proposal to find out whether a communicator or window has a failed rank in it, it’s relatively straightforward to get that functionality with a different function that we propose. MPI_COMM_AGREE will let you know if a rank on the communicator has failed and then you can use the query functions on the communicator to get the group of failed processes. > > >Testing: > >A new statement > > FAIL IMAGE > >is added. When executed by image N it causes image N to appear to have >failed as seen from the other images. This is included so that >programmers can test recovery algorithms without having to wait for an >actual failure. An image that executes this statement does not >continue execution. > >[MPI analog: One more function.] WB: You can emulate failures in MPI by just having a process exit without calling finalize or MPI_Abort. That’s erroneous behavior and will start kicking off error handlers in a HQI. > > >Continuation: > >The FORM TEAM statement allows the program to create a new team. In >the case of a failed image, the strategy would be to form a new team >consisting of the remaining active images. The CHANGE TEAM statement >causes a switch in the execution environment to the specified >team. Note that whether the program can meaningfully continue depends >on the algorithm being implemented and whether the program includes >code to switch teams to continue. > >[MPI analog: Fortran teams could be mapped to MPI communicators. One >difference is that Fortran allows you to omit a team specification in >most statements involving communication, in which case the "current" >team is used. The MPI spec might want to specify that MPI_COMM_WORLD >is redefined in the case that "team" is shrunk through rank failure, >to reduce the need to modify existing code.] WB: We decided against changing the makeup of MPI_COMM_WORLD mid-execution to follow the law of least astonishment. What we do provide is a function to shrink an existing communicator to create a new one without the failed processes. Then it’s just a matter of changing your handles to use the new communicator instead of MPI_COMM_WORLD. A well-behaved app should be duping MPI_COMM_WORLD first anyway, right? :) > > >Implementation issue: > >The program launching and management environment (PMI, ALPS, SLURM, >...) need to be modified to include an API that can provide failed >image information to the program and keep the remaining images >executing, as opposed to the current behavior of aborting all the >images. The API also needs a function that the program startup code >can call to inform the management environment whether it is employing >resilient features. It would certainly be advantageous to have the >same API used for both Fortran and MPI. WB: Agreed. We have a similar requirement from the launcher to inform MPI about failed processes. Technically, we don’t require it of the launcher as MPI prefers to remain launcher/runtime neutral, but it would be good if implementations could take advantage of the same systems. I know that for PMI in MPICH, we added a new key that stays updated with a list of failed processes. We don’t yet have an implementation on top of ALPS or SLURM so there isn’t a state of the art there as far as I know. In most instances, everything just aborts so that’s a behavior that we’d like to get changed too. As for informing the runtime about whether to enable resilience or not, we’ve put in the proposal language to say that the implementation is free to not provide the resilience features as long as it provides the API functions as no-ops. It would just never return MPI_ERR_PROC_FAILED. I expect that implementations that can’t/don’t want to support FT would do this and if people don’t want to give up the little overhead, they could configure it out at compile time (or provide two libraries). > >——————————End of Summary —————————————— > WB: Thanks for the summary. If you’d like to see a little more detail that Jeff is discussing, it’s in our ticket: #325: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/325 (main ticket #323: https://svn.mpi-forum.org/trac/mpi-forum-web/ticket/323) It sounds to me like the main points that are needed for the new standard are there. It’d be good to have another set of eyes on the RMA part in particular though to make sure we’re providing all the needed semantics. I wouldn’t say any of us in the FTWG are RMA or Fortran experts. Thanks, Wesley > > > >On Sep 5, 2015, at 7:09 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > >> So Fortran 2015 (TS 18508 - section 6 in the attachment) is going to support failed images. >> >> The OpenCoarrays folks (who are responsible for enabling GCC 5+ Fortran coarray support) have started looking into how to support this feature. They currently use MPI-3 RMA and GASNet as communication runtimes, but the need to support FT will likely push them in a new direction. The options they have mentioned thus far are undesirable. >> >> It would be great if there were more people who could help look at RMA FT, particularly as it pertains to Fortran 2015. Are any of the Fortran WG folks savvy on how those failures map to MPI concepts and whether or not the MPI RMA FT discussion is going in the right direction? >> >> Thanks, >> >> Jeff >> >> -- >> Jeff Hammond >> jeff.science(a)gmail.com >> http://jeffhammond.github.io/ >> <ISO-IECJTC1-SC22-WG5_N2056_Draft_TS_18508_Additional_Paralle.pdf>_______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > >Bill Long longb(a)cray.com >Fortran Technical Support & voice: 651-605-9024 >Bioinformatics Software Development fax: 651-605-9142 >Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101 > > >_______________________________________________ >mpiwg-fortran mailing list >mpiwg-fortran(a)lists.mpi-forum.org >http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran

1 0

Re: [MPIWG Fortran] Fortran coarrays - failed images
by Bill Long 07 Sep '15

07 Sep '15

Hi Jeff, The current version of the TS is WG5 document N2074. The failed image feature has not changed much recently, but still better to use the latest version. N2074 is currently out for WG5 review. Assuming it passes, this (minus the line numbers that we like but ISO doesn’t) is the version that will be sent to the ISO editors for publication. I think it would be an excellent idea for the TS/Fortran 2015 and MPI facilities for FT to be able to use common underlying infrastructure if possible. I have not thought about how a program that uses both the Fortran and the MPI facilities would work (or how the MPI spec should be written in that regard), but if the links into components like PMI or SLURM would be the same, that would certainly help. I wrote up a summary of the TS features for the benefit of the MPI FT experts, pasted in below. Cheers, Bill ISO TS 18508 includes features that can be used to help a program react to the failure of an image. It is intended to be a minimal capability. Facilities are included for notification, inquiry, testing, and simple continuation of execution. Background: The parallel programming model using coarrays that is included in Fortran 2008 assumes that the number of executing images remains constant for the entire program. As a consequence, failure of an image (typically from a hardware failure) aborted the whole program. The addition of teams in TS 18508 allows for the possibility of the number of images decreasing following a failure by forming a new team consisting of the active images and continuing execution in that team. While this was not the main motivation for introducing teams, this observation lead to the addition of minimal resilience facilities to the TS. Notification: The image control statements include an optional STAT= specifier will return an error status. The image selector syntax for remote references also allow an optional STAT= specifier. The new collective and atomic subroutines have an optional STAT argument that will also return an error status. An error status of zero indicates success. If the operation involved communication with a failed image, the status returned is equal to the named constant STAT_FAILED_IMAGE that is defined in the intrinsic module ISO_FORTRAN_ENV, and execution continues. If there is no status variable provided and the operation involves communication with a failed image, the program aborts. A negative value of STAT_FAILED_IMAGE indicates that the processor cannot detect processor failure, and a positive value indicates it can. This effectively makes provision of this facility optional. [MPI analog: Remote communication operations in MPI either return a status as a function result (C), or have an optional MPI subroutine argument that returns an error status (Fortran). A named constant MPI_FAILED_RANK could be added to the MPI spec to provide a corresponding capability.] Inquiry: Two inquiry functions are provided that can return information on the status of images. IMAGE_STATUS( N ) will return STAT_FAILED_IMAGE if image N has failed. This could be used to check on the health of an image just before a loop that involved many accesses to that image. FAILED_IMAGES( ) returns a 1-D array containing the numbers of the failed images. This can be used to compute which images should be omitted from a new team that can be used for continued execution. If there are no failed images, the returned array has size zero. [MPI analog: Adding corresponding functions ( IMAGE -> RANK ) to MPI would seem straightforward.] Testing: A new statement FAIL IMAGE is added. When executed by image N it causes image N to appear to have failed as seen from the other images. This is included so that programmers can test recovery algorithms without having to wait for an actual failure. An image that executes this statement does not continue execution. [MPI analog: One more function.] Continuation: The FORM TEAM statement allows the program to create a new team. In the case of a failed image, the strategy would be to form a new team consisting of the remaining active images. The CHANGE TEAM statement causes a switch in the execution environment to the specified team. Note that whether the program can meaningfully continue depends on the algorithm being implemented and whether the program includes code to switch teams to continue. [MPI analog: Fortran teams could be mapped to MPI communicators. One difference is that Fortran allows you to omit a team specification in most statements involving communication, in which case the "current" team is used. The MPI spec might want to specify that MPI_COMM_WORLD is redefined in the case that "team" is shrunk through rank failure, to reduce the need to modify existing code.] Implementation issue: The program launching and management environment (PMI, ALPS, SLURM, ...) need to be modified to include an API that can provide failed image information to the program and keep the remaining images executing, as opposed to the current behavior of aborting all the images. The API also needs a function that the program startup code can call to inform the management environment whether it is employing resilient features. It would certainly be advantageous to have the same API used for both Fortran and MPI. ——————————End of Summary —————————————— On Sep 5, 2015, at 7:09 PM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > So Fortran 2015 (TS 18508 - section 6 in the attachment) is going to support failed images. > > The OpenCoarrays folks (who are responsible for enabling GCC 5+ Fortran coarray support) have started looking into how to support this feature. They currently use MPI-3 RMA and GASNet as communication runtimes, but the need to support FT will likely push them in a new direction. The options they have mentioned thus far are undesirable. > > It would be great if there were more people who could help look at RMA FT, particularly as it pertains to Fortran 2015. Are any of the Fortran WG folks savvy on how those failures map to MPI concepts and whether or not the MPI RMA FT discussion is going in the right direction? > > Thanks, > > Jeff > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > <ISO-IECJTC1-SC22-WG5_N2056_Draft_TS_18508_Additional_Paralle.pdf>_______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101

1 0

Fortran coarrays - failed images
by Jeff Hammond 05 Sep '15

05 Sep '15

So Fortran 2015 (TS 18508 - section 6 in the attachment) is going to support failed images. The OpenCoarrays folks (who are responsible for enabling GCC 5+ Fortran coarray support) have started looking into how to support this feature. They currently use MPI-3 RMA and GASNet as communication runtimes, but the need to support FT will likely push them in a new direction. The options they have mentioned thus far are undesirable. It would be great if there were more people who could help look at RMA FT, particularly as it pertains to Fortran 2015. Are any of the Fortran WG folks savvy on how those failures map to MPI concepts and whether or not the MPI RMA FT discussion is going in the right direction? Thanks, Jeff -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Re: [MPIWG Fortran] Question about named constants
by William Gropp 01 Jul '15

01 Jul '15

Just to be very clear, I believe that the Open MPI implementation is valid and in conformance to the standard (in this case). Bill William Gropp Director, Parallel Computing Institute Thomas M. Siebel Chair in Computer Science Chief Scientist, NCSA University of Illinois Urbana-Champaign On Jun 30, 2015, at 11:40 PM, Craig Rasmusen <rasmus(a)cas.uoregon.edu> wrote: > I looked at the Open MPI implementation and it looks like in Fortran we made them pointers to memory initialized in C so that MPI_INTEGER would have the same value across languages. > > MPI_INTEGER is definitely not a Fortran parameter in the Open MPI implementation, > > -craig > > On Fri, Jun 19, 2015 at 5:45 AM, William Gropp <wgropp(a)illinois.edu> wrote: > Given the intent of the wording with respect to C, I don’t think the test is valid (in C, the clear intent was that handles could be pointers resolved only at link time). > > Bill > > On Jun 18, 2015, at 10:19 PM, Bill Long <longb(a)cray.com> wrote: > > > > > On Jun 18, 2015, at 5:52 PM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > > > >> In OpenMPI F08, those predefined MPI datatypes are not "parameter variables". Thus OMPI failed to compile an MPICH F08 test, which has > >> > >> INTEGER, PARAMETER :: nb=2 > >> TYPE(MPI_DATATYPE) :: types(nb)=(/MPI_DOUBLE_PRECISION, MPI_CHAR/) > >> > >> Jeff Squyres and I discussed in emails. I don't know whether this is correct according to the MPI spec. > > > > Given the wording in the spec, I’m not sure either. BTW, there is no such thing as a “parameter variable”. Something with the parameter attribute is a constant, and hence not a variable. It is true that an initialization expression in a type declaration is required to be a constant expression. If OMPI is declaring these datatypes as variables instead of constants, then there would be a problem compiling the above code using their module. > > > > Cheers, > > Bill > > > > > > > >> > >> > >> > >> --Junchao Zhang > >> > >> On Thu, Jun 18, 2015 at 12:48 PM, Bill Long <longb(a)cray.com> wrote: > >> > >> On Jun 18, 2015, at 11:05 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > >> > >>> Hello, Fortran experts, > >>> I am a little puzzled by MPI-3.1 Chapter 2.5.4, > >> > >> Not surprising. The (mis)use of terminology is confusing at best. > >> > >>> > >>> All named constants, with the exceptions noted below for Fortran, can be used in > >>> initialization expressions or assignments, but not necessarily in array declarations or as > >>> labels in C switch or Fortran select/case statements. This implies named constants > >>> to be link-time but not necessarily compile-time constants. > >> > >> In Fortran “named constant” is a defined term. They are declared with the PARAMETER attribute. And can be used in most places where literal constants can be used, including array declarations. They cannot be used as statement labels, though I think the reference to “Fortran select/case statements” intends to talk about case-values, and not labels. Named constants can be used as case-values. > >> > >>> > >>> Then in F08, should MPI_INTEGER be declared as a parameter variable? > >> > >> Yes. That’s what I did in my prototype mpi_f08 modules. > >> > >>> If no, then I get an error when compiling this initialization expression with gfortran > >>> type(MPI_Datatype) :: dtype = MPI_INTEGER > >>> > >>> Error: Parameter 'mpi_integer' at (1) has not been declared or is a variable, which does not reduce to a constant expression > >>> > >>> If yes, then of course I can use MPI_INTEGER in Fortran select/case. > >> > >> Right. > >> > >>> > >>> Another question is what is the meaning of "link-time constants". I could not find in MPI. Does it mean I can use MPI_INTEGER after main before MPI_Init? > >> > >> I suspect this is a historical artifact where some special head standing was needed to correctly define MPI_INTEGER based on whether the code was compiled with -i4 or -i8 (or equivalent, depending on the compiler). For the modern case of using the mpi_f08 module, you just compile the mpi_f08 module twice (once with -i4 and once with -i8) and then have the compile command select the right actual module to reference based on how the using program unit is compiled. Generally, values of objects are not set at link time; addresses of objects are set at link time. MPI objects that represent addresses (MPI_BOTTOM ??) are examples. If MPI_INTEGER is accessible, you can reference it. If should not have anything to do with whether MPI_Init has been called. > >> > >> Perhaps some of the MPI historians can shed more light on this terminology. I suspect it would be completely moot if we got rid of mpif.h. > >> > >> Cheers, > >> Bill > >> > >> > >>> > >>> Thank you! > >>> > >>> --Junchao Zhang > >>> _______________________________________________ > >>> mpiwg-fortran mailing list > >>> mpiwg-fortran(a)lists.mpi-forum.org > >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > >> > >> Bill Long longb(a)cray.com > >> Fortran Technical Support & voice: 651-605-9024 > >> Bioinformatics Software Development fax: 651-605-9142 > >> Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101 > >> > >> > >> _______________________________________________ > >> mpiwg-fortran mailing list > >> mpiwg-fortran(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > >> > >> _______________________________________________ > >> mpiwg-fortran mailing list > >> mpiwg-fortran(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > > > Bill Long longb(a)cray.com > > Fortran Technical Support & voice: 651-605-9024 > > Bioinformatics Software Development fax: 651-605-9142 > > Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101 > > > > > > _______________________________________________ > > mpiwg-fortran mailing list > > mpiwg-fortran(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran

1 0

Re: [MPIWG Fortran] Question about named constants
by Craig Rasmusen 30 Jun '15

30 Jun '15

I looked at the Open MPI implementation and it looks like in Fortran we made them pointers to memory initialized in C so that MPI_INTEGER would have the same value across languages. MPI_INTEGER is definitely not a Fortran parameter in the Open MPI implementation, -craig On Fri, Jun 19, 2015 at 5:45 AM, William Gropp <wgropp(a)illinois.edu> wrote: > Given the intent of the wording with respect to C, I don’t think the test > is valid (in C, the clear intent was that handles could be pointers > resolved only at link time). > > Bill > > On Jun 18, 2015, at 10:19 PM, Bill Long <longb(a)cray.com> wrote: > > > > > On Jun 18, 2015, at 5:52 PM, Junchao Zhang <jczhang(a)mcs.anl.gov> wrote: > > > >> In OpenMPI F08, those predefined MPI datatypes are not "parameter > variables". Thus OMPI failed to compile an MPICH F08 test, which has > >> > >> INTEGER, PARAMETER :: nb=2 > >> TYPE(MPI_DATATYPE) :: types(nb)=(/MPI_DOUBLE_PRECISION, MPI_CHAR/) > >> > >> Jeff Squyres and I discussed in emails. I don't know whether this is > correct according to the MPI spec. > > > > Given the wording in the spec, I’m not sure either. BTW, there is no > such thing as a “parameter variable”. Something with the parameter > attribute is a constant, and hence not a variable. It is true that an > initialization expression in a type declaration is required to be a > constant expression. If OMPI is declaring these datatypes as variables > instead of constants, then there would be a problem compiling the above > code using their module. > > > > Cheers, > > Bill > > > > > > > >> > >> > >> > >> --Junchao Zhang > >> > >> On Thu, Jun 18, 2015 at 12:48 PM, Bill Long <longb(a)cray.com> wrote: > >> > >> On Jun 18, 2015, at 11:05 AM, Junchao Zhang <jczhang(a)mcs.anl.gov> > wrote: > >> > >>> Hello, Fortran experts, > >>> I am a little puzzled by MPI-3.1 Chapter 2.5.4, > >> > >> Not surprising. The (mis)use of terminology is confusing at best. > >> > >>> > >>> All named constants, with the exceptions noted below for Fortran, can > be used in > >>> initialization expressions or assignments, but not necessarily in > array declarations or as > >>> labels in C switch or Fortran select/case statements. This implies > named constants > >>> to be link-time but not necessarily compile-time constants. > >> > >> In Fortran “named constant” is a defined term. They are declared with > the PARAMETER attribute. And can be used in most places where literal > constants can be used, including array declarations. They cannot be used > as statement labels, though I think the reference to “Fortran select/case > statements” intends to talk about case-values, and not labels. Named > constants can be used as case-values. > >> > >>> > >>> Then in F08, should MPI_INTEGER be declared as a parameter variable? > >> > >> Yes. That’s what I did in my prototype mpi_f08 modules. > >> > >>> If no, then I get an error when compiling this initialization > expression with gfortran > >>> type(MPI_Datatype) :: dtype = MPI_INTEGER > >>> > >>> Error: Parameter 'mpi_integer' at (1) has not been declared or is a > variable, which does not reduce to a constant expression > >>> > >>> If yes, then of course I can use MPI_INTEGER in Fortran select/case. > >> > >> Right. > >> > >>> > >>> Another question is what is the meaning of "link-time constants". I > could not find in MPI. Does it mean I can use MPI_INTEGER after main before > MPI_Init? > >> > >> I suspect this is a historical artifact where some special head > standing was needed to correctly define MPI_INTEGER based on whether the > code was compiled with -i4 or -i8 (or equivalent, depending on the > compiler). For the modern case of using the mpi_f08 module, you just > compile the mpi_f08 module twice (once with -i4 and once with -i8) and then > have the compile command select the right actual module to reference based > on how the using program unit is compiled. Generally, values of objects > are not set at link time; addresses of objects are set at link time. MPI > objects that represent addresses (MPI_BOTTOM ??) are examples. If > MPI_INTEGER is accessible, you can reference it. If should not have > anything to do with whether MPI_Init has been called. > >> > >> Perhaps some of the MPI historians can shed more light on this > terminology. I suspect it would be completely moot if we got rid of mpif.h. > >> > >> Cheers, > >> Bill > >> > >> > >>> > >>> Thank you! > >>> > >>> --Junchao Zhang > >>> _______________________________________________ > >>> mpiwg-fortran mailing list > >>> mpiwg-fortran(a)lists.mpi-forum.org > >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > >> > >> Bill Long > longb(a)cray.com > >> Fortran Technical Support & voice: > 651-605-9024 > >> Bioinformatics Software Development fax: > 651-605-9142 > >> Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101 > >> > >> > >> _______________________________________________ > >> mpiwg-fortran mailing list > >> mpiwg-fortran(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > >> > >> _______________________________________________ > >> mpiwg-fortran mailing list > >> mpiwg-fortran(a)lists.mpi-forum.org > >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > > > Bill Long > longb(a)cray.com > > Fortran Technical Support & voice: > 651-605-9024 > > Bioinformatics Software Development fax: > 651-605-9142 > > Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101 > > > > > > _______________________________________________ > > mpiwg-fortran mailing list > > mpiwg-fortran(a)lists.mpi-forum.org > > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >

1 0

Re: [MPIWG Fortran] What compilers support TS 29113?
by N.M. Maclaren 25 Jun '15

25 Jun '15

On Jun 24 2015, Bill Long wrote: > > I think the "except for" is a pretty significant hole here. Yes. What it means is that it works only for arrays of intrinsic types. That accounts for the majority of uses, but not all. And it's not a reasonably thing to document as a restriction. >It is > certainly possible for a C programmer to take the TS 29113 descriptions > and write a header file and the utility functions specified there. Which doesn't help in this case, because it's not the missing functions that are the problem, but the generated code. Regards, Nick Maclaren.

1 0

Re: [MPIWG Fortran] What compilers support TS 29113?
by Hubert Ritzdorf 25 Jun '15

25 Jun '15

My intention was to implement, test and work with the new Fortran features within my NEC version of the MPI library. Therefore I was looking for compiler support. My versions of intel compilers didn't support any features of TS 29113. I don't have access to Cray compilers. NEC compilers didn't support it. At this time, gfortran 4.9.1 supports parts of TS 29113 features and was my only possibility to use Fortran subarrays without copy in/copy out at MPI 3.0 interfaces. Therefore, I used gfortran 4.9.1 and implemented a work around for the standard include file. I implemented the TS 29113 support as special C functions for Fortran, not as "simple" Fortran wrappers. I simply wanted to share my experiences with TS29113 and gfortran and highlight the problems I found. I didn't want demoralizing programmers since gfortran doesn't claim full support of TS 29113 such as Bill Gropp already explained in his mail. I didn't test it together with MPICH. Hubert ________________________________________ From: mpiwg-fortran [mpiwg-fortran-bounces(a)lists.mpi-forum.org] on behalf of Bill Long [longb(a)cray.com] Sent: Wednesday, June 24, 2015 3:16 PM To: MPI-WG Fortran working group Subject: Re: [MPIWG Fortran] What compilers support TS 29113? I think the “except for” is a pretty significant hole here. It is certainly possible for a C programmer to take the TS 29113 descriptions and write a header file and the utility functions specified there. However, there are some missing parts to having a working implementation for Fortran users. To implement the allocate / deallocate functions you need to know how the Fortran compiler allocates and deallocates memory. Not a big issue for MPI, but this one is: The Fortran compiler has to do the descriptor conversions on the caller side if the C descriptor is different from the internal Fortran one. Compiler support for the new descriptor was the big missing piece in gfortran 4.9.1. I thought the intention was to change the gfortran descriptor to match the C one, to simplify calls. If that’s not in 5.1.0, there is still a big hole. Cheers, Bill On Jun 24, 2015, at 8:02 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > The "except for"="does not work" standard is a fantastic way to > demoralize every programmer this side of Don Knuth. Has MPICH ever > worked? > > Jeff > > On Wed, Jun 24, 2015 at 5:11 AM, William Gropp <wgropp(a)illinois.edu> wrote: >> Sounds interesting and useful. However, “except for” is the same as “does not work” when talking about standards. >> >> Bill >> >> William Gropp >> Director, Parallel Computing Institute >> Thomas M. Siebel Chair in Computer Science >> Chief Scientist, NCSA >> University of Illinois Urbana-Champaign >> >> >> >> >> >> On Jun 24, 2015, at 3:29 AM, Hubert Ritzdorf <Hubert.Ritzdorf(a)EMEA.NEC.COM> wrote: >> >>> Hi, >>> >>> I used gfortran 4.9.1 and I have build an include file which adopted the definitions of the missing >>> ISO_Fortran_binding.h to the implementation of gfortran (include file libgfortran.h) . >>> Using this I could implement the Fortran binding for TS29113. It works except for arrays of derived Fortran Types >>> which are always transferred via copy in/copy out. >>> >>> Hubert >>> ________________________________________ >>> From: mpiwg-fortran [mpiwg-fortran-bounces(a)lists.mpi-forum.org] on behalf of William Gropp [wgropp(a)illinois.edu] >>> Sent: Tuesday, June 23, 2015 9:34 PM >>> To: MPI-WG Fortran working group >>> Subject: Re: [MPIWG Fortran] What compilers support TS 29113? >>> >>> gfortran no, not even with 5.1.0 . It does have a lot of F 2008, but not even all of it, according to https://gcc.gnu.org/wiki/Fortran2008Status and https://gcc.gnu.org/wiki/TS29113Status . >>> Gfortran 5.1 fails the MPICH test because ISO_Fortran_binding.h isn’t available for GCC 5.1 (I don’t know if that’s the right test, but that’s what it does). See also https://gcc.gnu.org/onlinedocs/gfortran/Further-Interoperability-of-Fortran… . >>> >>> Bill >>> >>> William Gropp >>> Director, Parallel Computing Institute >>> Thomas M. Siebel Chair in Computer Science >>> Chief Scientist, NCSA >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> >>> >>> On Jun 23, 2015, at 2:01 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: >>> >>>> On Jun 23, 2015, at 11:51 AM, Bill Long <longb(a)cray.com> wrote: >>>>> >>>>> gfortran - yes (4.9 works on my Mac) >>>> >>>> I thought fortran 4.9 did not have the standardized TS 29113 dope vector...? >>>> >>>>> Cray - yes (8.3 for sure, or later). >>>> >>>> I knew about Cray, but I thought it was the only one. That may be incorrect and/or outdated information. >>>> >>>>> Not sure about others. People who build libraries (mpich, OpenMPI, ...) would know based on which ones can compiler their mpi_f08 module. >>>> >>>> Open MPI currently only has the non-TS-29113 interfaces for mpi_f08. We're looking into whether it's time to implement the TS 29113 interfaces; that's the origin of my question. :-) >>>> >>>> -- >>>> Jeff Squyres >>>> jsquyres(a)cisco.com >>>> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ >>>> >>>> _______________________________________________ >>>> mpiwg-fortran mailing list >>>> mpiwg-fortran(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >>> >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >>> >>> >>> Click https://www.mailcontrol.com/sr/op!XYE2yjUjGX2PQPOmvUiQSa3+T5MHvx+zSoc3i1rpp… to report this email as spam. >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >> >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101 _______________________________________________ mpiwg-fortran mailing list mpiwg-fortran(a)lists.mpi-forum.org http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran

1 0

Re: [MPIWG Fortran] What compilers support TS 29113?
by Bill Long 24 Jun '15

24 Jun '15

I think the “except for” is a pretty significant hole here. It is certainly possible for a C programmer to take the TS 29113 descriptions and write a header file and the utility functions specified there. However, there are some missing parts to having a working implementation for Fortran users. To implement the allocate / deallocate functions you need to know how the Fortran compiler allocates and deallocates memory. Not a big issue for MPI, but this one is: The Fortran compiler has to do the descriptor conversions on the caller side if the C descriptor is different from the internal Fortran one. Compiler support for the new descriptor was the big missing piece in gfortran 4.9.1. I thought the intention was to change the gfortran descriptor to match the C one, to simplify calls. If that’s not in 5.1.0, there is still a big hole. Cheers, Bill On Jun 24, 2015, at 8:02 AM, Jeff Hammond <jeff.science(a)gmail.com> wrote: > The "except for"="does not work" standard is a fantastic way to > demoralize every programmer this side of Don Knuth. Has MPICH ever > worked? > > Jeff > > On Wed, Jun 24, 2015 at 5:11 AM, William Gropp <wgropp(a)illinois.edu> wrote: >> Sounds interesting and useful. However, “except for” is the same as “does not work” when talking about standards. >> >> Bill >> >> William Gropp >> Director, Parallel Computing Institute >> Thomas M. Siebel Chair in Computer Science >> Chief Scientist, NCSA >> University of Illinois Urbana-Champaign >> >> >> >> >> >> On Jun 24, 2015, at 3:29 AM, Hubert Ritzdorf <Hubert.Ritzdorf(a)EMEA.NEC.COM> wrote: >> >>> Hi, >>> >>> I used gfortran 4.9.1 and I have build an include file which adopted the definitions of the missing >>> ISO_Fortran_binding.h to the implementation of gfortran (include file libgfortran.h) . >>> Using this I could implement the Fortran binding for TS29113. It works except for arrays of derived Fortran Types >>> which are always transferred via copy in/copy out. >>> >>> Hubert >>> ________________________________________ >>> From: mpiwg-fortran [mpiwg-fortran-bounces(a)lists.mpi-forum.org] on behalf of William Gropp [wgropp(a)illinois.edu] >>> Sent: Tuesday, June 23, 2015 9:34 PM >>> To: MPI-WG Fortran working group >>> Subject: Re: [MPIWG Fortran] What compilers support TS 29113? >>> >>> gfortran no, not even with 5.1.0 . It does have a lot of F 2008, but not even all of it, according to https://gcc.gnu.org/wiki/Fortran2008Status and https://gcc.gnu.org/wiki/TS29113Status . >>> Gfortran 5.1 fails the MPICH test because ISO_Fortran_binding.h isn’t available for GCC 5.1 (I don’t know if that’s the right test, but that’s what it does). See also https://gcc.gnu.org/onlinedocs/gfortran/Further-Interoperability-of-Fortran… . >>> >>> Bill >>> >>> William Gropp >>> Director, Parallel Computing Institute >>> Thomas M. Siebel Chair in Computer Science >>> Chief Scientist, NCSA >>> University of Illinois Urbana-Champaign >>> >>> >>> >>> >>> >>> On Jun 23, 2015, at 2:01 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: >>> >>>> On Jun 23, 2015, at 11:51 AM, Bill Long <longb(a)cray.com> wrote: >>>>> >>>>> gfortran - yes (4.9 works on my Mac) >>>> >>>> I thought fortran 4.9 did not have the standardized TS 29113 dope vector...? >>>> >>>>> Cray - yes (8.3 for sure, or later). >>>> >>>> I knew about Cray, but I thought it was the only one. That may be incorrect and/or outdated information. >>>> >>>>> Not sure about others. People who build libraries (mpich, OpenMPI, ...) would know based on which ones can compiler their mpi_f08 module. >>>> >>>> Open MPI currently only has the non-TS-29113 interfaces for mpi_f08. We're looking into whether it's time to implement the TS 29113 interfaces; that's the origin of my question. :-) >>>> >>>> -- >>>> Jeff Squyres >>>> jsquyres(a)cisco.com >>>> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ >>>> >>>> _______________________________________________ >>>> mpiwg-fortran mailing list >>>> mpiwg-fortran(a)lists.mpi-forum.org >>>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >>> >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >>> >>> >>> Click https://www.mailcontrol.com/sr/op!XYE2yjUjGX2PQPOmvUiQSa3+T5MHvx+zSoc3i1rpp… to report this email as spam. >>> _______________________________________________ >>> mpiwg-fortran mailing list >>> mpiwg-fortran(a)lists.mpi-forum.org >>> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran >> >> _______________________________________________ >> mpiwg-fortran mailing list >> mpiwg-fortran(a)lists.mpi-forum.org >> http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran > > > > -- > Jeff Hammond > jeff.science(a)gmail.com > http://jeffhammond.github.io/ > _______________________________________________ > mpiwg-fortran mailing list > mpiwg-fortran(a)lists.mpi-forum.org > http://lists.mpi-forum.org/mailman/listinfo.cgi/mpiwg-fortran Bill Long longb(a)cray.com Fortran Technical Support & voice: 651-605-9024 Bioinformatics Software Development fax: 651-605-9142 Cray Inc./ Cray Plaza, Suite 210/ 380 Jackson St./ St. Paul, MN 55101

1 0