- mpiwg-io - mailman3.cels.anl.gov

MPI-IO WG vote on deprecating split collectives next week, 11/2 @ 8am PT
by Quincey Koziol 22 Oct '24

22 Oct '24

Hi all, We’d like to have a working group vote on Edgar’s PR to deprecate the split collective I/O operations, https://github.com/mpiwg-io/mpi-standard/pull/1 We’ll vote on the change at the next meeting, on Friday 11/2, at 9-10am PT. If the vote is in favor, Edgar will take it to the next plenary meeting of the forum for its first reading. Quincey

1 0

Today's MPI-IO working group meeting is cancelled
by Quincey Koziol 05 Sep '24

05 Sep '24

Conflicts, etc. We’ll aim for 2 weeks from today. Quincey

1 0

MPI-IO Working Group
by Quincey Koziol 08 Aug '24

08 Aug '24

Apologies for the late notice, but today’s I/O working group meeting is canceled. We’ll try again in two weeks. Regards, Quincey MPI-IO Working Group Scheduled: Aug 8, 2024 at 3:00 PM to 4:00 PM, CDT Location: Microsoft Teams Meeting Invitees: Pritchard Jr., Howard (EXTERNAL), wang116(a)llnl.gov, zhouh(a)anl.gov, raffenet(a)mcs.anl.gov, juplehn(a)me.com, derobins(a)hdfgroup.org, byna.1(a)osu.edu, nfortne2(a)hdfgroup.org, edgar.gabriel(a)amd.com, skjellum(a)gmail.com, jhenderson(a)hdfgroup.org, mohamad.chaarawi(a)intel.com, john.mainzer(a)lifeboat.llc, mohror1(a)llnl.gov, elena.pourmal(a)lifeboat.llc, schuchart(a)icl.utk.edu, Raffenetti, Ken, Quincey Koziol, Houjun Tang (EXTERNAL), daniel.john.holmes(a)intel.com, epourmal(a)gmail.com, Jeff Hammond, jannek.squar(a)uni-hamburg.de, anna.fuchs(a)uni-hamburg.de, jlbez(a)lbl.gov, tony-skjellum(a)utc.edu, robl(a)mcs.anl.gov Apologies for the duplicate invite, but I don't have access to my Amazon calendar to update the old one. Please send me your Github username or email if you'd like to be added to the mpiwg-io Github org (https://github.com/mpiwg-io) ________________________________________________________________________________ Microsoft Teams meeting Join on your computer, mobile app or room device Click here to join the meeting<https://teams.microsoft.com/l/meetup-join/19%3ameeting_ZDQ5Y2M2ZTItZmM1Mi00…> Meeting ID: 237 625 329 218 Passcode: sus3WX Download Teams<https://www.microsoft.com/en-us/microsoft-teams/download-app> | Join on the web<https://www.microsoft.com/microsoft-teams/join-a-meeting> Join with a video conferencing device teams(a)vc.nvidia.com Video Conference ID: 111 437 713 2 Alternate VTC instructions<https://pexip.me/teams/vc.nvidia.com/1114377132> Or call in (audio only) +1 949-570-1120,,43312048#<tel:+19495701120,,43312048#> United States, Irvine Phone Conference ID: 433 120 48# Find a local number<https://dialin.teams.microsoft.com/c58d41ec-314f-4761-9381-98dfc4066d1b?id=…> | Reset PIN<https://dialin.teams.microsoft.com/usp/pstnconferencing> Learn More<https://aka.ms/JoinTeamsMeeting> | Meeting options<https://teams.microsoft.com/meetingOptions/?organizerId=1091c645-a4df-4cdd-…> ________________________________________________________________________________

1 0

Canceling today's meeting
by Quincey Koziol 25 Jul '24

25 Jul '24

Hi all, I haven’t had time to update the Latex info for the “no view” API routines due to some competing priorities here, so I’m going to cancel this week’s meeting. I will have them updated for the meeting in 2 weeks. Regards, Quincey

1 0

Canceling today's meeting
by Quincey Koziol 27 Jun '24

27 Jun '24

I’m triple-booked and Chen’s been working on other things.

1 0

Canceling today's meeting
by Quincey Koziol 13 Jun '24

13 Jun '24

Conflict for me & progress for Chen

1 0

MPI-IO Working Group canceled for today, 5/2
by Quincey Koziol 02 May '24

02 May '24

Not enough progress on items to discuss. Aiming for the next meeting, on 5/16. Quincey MPI-IO Working Group Scheduled: May 2, 2024 at 3:00 PM to 4:00 PM, CDT Location: Microsoft Teams Meeting Invitees: edgar.gabriel(a)amd.com, zhouh(a)anl.gov, Jeff Hammond, skjellum(a)gmail.com, anna.fuchs(a)uni-hamburg.de, john.mainzer(a)lifeboat.llc, mohror1(a)llnl.gov, jhenderson(a)hdfgroup.org, Quincey Koziol, Houjun Tang (EXTERNAL), epourmal(a)gmail.com, raffenet(a)mcs.anl.gov, jannek.squar(a)uni-hamburg.de, byna.1(a)osu.edu, nfortne2(a)hdfgroup.org, Raffenetti, Ken, elena.pourmal(a)lifeboat.llc, Pritchard Jr., Howard (EXTERNAL), jlbez(a)lbl.gov, derobins(a)hdfgroup.org, schuchart(a)icl.utk.edu, mohamad.chaarawi(a)intel.com, tony-skjellum(a)utc.edu, juplehn(a)me.com, robl(a)mcs.anl.gov, daniel.john.holmes(a)intel.com, wang116(a)llnl.gov Apologies for the duplicate invite, but I don't have access to my Amazon calendar to update the old one. Please send me your Github username or email if you'd like to be added to the mpiwg-io Github org (https://github.com/mpiwg-io) ________________________________________________________________________________ Microsoft Teams meeting Join on your computer, mobile app or room device Click here to join the meeting<https://teams.microsoft.com/l/meetup-join/19%3ameeting_ZDQ5Y2M2ZTItZmM1Mi00…> Meeting ID: 237 625 329 218 Passcode: sus3WX Download Teams<https://www.microsoft.com/en-us/microsoft-teams/download-app> | Join on the web<https://www.microsoft.com/microsoft-teams/join-a-meeting> Join with a video conferencing device teams(a)vc.nvidia.com Video Conference ID: 111 437 713 2 Alternate VTC instructions<https://pexip.me/teams/vc.nvidia.com/1114377132> Or call in (audio only) +1 949-570-1120,,43312048#<tel:+19495701120,,43312048#> United States, Irvine Phone Conference ID: 433 120 48# Find a local number<https://dialin.teams.microsoft.com/c58d41ec-314f-4761-9381-98dfc4066d1b?id=…> | Reset PIN<https://dialin.teams.microsoft.com/usp/pstnconferencing> Learn More<https://aka.ms/JoinTeamsMeeting> | Meeting options<https://teams.microsoft.com/meetingOptions/?organizerId=1091c645-a4df-4cdd-…> ________________________________________________________________________________

1 0

New file access routines for discussion today
by Quincey Koziol 18 Apr '24

18 Apr '24

1 0

MPI-IO meeting notes for 2024-04-04
by Quincey Koziol 18 Apr '24

18 Apr '24

Notes from the last meeting -Q ________________________________________________________________ We discuss 3 variants of changes to filew views, all designed to enable non-contiguous independent I/O operations, which are not possible with the current APIs and semantics. Options: A - Add an "view is local" key to the Info object that MPI_File_set_view() takes, which allows a rank-specific override of MPI_File_write() operations. For example, something like the code below: ========= /* Set "view is local" info key for MPI_File_set_view() call */ MPI_Info_create(&info); MPI_Info_set(info, "view_is_local", "true"); /* MPI_File_set_view() sets the file visible region to this process starts * at offset. Setting etype argument to MPI_INT means this file will be * accessed in the units of integer size. Setting filetype argument to * MPI_INT means a contiguous 4-byte region (assuming integer size if 4 * bytes) is recursively applied to the file to form the visible region to * the calling process, starting from its "offset" set in the offset * argument. In this example, the "file view" of a process is the entire * file starting from its offset. */ err = MPI_File_set_view(fh, offset, MPI_INT, MPI_INT, "native", info); CHECK_ERR(MPI_File_set_view); /* Each process writes 3 integers to the file region visible to it. * Note the file pointer will advance 3x4 bytes after this call. */ err = MPI_File_write(fh, &buf[0], 3, MPI_INT, &status); CHECK_ERR(MPI_File_set_view); ========= The option was rejected due to the difficulties in avoiding hangs on remote processes that perform collective operations and don't know that other processes have set local file views. Additionally, setting a file view is defined to be a collective operation currently and even if relax that requirement for setting a view with a "local" key, it may be impossible to switch _back_ to a collective view without performing a collective operation that the other processes aren't aware they should do. (I believe there were some other reasons that came up during the discussion also, if anyone else recalls them) --- B - Add a "views are local" key to the Info object that MPI_File_open() takes, allowing each rank to specify their own views. For example, something like the code below: ========= /* Set "view is local" info key for MPI_File_open() call */ MPI_Info_create(&info); MPI_Info_set(info, "view_is_local", "true"); /* open a file (create if the file does not exist) */ cmode = MPI_MODE_CREATE | MPI_MODE_RDWR; err = MPI_File_open(MPI_COMM_WORLD, filename, cmode, info, &fh); CHECK_ERR(MPI_File_open); ========= This was rejected due to the "spooky action at a distance" nature of changing the semantics of MPI_File_set_view() with an info key specified at file open. Also, many people trended toward thinking about info keys as 'hints', and setting this key was a very significant change to the required behavior of file views. --- C - Add non-view based read & write operations to the standard, where the datatype & offset in the file is specified with each I/O operation. For example, something like the code below: ========= /* Each process writes 3 integers to the file region visible to it. * Note the file pointer will advance 3x4 bytes after this call. */ err = MPI_File_new_write(fh, offset, MPI_INT, MPI_INT, &buf[0], 3, MPI_INT, CHECK_ERR(MPI_File_set_view); ========= Ultimately, this is the direction we decided to go. It can co-exist with existing file I/O operationa that rely on file views, while enabling applications & middleware that wish to perform non-contiguous independent operations. Action item: Quincey will come up with suggested API routines, limiting the scope to the following matrix: collective/independent X blocking/ nonblocking X "explicit offset"/"individual file pointer" ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 96,9 All

1 0

File for "local views" discussion in 10 minutes
by Quincey Koziol 04 Apr '24

04 Apr '24

1 0