January 2023 - mpiwg-persistence - mailman3.cels.anl.gov

Re: [mpiwg-persistence] [mpiwg-hybridpm] HACC WG Meeting
by Jim Dinan 25 Jan '23

25 Jan '23

Hi All, Let's meet at 10am ET today to discuss the partitioned communication issue Dan mentioned below. Cross-posting this meeting announcement to both HACC and Persistence WGs as the topic spans both. We'll meet on the teams meeting that is shared between the WGs: https://github.com/mpiwg-hybrid/mpi-standard/wiki Cheers, ~Jim. On Wed, Jan 25, 2023 at 6:28 AM Holmes, Daniel John < daniel.john.holmes(a)intel.com> wrote: > Hi Jim, > > > > We could look at “completion call before all partitions are marked ready” > – see attached email chain or visit the Persistence WG list archive: > > > https://lists.mpi-forum.org/pipermail/mpiwg-persistence/2023-January/000257… > > > > Agenda: > > 1. Confirm agreement of the WG that the answer I gave is the intended > canonical semantic > 2. Figure out better wording that states that intent clearly > 3. Assign a lead person who will drive the issue/PR to the March > voting meeting > > > > This falls between the remits of several WGs: HACC is currently the most > active of those with an interest. > > > > Best wishes, > > Dan > > > > *From:* mpiwg-hybridpm <mpiwg-hybridpm-bounces(a)lists.mpi-forum.org> *On > Behalf Of *Jim Dinan via mpiwg-hybridpm > *Sent:* 24 January 2023 21:42 > *To:* Hybrid working group mailing list < > mpiwg-hybridpm(a)lists.mpi-forum.org> > *Cc:* Jim Dinan <james.dinan(a)gmail.com> > *Subject:* [mpiwg-hybridpm] HACC WG Meeting > > > > Hi All, > > > > The HACC WG will meet tomorrow (Wed) at the usual time. Please send any > topics for this week's meeting. > > > > Cheers, > > ~Jim. >

1 0

Re: [mpiwg-persistence] Completion call before all partitions marked ready
by Skjellum, Anthony 24 Jan '23

24 Jan '23

Hi all — we can make a ticket —- Iets discuss in The WG as soon as next slot for this on a Wednesday … Tony Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> on behalf of Holmes, Daniel John via mpiwg-persistence <mpiwg-persistence(a)lists.mpi-forum.org> Sent: Tuesday, January 24, 2023 6:59:52 AM To: Joachim Protze <protze(a)itc.rwth-aachen.de>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Subject: Re: [mpiwg-persistence] Completion call before all partitions marked ready Hi Joachim, That exact question was raised and discussed in the Forum during the standardisation procedure for partitioned communication. The resulting intent, IMHO, was that calling a completion procedure before all partitions are marked ready is permitted, but will always respond with its "not completed" semantic -- MPI_TEST will set flag==false and MPI_WAIT will not return (until something changes -- in a multithreaded situation, another thread could make calls to MPI_Pready and release MPI_WAIT from what would otherwise be a deadlock). This modus operandi may be important for offload to accelerators, where the "other thread" that calls MPI_Pready is actually a kernel running on an accelerator device (once we have accelerator bindings for MPI_Pready and can guarantee sufficient preparedness of the partitioned communication operation that such a device call makes sense, which are both work in progress). As this question has arisen several times from different people, I would say that the intent is not stated in the document clearly enough. We should work on writing some better wording. Given that the intended semantics would not be changing, we might still have time to get a wording fix proposal into MPI-4.1 -- it needs to be read at the March voting meeting to hit that goal. Best wishes, Dan. -----Original Message----- From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> On Behalf Of Joachim Protze via mpiwg-persistence Sent: 21 September 2022 08:21 To: mpiwg-persistence(a)lists.mpi-forum.org Subject: [mpiwg-persistence] Completion call before all partitions marked ready Hello wg-persistence, Looking at the MPI 4.0 document, it is not clear to us, whether it is allowed to call a completion call for a partitioned communication request before all partitions are marked ready. A simple single-threaded example would be: ```C MPI_Psend_init(message, partitions, COUNT, MPI_DOUBLE, dest, tag, MPI_COMM_WORLD, MPI_INFO_NULL, &request); MPI_Start(&request); for(i = 0; i < partitions-1; ++i) { MPI_Pready(i, request); } MPI_Test(&request, &flag, MPI_STATUS_IGNORE); // flag will always be 0 MPI_Pready(partitions-1, request); MPI_Wait(&request, MPI_STATUS_IGNORE); MPI_Request_free(&request); ``` The question becomes more relevant in a multi-threaded context. One thread could finish the work early and call MPI_Wait to detect when all partitions were sent. From my understanding, the only requirement is that all partitions must be marked ready with explicit ready calls before the operation can complete. Replacing the test in above example with a wait call would result in deadlock. Best Joachim -- Dr. rer. nat. Joachim Protze IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 protze(a)itc.rwth-aachen.de https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.itc.rw… _______________________________________________ mpiwg-persistence mailing list mpiwg-persistence(a)lists.mpi-forum.org https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.mpi…

1 0

Re: [mpiwg-persistence] Completion call before all partitions marked ready
by Holmes, Daniel John 24 Jan '23

24 Jan '23

Hi Joachim, That exact question was raised and discussed in the Forum during the standardisation procedure for partitioned communication. The resulting intent, IMHO, was that calling a completion procedure before all partitions are marked ready is permitted, but will always respond with its "not completed" semantic -- MPI_TEST will set flag==false and MPI_WAIT will not return (until something changes -- in a multithreaded situation, another thread could make calls to MPI_Pready and release MPI_WAIT from what would otherwise be a deadlock). This modus operandi may be important for offload to accelerators, where the "other thread" that calls MPI_Pready is actually a kernel running on an accelerator device (once we have accelerator bindings for MPI_Pready and can guarantee sufficient preparedness of the partitioned communication operation that such a device call makes sense, which are both work in progress). As this question has arisen several times from different people, I would say that the intent is not stated in the document clearly enough. We should work on writing some better wording. Given that the intended semantics would not be changing, we might still have time to get a wording fix proposal into MPI-4.1 -- it needs to be read at the March voting meeting to hit that goal. Best wishes, Dan. -----Original Message----- From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> On Behalf Of Joachim Protze via mpiwg-persistence Sent: 21 September 2022 08:21 To: mpiwg-persistence(a)lists.mpi-forum.org Subject: [mpiwg-persistence] Completion call before all partitions marked ready Hello wg-persistence, Looking at the MPI 4.0 document, it is not clear to us, whether it is allowed to call a completion call for a partitioned communication request before all partitions are marked ready. A simple single-threaded example would be: ```C MPI_Psend_init(message, partitions, COUNT, MPI_DOUBLE, dest, tag, MPI_COMM_WORLD, MPI_INFO_NULL, &request); MPI_Start(&request); for(i = 0; i < partitions-1; ++i) { MPI_Pready(i, request); } MPI_Test(&request, &flag, MPI_STATUS_IGNORE); // flag will always be 0 MPI_Pready(partitions-1, request); MPI_Wait(&request, MPI_STATUS_IGNORE); MPI_Request_free(&request); ``` The question becomes more relevant in a multi-threaded context. One thread could finish the work early and call MPI_Wait to detect when all partitions were sent. From my understanding, the only requirement is that all partitions must be marked ready with explicit ready calls before the operation can complete. Replacing the test in above example with a wait call would result in deadlock. Best Joachim -- Dr. rer. nat. Joachim Protze IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 protze(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0

Re: [mpiwg-persistence] progress rule vs. partitioned communication
by Holmes, Daniel John 11 Jan '23

11 Jan '23

Hi Joachim, > A naive implementation can simply wait for the entire message buffer > to be marked ready before any transfer(s) occur and could wait until > the completion function is called on a request before transferring > data. IMHO, the request mentioned here is intended to be the receive request, not the send request. That should be clarified to avoid the misunderstanding you highlight. While Process 0 is blocked in Recv(1), it must guarantee progress (which includes progress for all enabled operations, which includes the partitioned send operation). While Process 1 is blocked in Wait(r), it must guarantee progress (which includes progress for all enabled operations, which includes the partitioned receive operation). Thus, Process 1 must return eventually and will reach Send(0), which guarantees that Process 0 will also finish eventually. Best wishes, Dan. -----Original Message----- From: mpiwg-persistence <mpiwg-persistence-bounces(a)lists.mpi-forum.org> On Behalf Of Joachim Jenke via mpiwg-persistence Sent: 11 January 2023 14:05 To: mpiwg-persistence(a)lists.mpi-forum.org Subject: [mpiwg-persistence] progress rule vs. partitioned communication Hi all, while working on deadlock detection for partitioned P2P communication we could not really come to a conclusion, whether a variant of the following non-deadlock scenario is a deadlock with partitioned communication: Process 0 Process 1 MPI_Isend(1,r) MPI_Recv(0) MPI_Recv(1) MPI_Send(0) MPI_Wait(r) Transformed into partitioned communication: Process 0 Process 1 MPI_Psend_init(1,r) MPI_Precv_init(0,r) MPI_Start(r) MPI_Start(r) MPI_Pready_range(all,r) MPI_Wait(r) MPI_Recv(1) MPI_Send(0) MPI_Wait(r) The first example should not deadlock according to the process paragraph in MPI4,p75,l13pp: > A call to MPI_WAIT that completes a receive will eventually terminate > and return if a matching send has been started , unless the send is > satisfied by another receive. In particular, if the matching send is > nonblocking, then the receive should complete even if no call is > executed by the sender to complete the send. The second example might potentially deadlock according to the rational in MPI4,p106,l3pp: > A naive implementation can simply wait for the entire message buffer > to be marked ready before any transfer(s) occur and could wait until > the completion function is called on a request before transferring > data. If calling the completion function is necessary to eventually send the data, the second example will deadlock with Process 0 stalling in the MPI_Recv and Process 1 stalling in the MPI_Wait. From my perspective this contradicts the process definition for MPI_WAIT completing a receive where the matching send has been started. Another question might be when a partitioned send is started in the sense of the process rule for MPI_WAIT. I checked the current mpi-4.x branch and the referenced wording seems to be still the same. Did we miss something? Best Joachim -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 jenke(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0

Re: [mpiwg-persistence] MPI Persistent/HACC/Partitioned WGs Weekly Meeting---ZOOM NOT TEAMS
by Skjellum, Anthony 11 Jan '23

11 Jan '23

Dear all: For January 18, 2023 meeting, we will focus on Pbuf_prepare issues, timeline, side document, etc. Please attend. We started this in the HACC meeting today. If you need the recording from today's HACC meeting, let me know please. Thanks, Tony Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: Skjellum, Anthony Sent: Wednesday, May 12, 2021 10:53 AM To: mpiwg-coll(a)lists.mpi-forum.org <mpiwg-coll(a)lists.mpi-forum.org>; mpiwg-hybridpm(a)lists.mpi-forum.urg <mpiwg-hybridpm(a)lists.mpi-forum.urg>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Cc: Subramoni, Hari <subramoni.1(a)osu.edu>; Purushotham Bangalore <pvbangalore(a)ua.edu>; Bronis R. de Supinski <bronis(a)llnl.gov>; Guillaume Mercier <guillaume.mercier(a)u-bordeaux.fr>; Raffenetti, Ken <raffenet(a)anl.gov>; Hugo Taboada <hugo.taboada(a)cea.fr>; Klemm, Michael <Michael.Klemm(a)amd.com>; Hines, Thomas; Schulz, Martin <martin.w.j.schulz(a)tum.de>; Garzaran, Maria <maria.garzaran(a)intel.com>; McMahon, Kevin; m.a.ruefenacht(a)gmail.com <m.a.ruefenacht(a)gmail.com>; Amanzholov, Muzakhir <msamanzhol42(a)tntech.edu>; Ryan Grant <regrant233(a)gmail.com>; Elis, Bengisu <bengisu.elis(a)tum.de>; Shipley, Riley Prescott <riley-shipley(a)utc.edu>; daniel.collis.holmes(a)gmail.com <daniel.collis.holmes(a)gmail.com>; de Supinski, Bronis R. <desupinski1(a)llnl.gov>; Christoph Niethammer <niethammer(a)hlrs.de>; Smith, Brian <smithbe(a)ornl.gov>; Thakur, Rajeev <thakur(a)anl.gov>; Wesley Bland <work(a)wesbland.com>; Schonbein, William Whitney <wwschon(a)sandia.gov>; Raffenetti, Kenneth J. <raffenet(a)mcs.anl.gov>; Suggs, Evan <evan-suggs(a)utc.edu>; jeff.science(a)gmail.com <jeff.science(a)gmail.com>; Patrick Bridges <patrickb(a)unm.edu>; Dosanjh, Matthew <mdosanj(a)sandia.gov>; Julien.JAEGER(a)CEA.FR <Julien.JAEGER(a)CEA.FR>; Ghafoor, Sheikh <SGhafoor(a)tntech.edu>; Holmes, Daniel John <daniel.john.holmes(a)intel.com> Subject: MPI Persistent/HACC/Partitioned WGs Weekly Meeting---ZOOM NOT TEAMS When: Wednesday, January 18, 2023 10:00 AM-11:00 AM. Where: NOTE: Zoom, not TEAMS, see below. As of May 12, 2021, WG materials being archived here https://drive.google.com/drive/folders/1dxWKDUxvzol5PQfMET2bwiiNs7rZfimX?us… Weekly: https://tennessee.zoom.us/meeting/tJIsfuqtrDsiE9S0ICfXsuy-47_0ZqWvG71z/ics?… Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/96193520890?pwd=T3k1NkFRQ3BJaUFkeXFQcmtYTlRHQT09 Password: 399650 ---- Hi there, Anthony Skjellum is inviting you to a scheduled Zoom meeting. Topic: Partition/HACC/etc Working Groups Time: Apr 6, 2022 10:00 AM Eastern Time (US and Canada) Every week on Wed, until Jan 25, 2023, 43 occurrence(s) Apr 6, 2022 10:00 AM Apr 13, 2022 10:00 AM Apr 20, 2022 10:00 AM Apr 27, 2022 10:00 AM May 4, 2022 10:00 AM May 11, 2022 10:00 AM May 18, 2022 10:00 AM May 25, 2022 10:00 AM Jun 1, 2022 10:00 AM Jun 8, 2022 10:00 AM Jun 15, 2022 10:00 AM Jun 22, 2022 10:00 AM Jun 29, 2022 10:00 AM Jul 6, 2022 10:00 AM Jul 13, 2022 10:00 AM Jul 20, 2022 10:00 AM Jul 27, 2022 10:00 AM Aug 3, 2022 10:00 AM Aug 10, 2022 10:00 AM Aug 17, 2022 10:00 AM Aug 24, 2022 10:00 AM Aug 31, 2022 10:00 AM Sep 7, 2022 10:00 AM Sep 14, 2022 10:00 AM Sep 21, 2022 10:00 AM Sep 28, 2022 10:00 AM Oct 5, 2022 10:00 AM Oct 12, 2022 10:00 AM Oct 19, 2022 10:00 AM Oct 26, 2022 10:00 AM Nov 2, 2022 10:00 AM Nov 9, 2022 10:00 AM Nov 16, 2022 10:00 AM Nov 23, 2022 10:00 AM Nov 30, 2022 10:00 AM Dec 7, 2022 10:00 AM Dec 14, 2022 10:00 AM Dec 21, 2022 10:00 AM Dec 28, 2022 10:00 AM Jan 4, 2023 10:00 AM Jan 11, 2023 10:00 AM Jan 18, 2023 10:00 AM Jan 25, 2023 10:00 AM Please download and import the following iCalendar (.ics) files to your calendar system. Weekly: https://tennessee.zoom.us/meeting/tJIsfuqtrDsiE9S0ICfXsuy-47_0ZqWvG71z/ics?… Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/96193520890?pwd=T3k1NkFRQ3BJaUFkeXFQcmtYTlRHQT09 Password: 399650 Or iPhone one-tap (US Toll): +16468769923,96193520890# or +13017158592,96193520890# Or Telephone: Dial: +1 646 876 9923 (US Toll) +1 301 715 8592 (US Toll) +1 312 626 6799 (US Toll) +1 669 900 6833 (US Toll) +1 253 215 8782 (US Toll) +1 346 248 7799 (US Toll) Meeting ID: 961 9352 0890 International numbers available: https://tennessee.zoom.us/u/adIMYx6Jko Or an H.323/SIP room system: H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) Meeting ID: 961 9352 0890 Password: 399650 SIP: 96193520890(a)zoomcrc.com Password: 399650 <https://teams.microsoft.com/meetingOptions/?organizerId=e544ff3d-a271-4377-…> ________________________________________________________________________________ ________________________________________________________________________________ Microsoft Teams meeting Join on your computer, mobile app or room device Click here to join the meeting<https://teams.microsoft.com/l/meetup-join/19%3ameeting_ZTZlNjdhMGItZjM4Mi00…> Meeting ID: 251 308 468 346 Passcode: jqEhgw Download Teams<https://www.microsoft.com/en-us/microsoft-teams/download-app> | Join on the web<https://www.microsoft.com/microsoft-teams/join-a-meeting> Learn More<https://aka.ms/JoinTeamsMeeting> | Meeting options<https://teams.microsoft.com/meetingOptions/?organizerId=e544ff3d-a271-4377-…> ________________________________________________________________________________

1 0

progress rule vs. partitioned communication
by Joachim Jenke 11 Jan '23

11 Jan '23

Hi all, while working on deadlock detection for partitioned P2P communication we could not really come to a conclusion, whether a variant of the following non-deadlock scenario is a deadlock with partitioned communication: Process 0 Process 1 MPI_Isend(1,r) MPI_Recv(0) MPI_Recv(1) MPI_Send(0) MPI_Wait(r) Transformed into partitioned communication: Process 0 Process 1 MPI_Psend_init(1,r) MPI_Precv_init(0,r) MPI_Start(r) MPI_Start(r) MPI_Pready_range(all,r) MPI_Wait(r) MPI_Recv(1) MPI_Send(0) MPI_Wait(r) The first example should not deadlock according to the process paragraph in MPI4,p75,l13pp: > A call to MPI_WAIT that completes a receive will eventually terminate > and return if a matching send has been started , unless the send is > satisfied by another receive. In particular, if the matching send is > nonblocking, then the receive should complete even if no call is > executed by the sender to complete the send. The second example might potentially deadlock according to the rational in MPI4,p106,l3pp: > A naive implementation can simply wait for the entire message buffer > to be marked ready before any transfer(s) occur and could wait until > the completion function is called on a request before transferring > data. If calling the completion function is necessary to eventually send the data, the second example will deadlock with Process 0 stalling in the MPI_Recv and Process 1 stalling in the MPI_Wait. From my perspective this contradicts the process definition for MPI_WAIT completing a receive where the matching send has been started. Another question might be when a partitioned send is started in the sense of the process rule for MPI_WAIT. I checked the current mpi-4.x branch and the referenced wording seems to be still the same. Did we miss something? Best Joachim -- Dr. rer. nat. Joachim Jenke IT Center Group: High Performance Computing Division: Computational Science and Engineering RWTH Aachen University Seffenter Weg 23 D 52074 Aachen (Germany) Tel: +49 241 80- 24765 Fax: +49 241 80-624765 jenke(a)itc.rwth-aachen.de www.itc.rwth-aachen.de

1 0

Re: [mpiwg-persistence] MPI Persistent/HACC/Partitioned WGs Weekly Meeting---ZOOM NOT TEAMS
by Skjellum, Anthony 04 Jan '23

04 Jan '23

Hi, I do not have an agenda for today, but giving Jim a chance to convene HACC if he wishes. Otherwise, we will cancel today. I will coordinate with Jim on the schedule for 2023 Spring and we will update you all. I hope to hold more regular meetings of my WG(s) this year. Thanks Tony Anthony Skjellum, PhD Professor of Computer Science and Chair of Excellence Director, SimCenter University of Tennessee at Chattanooga (UTC) tony-skjellum(a)utc.edu [or skjellum(a)gmail.com] cell: 205-807-4968 ________________________________ From: Skjellum, Anthony Sent: Wednesday, May 12, 2021 10:53 AM To: mpiwg-coll(a)lists.mpi-forum.org <mpiwg-coll(a)lists.mpi-forum.org>; mpiwg-hybridpm(a)lists.mpi-forum.urg <mpiwg-hybridpm(a)lists.mpi-forum.urg>; mpiwg-persistence(a)lists.mpi-forum.org <mpiwg-persistence(a)lists.mpi-forum.org> Cc: Subramoni, Hari <subramoni.1(a)osu.edu>; Purushotham Bangalore <pvbangalore(a)ua.edu>; Bronis R. de Supinski <bronis(a)llnl.gov>; Guillaume Mercier <guillaume.mercier(a)u-bordeaux.fr>; Raffenetti, Ken <raffenet(a)anl.gov>; Hugo Taboada <hugo.taboada(a)cea.fr>; Hines, Thomas; Schulz, Martin <martin.w.j.schulz(a)tum.de>; Garzaran, Maria <maria.garzaran(a)intel.com>; McMahon, Kevin; m.a.ruefenacht(a)gmail.com <m.a.ruefenacht(a)gmail.com>; Amanzholov, Muzakhir <msamanzhol42(a)tntech.edu>; Ryan Grant <regrant233(a)gmail.com>; Elis, Bengisu <bengisu.elis(a)tum.de>; Shipley, Riley Prescott <riley-shipley(a)utc.edu>; daniel.collis.holmes(a)gmail.com <daniel.collis.holmes(a)gmail.com>; de Supinski, Bronis R. <desupinski1(a)llnl.gov>; Christoph Niethammer <niethammer(a)hlrs.de>; Smith, Brian <smithbe(a)ornl.gov>; Thakur, Rajeev <thakur(a)anl.gov>; Wesley Bland <work(a)wesbland.com>; Schonbein, William Whitney <wwschon(a)sandia.gov>; Raffenetti, Kenneth J. <raffenet(a)mcs.anl.gov>; Suggs, Evan <evan-suggs(a)utc.edu>; jeff.science(a)gmail.com <jeff.science(a)gmail.com>; Patrick Bridges <patrickb(a)unm.edu>; Dosanjh, Matthew <mdosanj(a)sandia.gov>; Julien.JAEGER(a)CEA.FR <Julien.JAEGER(a)CEA.FR>; Ghafoor, Sheikh <SGhafoor(a)tntech.edu>; Holmes, Daniel John <daniel.john.holmes(a)intel.com>; Klemm, Michael <Michael.Klemm(a)amd.com> Subject: MPI Persistent/HACC/Partitioned WGs Weekly Meeting---ZOOM NOT TEAMS When: Wednesday, January 4, 2023 10:00 AM-11:00 AM. Where: NOTE: Zoom, not TEAMS, see below. As of May 12, 2021, WG materials being archived here https://drive.google.com/drive/folders/1dxWKDUxvzol5PQfMET2bwiiNs7rZfimX?us… Weekly: https://tennessee.zoom.us/meeting/tJIsfuqtrDsiE9S0ICfXsuy-47_0ZqWvG71z/ics?… Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/96193520890?pwd=T3k1NkFRQ3BJaUFkeXFQcmtYTlRHQT09 Password: 399650 ---- Hi there, Anthony Skjellum is inviting you to a scheduled Zoom meeting. Topic: Partition/HACC/etc Working Groups Time: Apr 6, 2022 10:00 AM Eastern Time (US and Canada) Every week on Wed, until Jan 25, 2023, 43 occurrence(s) Apr 6, 2022 10:00 AM Apr 13, 2022 10:00 AM Apr 20, 2022 10:00 AM Apr 27, 2022 10:00 AM May 4, 2022 10:00 AM May 11, 2022 10:00 AM May 18, 2022 10:00 AM May 25, 2022 10:00 AM Jun 1, 2022 10:00 AM Jun 8, 2022 10:00 AM Jun 15, 2022 10:00 AM Jun 22, 2022 10:00 AM Jun 29, 2022 10:00 AM Jul 6, 2022 10:00 AM Jul 13, 2022 10:00 AM Jul 20, 2022 10:00 AM Jul 27, 2022 10:00 AM Aug 3, 2022 10:00 AM Aug 10, 2022 10:00 AM Aug 17, 2022 10:00 AM Aug 24, 2022 10:00 AM Aug 31, 2022 10:00 AM Sep 7, 2022 10:00 AM Sep 14, 2022 10:00 AM Sep 21, 2022 10:00 AM Sep 28, 2022 10:00 AM Oct 5, 2022 10:00 AM Oct 12, 2022 10:00 AM Oct 19, 2022 10:00 AM Oct 26, 2022 10:00 AM Nov 2, 2022 10:00 AM Nov 9, 2022 10:00 AM Nov 16, 2022 10:00 AM Nov 23, 2022 10:00 AM Nov 30, 2022 10:00 AM Dec 7, 2022 10:00 AM Dec 14, 2022 10:00 AM Dec 21, 2022 10:00 AM Dec 28, 2022 10:00 AM Jan 4, 2023 10:00 AM Jan 11, 2023 10:00 AM Jan 18, 2023 10:00 AM Jan 25, 2023 10:00 AM Please download and import the following iCalendar (.ics) files to your calendar system. Weekly: https://tennessee.zoom.us/meeting/tJIsfuqtrDsiE9S0ICfXsuy-47_0ZqWvG71z/ics?… Join from PC, Mac, Linux, iOS or Android: https://tennessee.zoom.us/j/96193520890?pwd=T3k1NkFRQ3BJaUFkeXFQcmtYTlRHQT09 Password: 399650 Or iPhone one-tap (US Toll): +16468769923,96193520890# or +13017158592,96193520890# Or Telephone: Dial: +1 646 876 9923 (US Toll) +1 301 715 8592 (US Toll) +1 312 626 6799 (US Toll) +1 669 900 6833 (US Toll) +1 253 215 8782 (US Toll) +1 346 248 7799 (US Toll) Meeting ID: 961 9352 0890 International numbers available: https://tennessee.zoom.us/u/adIMYx6Jko Or an H.323/SIP room system: H.323: 162.255.37.11 (US West) or 162.255.36.11 (US East) Meeting ID: 961 9352 0890 Password: 399650 SIP: 96193520890(a)zoomcrc.com Password: 399650 <https://teams.microsoft.com/meetingOptions/?organizerId=e544ff3d-a271-4377-…> ________________________________________________________________________________ ________________________________________________________________________________ Microsoft Teams meeting Join on your computer, mobile app or room device Click here to join the meeting<https://teams.microsoft.com/l/meetup-join/19%3ameeting_ZTZlNjdhMGItZjM4Mi00…> Meeting ID: 251 308 468 346 Passcode: jqEhgw Download Teams<https://www.microsoft.com/en-us/microsoft-teams/download-app> | Join on the web<https://www.microsoft.com/microsoft-teams/join-a-meeting> Learn More<https://aka.ms/JoinTeamsMeeting> | Meeting options<https://teams.microsoft.com/meetingOptions/?organizerId=e544ff3d-a271-4377-…> ________________________________________________________________________________

1 0