- Allocations-admins - mailman3.cels.anl.gov

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Bair, Raymond A. 10 Jul '17

10 Jul '17

That is something to consider when installing the new bank software for Bebop. However, it would increase the number of allocation change requests, so that process would also need to be smoother. ----------------------------------------- Ray Bair Argonne National Laboratory On 7/10/17, 12:27 PM, "allocations-admins on behalf of Low, John J." <allocations-admins-bounces(a)lcrc.anl.gov on behalf of jlow(a)mcs.anl.gov> wrote: Ray, Blues looks oversubscribed because projects with large allocations use up their allocations early in the quarter and get additional allocations to keep their research going. I think we should consider adding to allocations and expiring unused allocations on a monthly rather quarterly basis. Users won’t have to wait long (less than a month) for more time, if their allocations are used up early in the month. John J. Low Principal Computational Science Specialist Computing, Environment and Life Sciences Building 240, 2143 9700 South Cass Avenue Argonne National Laboratory Argonne, IL 60439. 630-252-0045 www.linkedin.com/pub/john-low/15/8b0/5aa/ -----Original Message----- From: allocations-admins <allocations-admins-bounces(a)lcrc.anl.gov> on behalf of Halim Amer <aamer(a)anl.gov> Reply-To: LCRC Allocations Admins <allocations-admins(a)lcrc.anl.gov> Date: Monday, July 10, 2017 at 12:03 PM To: "Raymond A. Bair" <rbair(a)anl.gov>, "Balaji, Pavan" <balaji(a)anl.gov> Cc: LCRC Allocations Admins <allocations-admins(a)lcrc.anl.gov> Subject: Re: [allocations-admins] [LCRC Accounts] Project Allocation Request Hi Ray, Thank you for considering our request. We think 150k would get us going the upcoming month, so let's do as you suggested. Best, Halim www.mcs.anl.gov/~aamer On 7/10/17 11:40 AM, Bair, Raymond A. wrote: > Dear Pavan and Halim, > > I propose that we add enough time to carry the radix project for the > next 30 days. By that time Bebop should be available, and we can > provide more. Would 150K core-hours suffice? > > Regards, > > Ray > > > ----------------------------------------- > Ray Bair > Argonne National Laboratory > > > On 7/10/17, 11:33 AM, "allocations-admins on behalf of > accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on > behalf of accounts(a)lcrc.anl.gov> wrote: > > Hello, > > A change in allocation has been requested: > > Requester: aamer (Halim Amer) > Project: radix > Title: Scalable Parallel System Software > Description: This allocation will be used for optimizations to various > systems > software components: (1) programming models and runtime systems, (2) > data I/O and file systems, (3) fault tolerance, (4) operating systems, > and (5) data analysis and visualization. > > Primary goal of the project is to perform research on improving the > core systems software on high-end computing platforms. > > The radix allocation covers a large number of cross-cutting projects > involving subsets of researchers in the group. A few core projects > are presented here, but some of the techniques used are shared between > them and are not easily separable (e.g., MPI improvements or > resilience improvements): > > 1. Fault Tolerance Techniques: > > - Local memory management > > * Advanced SDC Detection through Data Monitoring > > + Adapt the prediction techniques used in the SZ lossy compressor > for SDC detection > > * Checkpoint size reduction to improve checkpoint/restart > performance and energy efficiency > > + Adapt FTI for using lossy compression of checkpoints > > * Impact of lossy compression on Checkpoint/restart > > + Investigate fundamentally and experimentally the relation > between discretization/truncation error with lossy compression > error > > * End-to-end Detection of SDCs in workflows featuring a lossy > compression stage > > + Compare combination of SDC detectors and lossy compressors with > respect to the accuracy of SDC detection. > > - Distributed memory management > > > * Integration of checkpointing in workflow environments > > + Adapt DECAF and/or SWIFT with FTI and perform performance > eveluation > > * MPI Fault Tolerance > > + Improvements to MPICH to deal with faults > > * Resilient-MPI and FTI Integration > > + Experimentation and evaluation of FTI coupled with > resilient-MPI > > 2. Lightweight, topology-aware, and thread-safe communication: > > - Improvements to MPI+threads > > * MPICH improvements to play well with threads, including improved > locking and lock-free data structures. > > * Communication progress improvements in the presence of threads. > > - Lightweight communication > > * Low instruction-count and low-overhead irregular data movement. > > - Topology-aware communication > > * Optimizations to MPI virtual topology functionality > > 3. Data I/O, RMA and Active Messages: > > - Dynamic Locality-Aware Workflows > > * Dynamically distribute the tasks among nodes using Asynchronous > Dynamic Load Balancer > > - Development and Evaluation of Composable Data Services > > * Exploration of lightweight, customized, domain-specific storage > services to improve the performance of data-intensive HPC > applications > > - High Performance Parallel I/O > > * The ROMIO MPI-IO implementation provides the foundation for many > components of the HPC I/O stack. Continue tuning, debugging, > and research efforts. > > 4. Data analysis and visualization: Testing and benchmarking of > improvements to data movement infrastructure such as DIY and Decaf > software to prepare for next-generation machine architectures and to > apply those improvements to applications across the lab that use those > software platforms, such as cosmology, climate science, and materials > science. The following areas will be tested on Fusion and Blues: > > - Relaxed BSP synchronization (e.g., for iterative algorithms, to > overlap communication with computation and support irregular > communication/computation patterns) > > * irregular communication patterns > > + short-circuit message completion > > - Load-balancing algorithms (work stealing, adaptive decompositions) > for unstructured time-varying mesh partitions (e.g., reactor > design) > > * work stealing algorithms > > + adaptive (e.g. k-d tree) domain decompositions > > - Integration with many-core thread models > > * Relaxed BSP communication with interchangeable thread models > > + Worker thread pool for executing block callback functions > > - Application domain decompositions > > * Add support for AMR (block-based and patch-based combustion and > astrophysics codes) > > + Add support for directly using existing simulation > decompositions (e.g., in climate codes) > Current: undetermined amount > Justification: 2M core hours requested: > > We have four core projects. All projects have been demonstrated at > scale in the previous years and form the core of the system software > stack used in production on the Fusion and Blues machines. > > We expect to run around ~40 studies for each project on Blues and on > Fusion, scaling from 1 node to 256 nodes (average runtime of 1 hour) > > - Blues: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x > 16 (cores) x 1 (hour) = 1.3M > > - Fusion: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x > 8 (cores) x 1 (hour) = 655K > > A small fraction of this would be used for debugging. > > Requested: 450000 > > A specific reason has been given: > Some of our projects consumed core hours at a higher rate than > anticipated. We also had a 300K core hours cut in the allocation we > anticipated. These factors contributed to this shortage we are > experiencing. Some of our important projects, such as MPI, critically > need this additional allocation. > > This needs to be approved and the final allocation amount decided upon. > > Thank You, > The LCRC Accounts System > _______________________________________________ > allocations-admins mailing list > allocations-admins(a)lcrc.anl.gov > https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins > > _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Roberts, John E. 10 Jul '17

10 Jul '17

I’ve granted 150K for them especially since we approved a small reservation for them after today’s maintenance. -- John Roberts Argonne National Laboratory CELS Systems jeroberts(a)anl.gov On 7/10/17, 12:03 PM, "allocations-admins on behalf of Halim Amer" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of aamer(a)anl.gov> wrote: Hi Ray, Thank you for considering our request. We think 150k would get us going the upcoming month, so let's do as you suggested. Best, Halim www.mcs.anl.gov/~aamer On 7/10/17 11:40 AM, Bair, Raymond A. wrote: > Dear Pavan and Halim, > > I propose that we add enough time to carry the radix project for the > next 30 days. By that time Bebop should be available, and we can > provide more. Would 150K core-hours suffice? > > Regards, > > Ray > > > ----------------------------------------- > Ray Bair > Argonne National Laboratory > > > On 7/10/17, 11:33 AM, "allocations-admins on behalf of > accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on > behalf of accounts(a)lcrc.anl.gov> wrote: > > Hello, > > A change in allocation has been requested: > > Requester: aamer (Halim Amer) > Project: radix > Title: Scalable Parallel System Software > Description: This allocation will be used for optimizations to various > systems > software components: (1) programming models and runtime systems, (2) > data I/O and file systems, (3) fault tolerance, (4) operating systems, > and (5) data analysis and visualization. > > Primary goal of the project is to perform research on improving the > core systems software on high-end computing platforms. > > The radix allocation covers a large number of cross-cutting projects > involving subsets of researchers in the group. A few core projects > are presented here, but some of the techniques used are shared between > them and are not easily separable (e.g., MPI improvements or > resilience improvements): > > 1. Fault Tolerance Techniques: > > - Local memory management > > * Advanced SDC Detection through Data Monitoring > > + Adapt the prediction techniques used in the SZ lossy compressor > for SDC detection > > * Checkpoint size reduction to improve checkpoint/restart > performance and energy efficiency > > + Adapt FTI for using lossy compression of checkpoints > > * Impact of lossy compression on Checkpoint/restart > > + Investigate fundamentally and experimentally the relation > between discretization/truncation error with lossy compression > error > > * End-to-end Detection of SDCs in workflows featuring a lossy > compression stage > > + Compare combination of SDC detectors and lossy compressors with > respect to the accuracy of SDC detection. > > - Distributed memory management > > > * Integration of checkpointing in workflow environments > > + Adapt DECAF and/or SWIFT with FTI and perform performance > eveluation > > * MPI Fault Tolerance > > + Improvements to MPICH to deal with faults > > * Resilient-MPI and FTI Integration > > + Experimentation and evaluation of FTI coupled with > resilient-MPI > > 2. Lightweight, topology-aware, and thread-safe communication: > > - Improvements to MPI+threads > > * MPICH improvements to play well with threads, including improved > locking and lock-free data structures. > > * Communication progress improvements in the presence of threads. > > - Lightweight communication > > * Low instruction-count and low-overhead irregular data movement. > > - Topology-aware communication > > * Optimizations to MPI virtual topology functionality > > 3. Data I/O, RMA and Active Messages: > > - Dynamic Locality-Aware Workflows > > * Dynamically distribute the tasks among nodes using Asynchronous > Dynamic Load Balancer > > - Development and Evaluation of Composable Data Services > > * Exploration of lightweight, customized, domain-specific storage > services to improve the performance of data-intensive HPC > applications > > - High Performance Parallel I/O > > * The ROMIO MPI-IO implementation provides the foundation for many > components of the HPC I/O stack. Continue tuning, debugging, > and research efforts. > > 4. Data analysis and visualization: Testing and benchmarking of > improvements to data movement infrastructure such as DIY and Decaf > software to prepare for next-generation machine architectures and to > apply those improvements to applications across the lab that use those > software platforms, such as cosmology, climate science, and materials > science. The following areas will be tested on Fusion and Blues: > > - Relaxed BSP synchronization (e.g., for iterative algorithms, to > overlap communication with computation and support irregular > communication/computation patterns) > > * irregular communication patterns > > + short-circuit message completion > > - Load-balancing algorithms (work stealing, adaptive decompositions) > for unstructured time-varying mesh partitions (e.g., reactor > design) > > * work stealing algorithms > > + adaptive (e.g. k-d tree) domain decompositions > > - Integration with many-core thread models > > * Relaxed BSP communication with interchangeable thread models > > + Worker thread pool for executing block callback functions > > - Application domain decompositions > > * Add support for AMR (block-based and patch-based combustion and > astrophysics codes) > > + Add support for directly using existing simulation > decompositions (e.g., in climate codes) > Current: undetermined amount > Justification: 2M core hours requested: > > We have four core projects. All projects have been demonstrated at > scale in the previous years and form the core of the system software > stack used in production on the Fusion and Blues machines. > > We expect to run around ~40 studies for each project on Blues and on > Fusion, scaling from 1 node to 256 nodes (average runtime of 1 hour) > > - Blues: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x > 16 (cores) x 1 (hour) = 1.3M > > - Fusion: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x > 8 (cores) x 1 (hour) = 655K > > A small fraction of this would be used for debugging. > > Requested: 450000 > > A specific reason has been given: > Some of our projects consumed core hours at a higher rate than > anticipated. We also had a 300K core hours cut in the allocation we > anticipated. These factors contributed to this shortage we are > experiencing. Some of our important projects, such as MPI, critically > need this additional allocation. > > This needs to be approved and the final allocation amount decided upon. > > Thank You, > The LCRC Accounts System > _______________________________________________ > allocations-admins mailing list > allocations-admins(a)lcrc.anl.gov > https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins > > _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Low, John J. 10 Jul '17

10 Jul '17

Ray, Blues looks oversubscribed because projects with large allocations use up their allocations early in the quarter and get additional allocations to keep their research going. I think we should consider adding to allocations and expiring unused allocations on a monthly rather quarterly basis. Users won’t have to wait long (less than a month) for more time, if their allocations are used up early in the month. John J. Low Principal Computational Science Specialist Computing, Environment and Life Sciences Building 240, 2143 9700 South Cass Avenue Argonne National Laboratory Argonne, IL 60439. 630-252-0045 www.linkedin.com/pub/john-low/15/8b0/5aa/ -----Original Message----- From: allocations-admins <allocations-admins-bounces(a)lcrc.anl.gov> on behalf of Halim Amer <aamer(a)anl.gov> Reply-To: LCRC Allocations Admins <allocations-admins(a)lcrc.anl.gov> Date: Monday, July 10, 2017 at 12:03 PM To: "Raymond A. Bair" <rbair(a)anl.gov>, "Balaji, Pavan" <balaji(a)anl.gov> Cc: LCRC Allocations Admins <allocations-admins(a)lcrc.anl.gov> Subject: Re: [allocations-admins] [LCRC Accounts] Project Allocation Request Hi Ray, Thank you for considering our request. We think 150k would get us going the upcoming month, so let's do as you suggested. Best, Halim www.mcs.anl.gov/~aamer On 7/10/17 11:40 AM, Bair, Raymond A. wrote: > Dear Pavan and Halim, > > I propose that we add enough time to carry the radix project for the > next 30 days. By that time Bebop should be available, and we can > provide more. Would 150K core-hours suffice? > > Regards, > > Ray > > > ----------------------------------------- > Ray Bair > Argonne National Laboratory > > > On 7/10/17, 11:33 AM, "allocations-admins on behalf of > accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on > behalf of accounts(a)lcrc.anl.gov> wrote: > > Hello, > > A change in allocation has been requested: > > Requester: aamer (Halim Amer) > Project: radix > Title: Scalable Parallel System Software > Description: This allocation will be used for optimizations to various > systems > software components: (1) programming models and runtime systems, (2) > data I/O and file systems, (3) fault tolerance, (4) operating systems, > and (5) data analysis and visualization. > > Primary goal of the project is to perform research on improving the > core systems software on high-end computing platforms. > > The radix allocation covers a large number of cross-cutting projects > involving subsets of researchers in the group. A few core projects > are presented here, but some of the techniques used are shared between > them and are not easily separable (e.g., MPI improvements or > resilience improvements): > > 1. Fault Tolerance Techniques: > > - Local memory management > > * Advanced SDC Detection through Data Monitoring > > + Adapt the prediction techniques used in the SZ lossy compressor > for SDC detection > > * Checkpoint size reduction to improve checkpoint/restart > performance and energy efficiency > > + Adapt FTI for using lossy compression of checkpoints > > * Impact of lossy compression on Checkpoint/restart > > + Investigate fundamentally and experimentally the relation > between discretization/truncation error with lossy compression > error > > * End-to-end Detection of SDCs in workflows featuring a lossy > compression stage > > + Compare combination of SDC detectors and lossy compressors with > respect to the accuracy of SDC detection. > > - Distributed memory management > > > * Integration of checkpointing in workflow environments > > + Adapt DECAF and/or SWIFT with FTI and perform performance > eveluation > > * MPI Fault Tolerance > > + Improvements to MPICH to deal with faults > > * Resilient-MPI and FTI Integration > > + Experimentation and evaluation of FTI coupled with > resilient-MPI > > 2. Lightweight, topology-aware, and thread-safe communication: > > - Improvements to MPI+threads > > * MPICH improvements to play well with threads, including improved > locking and lock-free data structures. > > * Communication progress improvements in the presence of threads. > > - Lightweight communication > > * Low instruction-count and low-overhead irregular data movement. > > - Topology-aware communication > > * Optimizations to MPI virtual topology functionality > > 3. Data I/O, RMA and Active Messages: > > - Dynamic Locality-Aware Workflows > > * Dynamically distribute the tasks among nodes using Asynchronous > Dynamic Load Balancer > > - Development and Evaluation of Composable Data Services > > * Exploration of lightweight, customized, domain-specific storage > services to improve the performance of data-intensive HPC > applications > > - High Performance Parallel I/O > > * The ROMIO MPI-IO implementation provides the foundation for many > components of the HPC I/O stack. Continue tuning, debugging, > and research efforts. > > 4. Data analysis and visualization: Testing and benchmarking of > improvements to data movement infrastructure such as DIY and Decaf > software to prepare for next-generation machine architectures and to > apply those improvements to applications across the lab that use those > software platforms, such as cosmology, climate science, and materials > science. The following areas will be tested on Fusion and Blues: > > - Relaxed BSP synchronization (e.g., for iterative algorithms, to > overlap communication with computation and support irregular > communication/computation patterns) > > * irregular communication patterns > > + short-circuit message completion > > - Load-balancing algorithms (work stealing, adaptive decompositions) > for unstructured time-varying mesh partitions (e.g., reactor > design) > > * work stealing algorithms > > + adaptive (e.g. k-d tree) domain decompositions > > - Integration with many-core thread models > > * Relaxed BSP communication with interchangeable thread models > > + Worker thread pool for executing block callback functions > > - Application domain decompositions > > * Add support for AMR (block-based and patch-based combustion and > astrophysics codes) > > + Add support for directly using existing simulation > decompositions (e.g., in climate codes) > Current: undetermined amount > Justification: 2M core hours requested: > > We have four core projects. All projects have been demonstrated at > scale in the previous years and form the core of the system software > stack used in production on the Fusion and Blues machines. > > We expect to run around ~40 studies for each project on Blues and on > Fusion, scaling from 1 node to 256 nodes (average runtime of 1 hour) > > - Blues: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x > 16 (cores) x 1 (hour) = 1.3M > > - Fusion: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x > 8 (cores) x 1 (hour) = 655K > > A small fraction of this would be used for debugging. > > Requested: 450000 > > A specific reason has been given: > Some of our projects consumed core hours at a higher rate than > anticipated. We also had a 300K core hours cut in the allocation we > anticipated. These factors contributed to this shortage we are > experiencing. Some of our important projects, such as MPI, critically > need this additional allocation. > > This needs to be approved and the final allocation amount decided upon. > > Thank You, > The LCRC Accounts System > _______________________________________________ > allocations-admins mailing list > allocations-admins(a)lcrc.anl.gov > https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins > > _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Halim Amer 10 Jul '17

10 Jul '17

Hi Ray, Thank you for considering our request. We think 150k would get us going the upcoming month, so let's do as you suggested. Best, Halim www.mcs.anl.gov/~aamer On 7/10/17 11:40 AM, Bair, Raymond A. wrote: > Dear Pavan and Halim, > > I propose that we add enough time to carry the radix project for the > next 30 days. By that time Bebop should be available, and we can > provide more. Would 150K core-hours suffice? > > Regards, > > Ray > > > ----------------------------------------- > Ray Bair > Argonne National Laboratory > > > On 7/10/17, 11:33 AM, "allocations-admins on behalf of > accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on > behalf of accounts(a)lcrc.anl.gov> wrote: > > Hello, > > A change in allocation has been requested: > > Requester: aamer (Halim Amer) > Project: radix > Title: Scalable Parallel System Software > Description: This allocation will be used for optimizations to various > systems > software components: (1) programming models and runtime systems, (2) > data I/O and file systems, (3) fault tolerance, (4) operating systems, > and (5) data analysis and visualization. > > Primary goal of the project is to perform research on improving the > core systems software on high-end computing platforms. > > The radix allocation covers a large number of cross-cutting projects > involving subsets of researchers in the group. A few core projects > are presented here, but some of the techniques used are shared between > them and are not easily separable (e.g., MPI improvements or > resilience improvements): > > 1. Fault Tolerance Techniques: > > - Local memory management > > * Advanced SDC Detection through Data Monitoring > > + Adapt the prediction techniques used in the SZ lossy compressor > for SDC detection > > * Checkpoint size reduction to improve checkpoint/restart > performance and energy efficiency > > + Adapt FTI for using lossy compression of checkpoints > > * Impact of lossy compression on Checkpoint/restart > > + Investigate fundamentally and experimentally the relation > between discretization/truncation error with lossy compression > error > > * End-to-end Detection of SDCs in workflows featuring a lossy > compression stage > > + Compare combination of SDC detectors and lossy compressors with > respect to the accuracy of SDC detection. > > - Distributed memory management > > > * Integration of checkpointing in workflow environments > > + Adapt DECAF and/or SWIFT with FTI and perform performance > eveluation > > * MPI Fault Tolerance > > + Improvements to MPICH to deal with faults > > * Resilient-MPI and FTI Integration > > + Experimentation and evaluation of FTI coupled with > resilient-MPI > > 2. Lightweight, topology-aware, and thread-safe communication: > > - Improvements to MPI+threads > > * MPICH improvements to play well with threads, including improved > locking and lock-free data structures. > > * Communication progress improvements in the presence of threads. > > - Lightweight communication > > * Low instruction-count and low-overhead irregular data movement. > > - Topology-aware communication > > * Optimizations to MPI virtual topology functionality > > 3. Data I/O, RMA and Active Messages: > > - Dynamic Locality-Aware Workflows > > * Dynamically distribute the tasks among nodes using Asynchronous > Dynamic Load Balancer > > - Development and Evaluation of Composable Data Services > > * Exploration of lightweight, customized, domain-specific storage > services to improve the performance of data-intensive HPC > applications > > - High Performance Parallel I/O > > * The ROMIO MPI-IO implementation provides the foundation for many > components of the HPC I/O stack. Continue tuning, debugging, > and research efforts. > > 4. Data analysis and visualization: Testing and benchmarking of > improvements to data movement infrastructure such as DIY and Decaf > software to prepare for next-generation machine architectures and to > apply those improvements to applications across the lab that use those > software platforms, such as cosmology, climate science, and materials > science. The following areas will be tested on Fusion and Blues: > > - Relaxed BSP synchronization (e.g., for iterative algorithms, to > overlap communication with computation and support irregular > communication/computation patterns) > > * irregular communication patterns > > + short-circuit message completion > > - Load-balancing algorithms (work stealing, adaptive decompositions) > for unstructured time-varying mesh partitions (e.g., reactor > design) > > * work stealing algorithms > > + adaptive (e.g. k-d tree) domain decompositions > > - Integration with many-core thread models > > * Relaxed BSP communication with interchangeable thread models > > + Worker thread pool for executing block callback functions > > - Application domain decompositions > > * Add support for AMR (block-based and patch-based combustion and > astrophysics codes) > > + Add support for directly using existing simulation > decompositions (e.g., in climate codes) > Current: undetermined amount > Justification: 2M core hours requested: > > We have four core projects. All projects have been demonstrated at > scale in the previous years and form the core of the system software > stack used in production on the Fusion and Blues machines. > > We expect to run around ~40 studies for each project on Blues and on > Fusion, scaling from 1 node to 256 nodes (average runtime of 1 hour) > > - Blues: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x > 16 (cores) x 1 (hour) = 1.3M > > - Fusion: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x > 8 (cores) x 1 (hour) = 655K > > A small fraction of this would be used for debugging. > > Requested: 450000 > > A specific reason has been given: > Some of our projects consumed core hours at a higher rate than > anticipated. We also had a 300K core hours cut in the allocation we > anticipated. These factors contributed to this shortage we are > experiencing. Some of our important projects, such as MPI, critically > need this additional allocation. > > This needs to be approved and the final allocation amount decided upon. > > Thank You, > The LCRC Accounts System > _______________________________________________ > allocations-admins mailing list > allocations-admins(a)lcrc.anl.gov > https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins > >

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Bair, Raymond A. 10 Jul '17

10 Jul '17

Dear Pavan and Halim, I propose that we add enough time to carry the radix project for the next 30 days. By that time Bebop should be available, and we can provide more. Would 150K core-hours suffice? Regards, Ray ----------------------------------------- Ray Bair Argonne National Laboratory On 7/10/17, 11:33 AM, "allocations-admins on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: Hello, A change in allocation has been requested: Requester: aamer (Halim Amer) Project: radix Title: Scalable Parallel System Software Description: This allocation will be used for optimizations to various systems software components: (1) programming models and runtime systems, (2) data I/O and file systems, (3) fault tolerance, (4) operating systems, and (5) data analysis and visualization. Primary goal of the project is to perform research on improving the core systems software on high-end computing platforms. The radix allocation covers a large number of cross-cutting projects involving subsets of researchers in the group. A few core projects are presented here, but some of the techniques used are shared between them and are not easily separable (e.g., MPI improvements or resilience improvements): 1. Fault Tolerance Techniques: - Local memory management * Advanced SDC Detection through Data Monitoring + Adapt the prediction techniques used in the SZ lossy compressor for SDC detection * Checkpoint size reduction to improve checkpoint/restart performance and energy efficiency + Adapt FTI for using lossy compression of checkpoints * Impact of lossy compression on Checkpoint/restart + Investigate fundamentally and experimentally the relation between discretization/truncation error with lossy compression error * End-to-end Detection of SDCs in workflows featuring a lossy compression stage + Compare combination of SDC detectors and lossy compressors with respect to the accuracy of SDC detection. - Distributed memory management * Integration of checkpointing in workflow environments + Adapt DECAF and/or SWIFT with FTI and perform performance eveluation * MPI Fault Tolerance + Improvements to MPICH to deal with faults * Resilient-MPI and FTI Integration + Experimentation and evaluation of FTI coupled with resilient-MPI 2. Lightweight, topology-aware, and thread-safe communication: - Improvements to MPI+threads * MPICH improvements to play well with threads, including improved locking and lock-free data structures. * Communication progress improvements in the presence of threads. - Lightweight communication * Low instruction-count and low-overhead irregular data movement. - Topology-aware communication * Optimizations to MPI virtual topology functionality 3. Data I/O, RMA and Active Messages: - Dynamic Locality-Aware Workflows * Dynamically distribute the tasks among nodes using Asynchronous Dynamic Load Balancer - Development and Evaluation of Composable Data Services * Exploration of lightweight, customized, domain-specific storage services to improve the performance of data-intensive HPC applications - High Performance Parallel I/O * The ROMIO MPI-IO implementation provides the foundation for many components of the HPC I/O stack. Continue tuning, debugging, and research efforts. 4. Data analysis and visualization: Testing and benchmarking of improvements to data movement infrastructure such as DIY and Decaf software to prepare for next-generation machine architectures and to apply those improvements to applications across the lab that use those software platforms, such as cosmology, climate science, and materials science. The following areas will be tested on Fusion and Blues: - Relaxed BSP synchronization (e.g., for iterative algorithms, to overlap communication with computation and support irregular communication/computation patterns) * irregular communication patterns + short-circuit message completion - Load-balancing algorithms (work stealing, adaptive decompositions) for unstructured time-varying mesh partitions (e.g., reactor design) * work stealing algorithms + adaptive (e.g. k-d tree) domain decompositions - Integration with many-core thread models * Relaxed BSP communication with interchangeable thread models + Worker thread pool for executing block callback functions - Application domain decompositions * Add support for AMR (block-based and patch-based combustion and astrophysics codes) + Add support for directly using existing simulation decompositions (e.g., in climate codes) Current: undetermined amount Justification: 2M core hours requested: We have four core projects. All projects have been demonstrated at scale in the previous years and form the core of the system software stack used in production on the Fusion and Blues machines. We expect to run around ~40 studies for each project on Blues and on Fusion, scaling from 1 node to 256 nodes (average runtime of 1 hour) - Blues: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x 16 (cores) x 1 (hour) = 1.3M - Fusion: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x 8 (cores) x 1 (hour) = 655K A small fraction of this would be used for debugging. Requested: 450000 A specific reason has been given: Some of our projects consumed core hours at a higher rate than anticipated. We also had a 300K core hours cut in the allocation we anticipated. These factors contributed to this shortage we are experiencing. Some of our important projects, such as MPI, critically need this additional allocation. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

[LCRC Accounts] Project Allocation Request
by accounts＠lcrc.anl.gov 10 Jul '17

10 Jul '17

Hello, A change in allocation has been requested: Requester: aamer (Halim Amer) Project: radix Title: Scalable Parallel System Software Description: This allocation will be used for optimizations to various systems software components: (1) programming models and runtime systems, (2) data I/O and file systems, (3) fault tolerance, (4) operating systems, and (5) data analysis and visualization. Primary goal of the project is to perform research on improving the core systems software on high-end computing platforms. The radix allocation covers a large number of cross-cutting projects involving subsets of researchers in the group. A few core projects are presented here, but some of the techniques used are shared between them and are not easily separable (e.g., MPI improvements or resilience improvements): 1. Fault Tolerance Techniques: - Local memory management * Advanced SDC Detection through Data Monitoring + Adapt the prediction techniques used in the SZ lossy compressor for SDC detection * Checkpoint size reduction to improve checkpoint/restart performance and energy efficiency + Adapt FTI for using lossy compression of checkpoints * Impact of lossy compression on Checkpoint/restart + Investigate fundamentally and experimentally the relation between discretization/truncation error with lossy compression error * End-to-end Detection of SDCs in workflows featuring a lossy compression stage + Compare combination of SDC detectors and lossy compressors with respect to the accuracy of SDC detection. - Distributed memory management * Integration of checkpointing in workflow environments + Adapt DECAF and/or SWIFT with FTI and perform performance eveluation * MPI Fault Tolerance + Improvements to MPICH to deal with faults * Resilient-MPI and FTI Integration + Experimentation and evaluation of FTI coupled with resilient-MPI 2. Lightweight, topology-aware, and thread-safe communication: - Improvements to MPI+threads * MPICH improvements to play well with threads, including improved locking and lock-free data structures. * Communication progress improvements in the presence of threads. - Lightweight communication * Low instruction-count and low-overhead irregular data movement. - Topology-aware communication * Optimizations to MPI virtual topology functionality 3. Data I/O, RMA and Active Messages: - Dynamic Locality-Aware Workflows * Dynamically distribute the tasks among nodes using Asynchronous Dynamic Load Balancer - Development and Evaluation of Composable Data Services * Exploration of lightweight, customized, domain-specific storage services to improve the performance of data-intensive HPC applications - High Performance Parallel I/O * The ROMIO MPI-IO implementation provides the foundation for many components of the HPC I/O stack. Continue tuning, debugging, and research efforts. 4. Data analysis and visualization: Testing and benchmarking of improvements to data movement infrastructure such as DIY and Decaf software to prepare for next-generation machine architectures and to apply those improvements to applications across the lab that use those software platforms, such as cosmology, climate science, and materials science. The following areas will be tested on Fusion and Blues: - Relaxed BSP synchronization (e.g., for iterative algorithms, to overlap communication with computation and support irregular communication/computation patterns) * irregular communication patterns + short-circuit message completion - Load-balancing algorithms (work stealing, adaptive decompositions) for unstructured time-varying mesh partitions (e.g., reactor design) * work stealing algorithms + adaptive (e.g. k-d tree) domain decompositions - Integration with many-core thread models * Relaxed BSP communication with interchangeable thread models + Worker thread pool for executing block callback functions - Application domain decompositions * Add support for AMR (block-based and patch-based combustion and astrophysics codes) + Add support for directly using existing simulation decompositions (e.g., in climate codes) Current: undetermined amount Justification: 2M core hours requested: We have four core projects. All projects have been demonstrated at scale in the previous years and form the core of the system software stack used in production on the Fusion and Blues machines. We expect to run around ~40 studies for each project on Blues and on Fusion, scaling from 1 node to 256 nodes (average runtime of 1 hour) - Blues: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x 16 (cores) x 1 (hour) = 1.3M - Fusion: 4 (projects) x 40 (studies) x 512 (1 to 256 nodes) x 8 (cores) x 1 (hour) = 655K A small fraction of this would be used for debugging. Requested: 450000 A specific reason has been given: Some of our projects consumed core hours at a higher rate than anticipated. We also had a 300K core hours cut in the allocation we anticipated. These factors contributed to this shortage we are experiencing. Some of our important projects, such as MPI, critically need this additional allocation. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Roberts, John E. 10 Jul '17

10 Jul '17

This time has been granted. Thanks. -- John Roberts Argonne National Laboratory CELS Systems jeroberts(a)anl.gov On 7/10/17, 11:05 AM, "allocations-admins on behalf of Bair, Raymond A." <allocations-admins-bounces(a)lcrc.anl.gov on behalf of rbair(a)anl.gov> wrote: I propose that we add 200K to project Materials_Genome_SCs Ray ----------------------------------------- Ray Bair Argonne National Laboratory On 7/10/17, 10:42 AM, "Sanchez Botana, Antia" <abotana(a)anl.gov> wrote: Ray, Thanks a lot for your response. I just did the calculation and 200k hours would be safe for me, I just was not sure how easy it was to get the additional allocation. The last runs I mentioned in the request (that I need to run on biggpu) are the ones that consume by far the most. Thanks again Antia ________________________________________ From: Bair, Raymond A. Sent: Monday, July 10, 2017 9:36 AM To: Sanchez Botana, Antia Cc: LCRC Allocations Admins Subject: Re: [allocations-admins] [LCRC Accounts] Project Allocation Request Dear Anita Sanchez Botana, We can add the requested 100K core-hours to your project, but will that be enough to finish your project (at least the FY2017 part, i.e., through Sept. 30)? If not, can you estimate the number of runs needed in FY2017 and the core-hours per run, to get a total? Regards, Ray ----------------------------------------- Ray Bair Chief Computational Scientist for Applications Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751 On 7/7/17, 11:18 AM, "allocations-admins on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: Hello, A change in allocation has been requested: Requester: botana (Antia Sanchez Botana) Project: Materials_Genome_SCs Title: Materials Genome Approach for Discovery of HTS Description: We intend to create a framework for designing novel superconductors combining first principles electronic-structure tools, materials databases, and evolutionary algorithms, capable of exploring large configurational spaces. We will continue applying this framework to transition metal nitrochlorides (TNCl (T= Ti, Zr, Hf)) and layered nickelates. We will use different doping mechanisms and atomic substitutions to design novel compositions. We will intensively use DFT codes such as VASP, WIEN2k and Quantum Espresso as well as crystal structure prediction methods that merge ab initio total energy calculations and a specific evolutionary algorithm like USPEX. Jobs requesting up to 10 nodes will be launched for a typical DFT-based calculation. Current: undetermined amount Justification: Requested: 100000 A specific reason has been given: I am currently running high priority calculations that need to be performed in the biggpu nodes given the scratch storage on the rest of the nodes of 15gb that is not enough for these (large files output of 50gb are produced). As a consequence, I am being charged twice and hence already consumed half of my allocation for the current quarter in only one week. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Bair, Raymond A. 10 Jul '17

10 Jul '17

I propose that we add 200K to project Materials_Genome_SCs Ray ----------------------------------------- Ray Bair Argonne National Laboratory On 7/10/17, 10:42 AM, "Sanchez Botana, Antia" <abotana(a)anl.gov> wrote: Ray, Thanks a lot for your response. I just did the calculation and 200k hours would be safe for me, I just was not sure how easy it was to get the additional allocation. The last runs I mentioned in the request (that I need to run on biggpu) are the ones that consume by far the most. Thanks again Antia ________________________________________ From: Bair, Raymond A. Sent: Monday, July 10, 2017 9:36 AM To: Sanchez Botana, Antia Cc: LCRC Allocations Admins Subject: Re: [allocations-admins] [LCRC Accounts] Project Allocation Request Dear Anita Sanchez Botana, We can add the requested 100K core-hours to your project, but will that be enough to finish your project (at least the FY2017 part, i.e., through Sept. 30)? If not, can you estimate the number of runs needed in FY2017 and the core-hours per run, to get a total? Regards, Ray ----------------------------------------- Ray Bair Chief Computational Scientist for Applications Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751 On 7/7/17, 11:18 AM, "allocations-admins on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: Hello, A change in allocation has been requested: Requester: botana (Antia Sanchez Botana) Project: Materials_Genome_SCs Title: Materials Genome Approach for Discovery of HTS Description: We intend to create a framework for designing novel superconductors combining first principles electronic-structure tools, materials databases, and evolutionary algorithms, capable of exploring large configurational spaces. We will continue applying this framework to transition metal nitrochlorides (TNCl (T= Ti, Zr, Hf)) and layered nickelates. We will use different doping mechanisms and atomic substitutions to design novel compositions. We will intensively use DFT codes such as VASP, WIEN2k and Quantum Espresso as well as crystal structure prediction methods that merge ab initio total energy calculations and a specific evolutionary algorithm like USPEX. Jobs requesting up to 10 nodes will be launched for a typical DFT-based calculation. Current: undetermined amount Justification: Requested: 100000 A specific reason has been given: I am currently running high priority calculations that need to be performed in the biggpu nodes given the scratch storage on the rest of the nodes of 15gb that is not enough for these (large files output of 50gb are produced). As a consequence, I am being charged twice and hence already consumed half of my allocation for the current quarter in only one week. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Sanchez Botana, Antia 10 Jul '17

10 Jul '17

Ray, Thanks a lot for your response. I just did the calculation and 200k hours would be safe for me, I just was not sure how easy it was to get the additional allocation. The last runs I mentioned in the request (that I need to run on biggpu) are the ones that consume by far the most. Thanks again Antia ________________________________ From: Bair, Raymond A. Sent: Monday, July 10, 2017 9:36 AM To: Sanchez Botana, Antia Cc: LCRC Allocations Admins Subject: Re: [allocations-admins] [LCRC Accounts] Project Allocation Request Dear Anita Sanchez Botana, We can add the requested 100K core-hours to your project, but will that be enough to finish your project (at least the FY2017 part, i.e., through Sept. 30)? If not, can you estimate the number of runs needed in FY2017 and the core-hours per run, to get a total? Regards, Ray ----------------------------------------- Ray Bair Chief Computational Scientist for Applications Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751 On 7/7/17, 11:18 AM, "allocations-admins on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: Hello, A change in allocation has been requested: Requester: botana (Antia Sanchez Botana) Project: Materials_Genome_SCs Title: Materials Genome Approach for Discovery of HTS Description: We intend to create a framework for designing novel superconductors combining first principles electronic-structure tools, materials databases, and evolutionary algorithms, capable of exploring large configurational spaces. We will continue applying this framework to transition metal nitrochlorides (TNCl (T= Ti, Zr, Hf)) and layered nickelates. We will use different doping mechanisms and atomic substitutions to design novel compositions. We will intensively use DFT codes such as VASP, WIEN2k and Quantum Espresso as well as crystal structure prediction methods that merge ab initio total energy calculations and a specific evolutionary algorithm like USPEX. Jobs requesting up to 10 nodes will be launched for a typical DFT-based calculation. Current: undetermined amount Justification: Requested: 100000 A specific reason has been given: I am currently running high priority calculations that need to be performed in the biggpu nodes given the scratch storage on the rest of the nodes of 15gb that is not enough for these (large files output of 50gb are produced). As a consequence, I am being charged twice and hence already consumed half of my allocation for the current quarter in only one week. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0

Re: [allocations-admins] [LCRC Accounts] Project Allocation Request
by Bair, Raymond A. 10 Jul '17

10 Jul '17

Dear Anita Sanchez Botana, We can add the requested 100K core-hours to your project, but will that be enough to finish your project (at least the FY2017 part, i.e., through Sept. 30)? If not, can you estimate the number of runs needed in FY2017 and the core-hours per run, to get a total? Regards, Ray ----------------------------------------- Ray Bair Chief Computational Scientist for Applications Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751 On 7/7/17, 11:18 AM, "allocations-admins on behalf of accounts(a)lcrc.anl.gov" <allocations-admins-bounces(a)lcrc.anl.gov on behalf of accounts(a)lcrc.anl.gov> wrote: Hello, A change in allocation has been requested: Requester: botana (Antia Sanchez Botana) Project: Materials_Genome_SCs Title: Materials Genome Approach for Discovery of HTS Description: We intend to create a framework for designing novel superconductors combining first principles electronic-structure tools, materials databases, and evolutionary algorithms, capable of exploring large configurational spaces. We will continue applying this framework to transition metal nitrochlorides (TNCl (T= Ti, Zr, Hf)) and layered nickelates. We will use different doping mechanisms and atomic substitutions to design novel compositions. We will intensively use DFT codes such as VASP, WIEN2k and Quantum Espresso as well as crystal structure prediction methods that merge ab initio total energy calculations and a specific evolutionary algorithm like USPEX. Jobs requesting up to 10 nodes will be launched for a typical DFT-based calculation. Current: undetermined amount Justification: Requested: 100000 A specific reason has been given: I am currently running high priority calculations that need to be performed in the biggpu nodes given the scratch storage on the rest of the nodes of 15gb that is not enough for these (large files output of 50gb are produced). As a consequence, I am being charged twice and hence already consumed half of my allocation for the current quarter in only one week. This needs to be approved and the final allocation amount decided upon. Thank You, The LCRC Accounts System _______________________________________________ allocations-admins mailing list allocations-admins(a)lcrc.anl.gov https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

1 0