In the past the committee provided an opportunity for the PIs to make revisions. Ray ----------------------------------------- Ray Bair Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751 On 7/22/14, 4:25 PM, "Balaji, Pavan" <[email protected]> wrote:

Thanks, Ray. I assume if we submit a proposal based on what we think is appropriate, and if the committee doesn’t agree they won’t simply reject the proposal and give us an opportunity to change it as appropriate? Can you clarify?

Assuming that’s true, we’d like to propose a single large project for FY2015. If the committee wants us to break it up into smaller projects, I’d at least like an opportunity to explain the connections between the projects to them. Even after that if they believe that smaller projects are the right thing to do, we’ll be happy to split them up.

Regards,

— Pavan

On Jul 22, 2014, at 4:19 PM, Bair, Raymond A. <[email protected]> wrote:

...

Pavan,

At this point we are talking about the FY2015 project proposals, and that seems consistent with your email.

In the past the Allocations Committee has asked people to break up multi-purpose projects as a condition for approval. Of course I cannot read their minds, so you will need to take the approach that seems best. Keep in mind that a person can be part of more than one project. While it may be convenient to separate people into projects according to which projects fund them, other approaches are possible.

Ray

----------------------------------------- Ray Bair Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751

On 7/22/14, 4:04 PM, "Balaji, Pavan" <[email protected]> wrote:

...

Hi Ray,

I chatted about this with a bunch of folks in radix. Given the close tie-ins between the projects in radix, several of the experiments span multiple projects. For example, almost all of Wesley Bland’s work is a cross between MPI and Franck’s fault tolerance work. Similarly, almost all of the work done by a few students in the group (Min Si and Aamer Halim) is a cross between MPI and Pete’s Argo project. Some projects that Franck’s postdocs are doing is a cross between his FT research and Pete’s Argo project. Splitting these into smaller projects might be more of a hassle than what I initially expected.

If we can write up a description of what the connections between the different pieces are, and have a more detailed explanation of the work going on, will that work?

Given the increase in our usage, for next year, we are planning to ask for 2M core-hours. I was told that we’ll need to give a short presentation explaining this request. I’ll be happy to do that as well, and can add clarify the connections during that presentation as well.

Please let us know what you think.

Regards,

— Pavan

On Jul 17, 2014, at 2:30 PM, Bair, Raymond A. <[email protected]> wrote:

...

Dear Pavan and Rob,

Looking at the project description and FY13 project report for "radix," it appears to be a collection of different projects in the Radix group. Usually, LCRC does not approve composite projects (allocations for a combination of several distinct projects). We may make an exception for small composite projects, and radix was relatively small until mid-FY13. The initial FY14 request was 496K hours. Since then additional requests have grown the project to 1,370K hours. That puts it into the category with the largest projects in LCRC. From this point forward you should break up the radix project into its major components and submit separate project requests for them.

Regards,

Ray

----------------------------------------- Ray Bair Computing, Environment, and Life Sciences Argonne National Laboratory and the University of Chicago TCS Building 240, Room 4122 9700 South Cass Avenue Argonne, IL 60439 email: rbair(at)anl.gov Phone: (630)252-5751

On 7/17/14, 2:12 PM, "[email protected]" <[email protected]> wrote:

...

Hello,

A change in allocation has been requested:

Requester: balaji (Pavan Balaji) Project: radix Title: Scalable Parallel System Software Description: This is a Computer Science request for 60,000 cpu-hours on Blues for development and evaluation of a family of tools and libraries with a direct benefit to all users of high-performance computing resources. The Radix group in Argonnne's Mathematics and Computer Science division conducts research and development on facilities to better enable efficient Parallel Programming, I/O, and Scientific Understanding.

The Blues system represents an attractive milestone for our software development: projects start out on our breadboard cluster and can further explore scalability in the presence of a high speed network on Blues before making the jump to BlueGene-scale parallelism.

Radix experiments often are concerned with latency. The infiniband network on Blues provides an excellent low-latency environment for message passing and storage management research while also providing ample bandwidth for I/O and message bandwidth testing.

For FY2014 our Messaging middleware experiments include the usual MPICH development. We will study the continued improvement of various MPI features including scalability and memory usage, performance tuning, high-level libraries on top of MPI, fault tolerance capabilities, one-sided communication, and collective operations. This allocation will also be used for studying MPI extensions including active messages, and compiler support for MPI refactoring.

The NWChem library will continue to improve the performance of an important scientific application that is widely used for science by Argonne researchers. Specifically, we will develop inspect-executor load-balancing techniques that will reduce network contention and improve the overall performance and scalability of an important family of methods (coupled-cluster methods). Furthermore, we will explore hybrid programming models (that is, adding threading to the existing GA/ARMCI/MPI model), particularly in the TCE module, which has been used extensively by researchers in Argonne MSD and CSE. All the aforementioned developments will be deployed on Blues in a production fashion and made available to all users. This allocation will enable Jeff Hammond to maintain the NWChem installations on Blues, as he has done for Fusion in the past.

We work closely with application scientists, and develop application tools under the 'radix' allocation. For one example, the KMI project defines a high-level standardized interface for computational biology applications relying on distributed search and match semantics for biological reads. We will study efficient data management and data movement techniques to allow scaling to large number of processes.

The GPFS file system on Blues will allow us to experiment with I/O strategies on a modest scale before evaluating them at leadership scale on Mira.

For FY 2014 our file system research extends into data storage approaches suitable for exascale. Our research into object storage protocols, including high performance access and replication, fits well on Blues. Our Exascale storage efforts will explore scalable algorithms for use in next generation HPC storage systems. The Blues allocation will be used to evaluate programming models, fault detection algorithms, synchronization primitives, and fault tolerance strategies for a prototype object storage system.

In FY 2014 we are continuing research into new high level I/O libraries. We anticipate Blues and Vesta will be our two main test platforms.

Radix projects also focus on deriving insight both about application behavior as well as scientific insight. 16-core nodes make efforts like in-situ visualization, where for example an application renders a frame of a movie, ever more feasible. Even without fancy graphics accelerators, the processing power on Blues nodes makes such analysis and visualization feasible, especially if the visualization can take advantage of multiple cores and multiple nodes. Naturally, the radix visualization tools have demonstrated scalability to Blues sizes and beyond on Blue Gene, so making full utilization of Blues should not be a concern.

As should be evident from the variety of planned experiments, the Radix group plans to make full use of Blues.

With Argonne employees and student collaborators, we expect around 20 members in FY2014.

We plan on about 0% of our jobs being single core jobs. It's crazy that you even have to ask!

While Blues may contain a modest number of compute nodes, the machine still represents an attractive platform for testing: very fast, low latency interconnect; 16 cores per node; The testing and experiments the Radix group can carry out on Blues ensures the tools the group develops today will remain relevant even as supercomputers grow. Quite a few radix-developed projects are part of the Blues software stack: CPU hours for this project yield improvements and benefits not just for the radix group, but for all users of Blues and indeed users of high-end computational resources worldwide.

Current: undetermined amount Justification:

Requested: 300000

A specific reason has been given: We are working on several enhancements for system software techniques, including MPI, FT, I/O and visualization. Several of the projects took more core hours than originally estimated, due to which we have run out of our original allocation.

This needs to be approved and the final allocation amount decided upon.

Thank You, The LCRC Accounts System _______________________________________________ allocations-admins mailing list [email protected] https://lists.lcrc.anl.gov/mailman/listinfo/allocations-admins

-- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji

-- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji