Hello, A new project on the LCRC cluster has been requested. Please forward the information on to the LCRC Allocation sub-committee. Applicant's name: Naiyuan Chiang Applicant's institution: ANL Applicant's division: MCS Project Name: NEXTGENOPT Project title: Next generation optimization Associated funding: Office of Science Other Systems: Fusion Science: Advances in algorithms for stochastic optimization (SO) and uncertainty quantification (UQ) are critical for the efficient design and real-time optimization of next-generation national infrastructures. Anticipating and mitigating uncertainty of weather, demands, and contingencies in a more integrated environment is necessary to mitigate market volatility and prevent cascading failures that can ultimately lead to catastrophic shortages of supply. The integrated monitoring and optimization of infrastructures systems will generate huge amounts of data that cannot be possibly processed without exploiting underlying probabilistic structures (e.g., sparsity in spatio-temporal correlation patterns) and without exploiting the properties of the particular decision-making objective at hand (e.g., stochastic, PDEs). It is thus at the heart of our current research the philosophy that a structure-oriented and integrated approach to UQ/SO is necessary. Project description: Sponsored by a Department of Energy (DOE) Early Career Award, we are currently developing new algorithms for UQ/SO that have a holistic view of the entire data-modeling-optimization process. A particular example of such algorithms is a new clustering-based interior-point strategy for stochastic programs in which scenarios are compressed based on their influence on the first-stage decision, not on data. This has shown to lead to drastic compression rates (and solution times) of above 80%, which cannot be achieved with data compression alone (see Figure 1). The compression is done adaptively, at the linear algebra level, by looking at the different contributions of the scenarios on the Schur complement. We are also currently developing new interior-point algorithms for nonconvex structured optimization capable of dealing with complex multi-scale physics (typically involving large networks of partial differential equations) and stochastic components. These problems arise, for instance, from natural gas inventory (line-pack) management (see Figure 2). This requires of a redesign of existing algorithms and modeling environments capable of conveying problem structure down to the linear algebra kernels. In addition, hybrid linear algebra kernels need to be designed to exploit different structural facets. Problems with billions of variables are being targeted. Project URL: http://www.mcs.anl.gov/~vzavala/earlycareer.html Requested allocation: 1000000 Q1: 250000 Q2: 250000 Q3: 250000 Q4: 250000 Justification: This is the scalability of our optimization problem with up to 50 MPI processes on FUSION: 1MPI, walltime: 2:29:58 = 8998 sec 2MPI, walltime: 1:24:48 = 5088 sec 5MPI, walltime: 0:36:08 = 2168 sec 10MPI, walltime: 0:18:17 = 1097 sec 25MPI, walltime: 0:07:40 = 460 sec 50MPI, walltime: 0:04:07 = 247 sec The requester has used undetermined amount hours of their initial startup project. In addition to approving an initial amount, please specify a Category and Subcategory for this project. For a list of the current selection of approved categories, please see: https://wiki.lcrc.anl.gov/wiki/Processes/Categories Once the Allocation committee has approved the project, please go to the Project Management page to create it: https://accounts.lcrc.anl.gov/projects.php Thank You, The LCRC Accounts System