Hello,
A yearly allocation for the LCRC cluster has been requested with the
following updated information:
Submitter/PI: Daniel Duke
Project Name: XRayFuelSpray
Division: ES
Project title: Fuel Injection and Sprays Studied Using X-Ray Diagnostics
Associated funding: DOE-EERE
Other Systems: None
Science: At the Advanced Photon Source Sector 7-BM at Argonne National Laboratory, we are presently engaged in an ongoing investigation of sprays in high-pressure fuel injection systems using X-ray diagnostic techniques. This work is funded by DOE-EERE. An improved understanding of spray formation physics is essential to the development of more efficient engine technology.
As a part of this ongoing research, we are investigating the presence of cavitation phenomena in fuel injection systems and how these phenomena impact spray formation. Cavitation is a problem in fuel injection systems that operate under high pressure. At the Advanced Photon Source, we are undertaking a series of experiments to develop novel methodologies to measure cavitation inside fuel injectors and model nozzles using a range of x-ray techniques. In order to develop these novel experimental techniques, it is necessary to have an accurate and detailed prediction of the cavitation zones inside the model nozzle at precisely defined boundary conditions, to provide a common point of comparison for different experiments. To achieve this, numerical simulations of the fluid flow are necessary. The prediction of cavitation in a turbulent flow at high pressure requires the use of a high-resolution three-dimensional mesh with high levels of local refinement. As such, the computation
times for these highly resolved meshes are beyond the reasonable capability of a desktop computer. High-performance computing allows us to perform these simulations.
Cavitation simulations performed using OpenFOAM over the last two years under this project have been extremely useful in guiding our experimental plans. The data have also provided a useful validation tool for improved model development. Continued improvement of high-fidelity LES in simplified submerged nozzle geometries allows us to better understand the fundamentals of nozzle cavitation. Secondly, the cavitation simulations in realistic production fuel injector geometries assist in the interpretation of x-ray phase contrast imaging of in-situ injector cavitation.
In the following year, we plan to extend high-fidelity cavitation simulations by undertaking longer simulation times in order to capture lower-frequency hydrodynamic phenomena which we have observed in our experiments. Since most LES are only run until for a few milliseconds delta-T until a sufficiently steady solution is achieved (and statistics are suitably converged), low frequency oscillations (below 1 KHz) have not before been studied using simulations. Improvements in efficiency will allows us to pursue longer run-time simulations.
We are currently investigating the application of x-ray fluorescence measurements for cavitating nozzle flow. This technique has the advantage of allowing us to measure both liquid density and dissolved gas concentration simultaneously. Such data will be highly valuable in the development of non-condensable gas modeling in cavitating flows; this is a problem of great interest to the modeling community. The combination of high-resolution numerical simulations and x-ray measurement techniques will improve our understanding of cavitation in fuel injection systems, increasing the impact of our work.
Over the first quarter of the next year, the project will incorporate a new sub-task focusing on gaseous jets at high injection pressure conditions. CFD calculations of the entire injection event (including opening and closing transients) will be performed with the CFD code CONVERGE and the numerical results will be validated against x-ray radiography data. The nozzle internal flow will be also simulated in order to improve the accuracy of inlet boundary conditions. The main effort will be aimed at the characterization of the shock structure in the under-expanded region and its effect on jet propagation and mixing with air.
Over the next year, the X-ray Fuel Spray project will also expand to include more tomographic x-ray radiography, particularly of gasoline direct injection sprays. Moving from line-of-sight radiography to tomography requires tomographic reconstruction of the data created at APS. We intend to use Blues (and Fusion) to perform rapid turnaround high-resolution reconstructions using the TomoPy software developed at APS. We may consider the purchase of dedicated nodes for this purpose in future.
Project description: The cavitation modeling component of the project is being performed using the OpenFOAM framework. A number of state-of-the-art numerical solvers have been implemented in OpenFOAM; namely incompressible-liquid and fully compressible homogeneous relaxation cavitation models and more recently a fully compressible homogeneous relaxation cavitation model which includes non-condensable gas modeling (HRMFoam). These codes were developed by Schmidt et al at the University of Massachusetts-Amherst, and we continue to collaborate with them on code development and efficiency improvements.
In the last year, we have made some substantial improvements to the cavitation model and the parallel efficiency of its implementation on Blues/Fusion. With the support of LCRC technical staff and our collaborators at the University of Massachusetts-Amherst, we have implemented a custom build of OpenFOAM-3.0 on Blues against the system native MPI, and we have made the necessary changes to the HRMFoam solver’s source code in order to move from the older OpenFOAM-1.6 to the new 3.0 framework. This has led to substantial gains in parallel performance; up to 20% on 32 cores and 12% on 64 cores on Blues. A scalability study has been performed with the new code, and details are given in the Large Allocation Efficiency statement.
In the new OpenFOAM-3.0 build, we have successfully implemented both Large Eddy Simulation (LES) and Realizable k-epsilon (KE) turbulence models. Furthermore, we have implemented code which allows us to take virtual projections of the three-dimensional simulations so that they may be quantitatively compared with the x-ray experiments.
In the following year, we plan to extend high-fidelity cavitation simulations by undertaking longer simulation times in order to capture lower-frequency hydrodynamic phenomena which we have observed in our experiments. Improvements in efficiency will allows us to pursue longer run-time simulations. We also intend to simulate the effects of varying pressure and flow rate on cavitation, in order to compare against new x-ray fluorescence data from APS.
The gas jet simulations which we plan to undertake in Q1 are performed with the CONVERGE CFD software, its use is already well established at LCRC.
Tomographic reconstruction of gasoline direct injection spray measurements from APS will be conducted using the TomoPy software package developed at APS. We intend to run single-node jobs to efficiently perform multiple reconstructions of large (multi-GB) datasets. We have also successfully compiled a Paraview server on our Fusion dedicated nodes which allows us to rapidly visualize the results of our simulations and reconstructions. Using Blues and Fusion will allow us to achieve rapid turnaround of results during beamtime. This will be a significant aid to the experimental program. As the quantity of tomography work increases, we may consider purchasing more dedicated nodes.
Industry partnership: not applicable
Project URL:
Current FY Hours Used: undetermined amount
New FY Requested allocation: 600000
Q1: 195000
Q2: 135000
Q3: 135000
Q4: 135000
Justification: Our requested allocation for next year will exceed 0.5M core-hours in order to encompass the gas jet simulations in Q1 and tomographic reconstruction work in Q2-4.
The majority of core hours will still be dedicated to OpenFOAM simulations, which will be undertaken on both dedicated Fusion nodes and also on Blues, using OpenFOAM-3.0. We have demonstrated notable improvements in scalability on Blues as detailed above.
A scalability study of OpenFOAM-3.0 performance with multi-million cell meshes has been undertaken, demonstrating excellent linear scalability. We observe 81% efficiency relative to single-CPU performance, scaling linearly up to 320 cores (20 nodes) with a 15-million cell mesh (a typical upper limit). We have implemented Scotch decomposition in order to ensure good load balancing & minimization of cross-processor communication boundaries. The present OpenFOAM simulations use static meshes, so load rebalancing at runtime is not a concern. We intend to run typical job sizes of 64 to 128 cores; smaller than the maximum job size for which good scalability has been demonstrated. Further details regarding scalability studies can be obtained by contacting the project PIs.
Storage requirements: At the present time, we do not expect to exceed our project storage space allocation. Regarding data storage for tomographic reconstruction, the project space will only be used for temporary storage. Data will be moved to our own file servers once reconstructions are completed.
Thank You,
The LCRC Accounts System