November 2020 - Ecp-aurora-sdk-discuss

new Kokkos module
by Steinbrecher, Patrick 30 Nov '20

30 Nov '20

Dear all, The Kokkos module on JLSE has been updated using the compiler gold release. Compilation and running is the same as before with beta10. To build your application, just do: module load oneapi module load kokkos Use these updated compiler flags: -std=c++17 -Wno-openmp-mapping -fiopenmp -fopenmp-targets=spir64="-mllvm -vpo-paropt-enable-64bit-opencl-atomics=true" -D__STRICT_ANSI__ And link with: -lkokkoscore -lkokkoscontainers and all .o files given by: /soft/restricted/CNDA/sdk/2020.11.30.1/oneapi/compiler/latest/linux/lib/libomp-*.o If you want to run on ATS you still have to use: module load oneapi-ats module load kokkos/beta10_ats Regards, Patrick

1 0

New Aurora SDK available in JLSE
by Chan-nui, Christopher 23 Nov '20

23 Nov '20

Hello, We are pleased to announce that a new Aurora SDK has been deployed in JLSE. This version 2020.11.30.001 includes multiple improvements and bug fixes. For additional details please refer to the latest release notes in the Wiki: https://wiki.jlse.anl.gov/display/inteldga/DGA+SDK+Release+Notes Public Documentation ==================== * ALCF oneAPI overview - https://www.alcf.anl.gov/support-center/aurora/oneapi-overview * Intel GPGPU Software Information - https://dgpu-docs.intel.com/ * Intel oneAPI Beta Documentation - https://software.intel.com/content/www/us/en/develop/tools/oneapi/documenta… Please let us know of any issues with the release or the environment settings. -- Christopher Chan-Nui christopher.chan-nui(a)intel.com

1 0

New Aurora SDK coming to JLSE today
by Chan-nui, Christopher 23 Nov '20

23 Nov '20

Hello, A new Aurora SDK will be installed today at 3:00 PM CST. Running jobs will not be affected. Default SDK modulefiles will be updated at that time. -- Christopher Chan-Nui christopher.chan-nui(a)intel.com

1 0

Re: [Ecp-aurora-sdk-discuss] jsrun?
by Kosinovsky, Greg 20 Nov '20

20 Nov '20

Thanks a lot, Victor. Greg -----Original Message----- From: Anisimov, Victor <vanisimov(a)anl.gov> Sent: Friday, November 20, 2020 7:55 AM To: Kosinovsky, Greg <kosinovsky1(a)llnl.gov> Cc: ecp-aurora-sdk-discuss(a)jlse.anl.gov Subject: Re: [Ecp-aurora-sdk-discuss] jsrun? Hi Greg, If you are using an interactive session and if you have compiled your application with mpi, you will need to use mpirun to launch the job on a compute node. For mpirun to be in the search path you will need to load the same mpi module that you used to compile your application. To see available mpi modules type "module use /soft/restricted/CNDA/modulefiles; module avail mpi". The version of mpi to be installed on Aurara is mpi/aurora_mpich/drop34. In general, to run batch jobs you will need to get familiar with Cobalt scheduler that Argonne uses on all machines. For information, see https://www.alcf.anl.gov/support-center/theta/running-jobs-and-submission-s… Please let us know if you have any questions. Thank you, Victor -- Victor Anisimov, PhD Application Performance Engineer Argonne Leadership Computing Facility 9700 S. Cass Ave, Bldg 240, Room 1121 Lemont, IL 60439 > On Nov 20, 2020, at 9:27 AM, Kosinovsky, Greg via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> wrote: > > Is jsrun or similar command available currently? I couldn’t find one either on the node or in the documentation. > Thank you, > Greg > -- > Ecp-aurora-sdk-discuss mailing list > Ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/ecp-aurora-sdk-discuss

1 0

Re: [Ecp-aurora-sdk-discuss] jsrun?
by Anisimov, Victor 20 Nov '20

20 Nov '20

Hi Greg, If you are using an interactive session and if you have compiled your application with mpi, you will need to use mpirun to launch the job on a compute node. For mpirun to be in the search path you will need to load the same mpi module that you used to compile your application. To see available mpi modules type "module use /soft/restricted/CNDA/modulefiles; module avail mpi". The version of mpi to be installed on Aurara is mpi/aurora_mpich/drop34. In general, to run batch jobs you will need to get familiar with Cobalt scheduler that Argonne uses on all machines. For information, see https://www.alcf.anl.gov/support-center/theta/running-jobs-and-submission-s… Please let us know if you have any questions. Thank you, Victor -- Victor Anisimov, PhD Application Performance Engineer Argonne Leadership Computing Facility 9700 S. Cass Ave, Bldg 240, Room 1121 Lemont, IL 60439 > On Nov 20, 2020, at 9:27 AM, Kosinovsky, Greg via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> wrote: > > Is jsrun or similar command available currently? I couldn’t find one either on the node or in the documentation. > Thank you, > Greg > -- > Ecp-aurora-sdk-discuss mailing list > Ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/ecp-aurora-sdk-discuss

1 0

jsrun?
by Kosinovsky, Greg 20 Nov '20

20 Nov '20

Is jsrun or similar command available currently? I couldn't find one either on the node or in the documentation. Thank you, Greg

1 0

Re: [Ecp-aurora-sdk-discuss] Spack Configuration/Usage Question
by Kosinovsky, Greg 18 Nov '20

18 Nov '20

Thank you, Servesh. Greg From: Muralidharan, Servesh <servesh(a)anl.gov> Sent: Wednesday, November 18, 2020 11:52 AM To: Kosinovsky, Greg <kosinovsky1(a)llnl.gov>; ecp-aurora-sdk-discuss(a)jlse.anl.gov Subject: Re: Spack Configuration/Usage Question Greg, The location /soft is destined for system wide install which is limited to admins. If you would like to install Spack packages yourself, you can do so by setting up a Spack tree configured for your home folder. To use JLSE supported packages or otherwise, you can do so via modules. Example: https://wiki.jlse.anl.gov/display/inteldga/A21+SDK+Software -- Servesh From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> Date: Wednesday, November 18, 2020 at 1:32 PM To: ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]> <ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]>> Subject: [Ecp-aurora-sdk-discuss] Spack Configuration/Usage Question Hi Everyone. I ran into problem trying to run spack yesterday. I added /soft/packaging/spack/bin to my Path, and I was seemingly able to run it. At least "spack list" and "spack list mpi" worked. But when I ran "spack install openmpi", I got the following error: [a603033443@jlselogin2 build]$ spack install openmpi ==> Error: Can't take write lock on read-only file: /soft/packaging/spack/builds/.spack-db/prefix_lock I contacted Todd Gamblin (who is the author of spack), and he said this error had to do with the way spack was configured in ANL. Can someone please let me know how I should properly be using spack not to run into this issue on jlselogin2? Thank you, Greg P.S. I heard that there is a Slack channel with good information on Aurora Early Adoption systems. Can someone please point me to it.

1 0

Re: [Ecp-aurora-sdk-discuss] Spack Configuration/Usage Question
by Willmore, Frank Ted 18 Nov '20

18 Nov '20

Hi Greg-- A few notes: * There is more than one spack installation available in JLSE, and these are oriented toward supporting others efforts: * /soft/spack is maintained by me, and is largely oriented toward integration. * /soft/packaging/spack is maintained by Servesh, and is more oriented toward performance. * Oneapi is now officially supported in spack, as of v0.16.0 (released today) and your best bet may be to clone your own spack installation. * There is documentation available in the JLSE wiki: * https://wiki.jlse.anl.gov/display/inteldga/Interfacing+OneAPI+compilers+to+… * https://wiki.jlse.anl.gov/pages/viewpage.action?pageId=46923969 for CI * Good luck! Cheers, --Frank Frank Willmore, Ph.D. ALCF Operations Argonne National Laboratory willmore(a)anl.gov<mailto:[email protected]> ________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of Kosinovsky, Greg via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Sent: Wednesday, November 18, 2020 1:32 PM To: ecp-aurora-sdk-discuss(a)jlse.anl.gov <ecp-aurora-sdk-discuss(a)jlse.anl.gov> Subject: [Ecp-aurora-sdk-discuss] Spack Configuration/Usage Question Hi Everyone. I ran into problem trying to run spack yesterday. I added /soft/packaging/spack/bin to my Path, and I was seemingly able to run it. At least “spack list” and “spack list mpi” worked. But when I ran “spack install openmpi”, I got the following error: [a603033443@jlselogin2 build]$ spack install openmpi ==> Error: Can't take write lock on read-only file: /soft/packaging/spack/builds/.spack-db/prefix_lock I contacted Todd Gamblin (who is the author of spack), and he said this error had to do with the way spack was configured in ANL. Can someone please let me know how I should properly be using spack not to run into this issue on jlselogin2? Thank you, Greg P.S. I heard that there is a Slack channel with good information on Aurora Early Adoption systems. Can someone please point me to it.

1 0

Re: [Ecp-aurora-sdk-discuss] Spack Configuration/Usage Question
by Muralidharan, Servesh 18 Nov '20

18 Nov '20

Greg, The location /soft is destined for system wide install which is limited to admins. If you would like to install Spack packages yourself, you can do so by setting up a Spack tree configured for your home folder. To use JLSE supported packages or otherwise, you can do so via modules. Example: https://wiki.jlse.anl.gov/display/inteldga/A21+SDK+Software -- Servesh From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> Date: Wednesday, November 18, 2020 at 1:32 PM To: ecp-aurora-sdk-discuss(a)jlse.anl.gov <ecp-aurora-sdk-discuss(a)jlse.anl.gov> Subject: [Ecp-aurora-sdk-discuss] Spack Configuration/Usage Question Hi Everyone. I ran into problem trying to run spack yesterday. I added /soft/packaging/spack/bin to my Path, and I was seemingly able to run it. At least “spack list” and “spack list mpi” worked. But when I ran “spack install openmpi”, I got the following error: [a603033443@jlselogin2 build]$ spack install openmpi ==> Error: Can't take write lock on read-only file: /soft/packaging/spack/builds/.spack-db/prefix_lock I contacted Todd Gamblin (who is the author of spack), and he said this error had to do with the way spack was configured in ANL. Can someone please let me know how I should properly be using spack not to run into this issue on jlselogin2? Thank you, Greg P.S. I heard that there is a Slack channel with good information on Aurora Early Adoption systems. Can someone please point me to it.

1 0

Re: [Ecp-aurora-sdk-discuss] Spack Configuration/Usage Question
by Bryce Allen 18 Nov '20

18 Nov '20

My perspective as a long time spack user (but not an expert and not a JLSE admin): I don't think the spack install on JLSE is designed to work this way - it has some pre-installed packages that you can use, but you can't add packages to it because you don't have the correct permissions. The way I typically use spack is by installing the latest release branch from source in my home directory, following the guide: https://spack-tutorial.readthedocs.io/en/latest/tutorial_basics.html#basics… I think there may be a way to point a user-install spack install at a system installed spack, to avoid having to re-install the packages or manually define a packages.yaml pointing at them (see https://spack-tutorial.readthedocs.io/en/latest/tutorial_configuration.html…) but I have not explored this yet. I typically construct a packages.yaml, which is tedious but only needs to be done once per target machine. Hope that helps, Bryce On 11/18/20 2:32 PM, Kosinovsky, Greg via Ecp-aurora-sdk-discuss wrote: > > Hi Everyone. > > I ran into problem trying to run spack yesterday. I added > /soft/packaging/spack/bin to my Path, and I was seemingly able to run > it. At least “spack list” and “spack list mpi” worked. > > But when I ran “spack install openmpi”, I got the following error: > > [a603033443@jlselogin2 build]$ spack install openmpi > > ==> Error: Can't take write lock on read-only file: > /soft/packaging/spack/builds/.spack-db/prefix_lock > > I contacted Todd Gamblin (who is the author of spack), and he said > this error had to do with the way spack was configured in ANL. > > Can someone please let me know how I should properly be using spack > not to run into this issue on jlselogin2? > > Thank you, > > Greg > > P.S. I heard that there is a Slack channel with good information on > Aurora Early Adoption systems. Can someone please point me to it. > >

1 0