On Sep 20, 2019, at 3:59 PM, Harms, Kevin <harms@alcf.anl.gov> wrote:

 hm... what does your Makefile look like? I think Brian built against gcc8 and not gcc7.4. You need to add this to the CXXFLAGS: --gcc-toolchain=/soft/compilers/gcc/8.2.0/linux-rhel7-x86_64

kevin

________________________________________
From: Peterka, Thomas <tpeterka@mcs.anl.gov>
Sent: Friday, September 20, 2019 3:51 PM
To: Harms, Kevin
Cc: Ecp-aurora-sdk-discuss@lists.jlse.anl.gov
Subject: Re: [Ecp-aurora-sdk-discuss] OMP target offload and C++ containers

Kevin,

I did see that in your slides and tried it, but I wasn’t able to recompile my code with the new module. I’m seeing errors like this:

[100%] Building CXX object examples/fixed/CMakeFiles/fixed.dir/fixed.cpp.o
In file included from /home/tpeterka/software/mfa/examples/fixed/fixed.cpp:10:
In file included from /home/tpeterka/software/mfa/include/mfa/mfa.hpp:31:
In file included from /home/bhomerding/install/sycl-usmapi/include/CL/sycl.hpp:11:
In file included from /home/bhomerding/install/sycl-usmapi/include/CL/sycl/accessor.hpp:13:
In file included from /home/bhomerding/install/sycl-usmapi/include/CL/sycl/buffer.hpp:10:
In file included from /home/bhomerding/install/sycl-usmapi/include/CL/sycl/detail/buffer_impl.hpp:19:
In file included from /home/bhomerding/install/sycl-usmapi/include/CL/sycl/detail/scheduler/scheduler.hpp:11:
In file included from /home/bhomerding/install/sycl-usmapi/include/CL/sycl/detail/cg.hpp:19:
In file included from /home/bhomerding/install/sycl-usmapi/include/CL/sycl/nd_item.hpp:16:
In file included from /home/bhomerding/install/sycl-usmapi/include/CL/sycl/intel/sub_group.hpp:16:
In file included from /home/bhomerding/install/sycl-usmapi/include/CL/sycl/types.hpp:55:
In file included from /soft/compilers/gcc/7.4.0/linux-rhel7-x86_64/include/c++/7.4.0/cfenv:41:
/soft/compilers/gcc/7.4.0/linux-rhel7-x86_64/include/c++/7.4.0/fenv.h:58:11: error: no member named 'fenv_t' in the global namespace; did you mean 'dev_t'?
 using ::fenv_t;
       ~~^

Maybe is something is incompatible with that module and my build setup, which worked before with the first three modules below.

 1) dpcpp/2019.6.30.011    2) parallel-studio-2019   3) iprof                  4) sycl-usmapi

Also, your example was for a malloc’ing a C-style array, but I didn’t see how to use that for a std::vector.

Anyway, I appreciate the suggestions. I can wait until the next SDK drop. How will I know when that happens?

Thanks again,
Tom
-----------------
Tom Peterka
Mathematics and Computer Science Division
Argonne National Laboratory
9700 S. Cass Ave.
Lemont, IL 60439
Bldg 240, Rm 2148
e-mail: tpeterka@mcs.anl.gov<mailto:tpeterka@mcs.anl.gov>
web: www.mcs.anl.gov/~tpeterka
phone: 630-252-7198

On Sep 20, 2019, at 3:13 PM, Harms, Kevin <harms@alcf.anl.gov<mailto:harms@alcf.anl.gov>> wrote:

Tom,

we expect the next SDK drop to have USM, but until then, we have a temporary install that can be used like this:

module use /home/bhomerding/install/modulefiles
module load sycl-usmapi

Only the CPU device is supported currently. You still need to run on the iris nodes to test.
There is a simple example here: https://github.com/kevin-harms/sycltrain/tree/master/usm

kevin


________________________________________
From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces@lists.jlse.anl.gov<mailto:ecp-aurora-sdk-discuss-bounces@lists.jlse.anl.gov>> on behalf of Peterka, Thomas via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss@lists.jlse.anl.gov<mailto:ecp-aurora-sdk-discuss@lists.jlse.anl.gov>>
Sent: Friday, September 20, 2019 2:53 PM
To: Hammond, Jeff R; Ecp-aurora-sdk-discuss@lists.jlse.anl.gov<mailto:Ecp-aurora-sdk-discuss@lists.jlse.anl.gov>; Romano, Paul K.
Subject: Re: [Ecp-aurora-sdk-discuss] OMP target offload and C++ containers

Jeff, Paul, others,

The link Jeff pointed to:

https://github.com/intel/llvm/blob/sycl/sycl/test/usm/allocator_vector.cpp

would be be a good workaround, except that the clang++ compiler reports that there is no member named ‘usm' in namespace 'cl::sycl'. Is this because USM is not implemented yet in the SDK, as was said in the workshop, or did I miss something?

I’ll reiterate what Paul said about the need to have STL  in the kernel. I wouldn’t be opposed to constructing STL objects outside the kernel on the host, but I do need to use them in the kernel, perhaps restricted to not reallocating their memory (eg,, not push_back or resize).

I’m looking at SYCL, not OpenMP. I thought this was the idea of SYCL—that I could write C++ kernels. C++ w/o STL would not work for me.

Any suggestions appreciated. This seems to be quite important to deciding what programming model to use for C++ code.

Thanks,
Tom
-----------------
Tom Peterka
Mathematics and Computer Science Division
Argonne National Laboratory
9700 S. Cass Ave.
Lemont, IL 60439
Bldg 240, Rm 2148
e-mail: tpeterka@mcs.anl.gov<mailto:tpeterka@mcs.anl.gov><mailto:tpeterka@mcs.anl.gov>
web: www.mcs.anl.gov/~tpeterka<http://www.mcs.anl.gov/~tpeterka>
phone: 630-252-7198

On Sep 19, 2019, at 11:57 AM, Hammond, Jeff R via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss@lists.jlse.anl.gov<mailto:ecp-aurora-sdk-discuss@lists.jlse.anl.gov><mailto:ecp-aurora-sdk-discuss@lists.jlse.anl.gov>> wrote:

In general, allocating memory on devices is a bad idea.  It can be implemented but only by pre-allocating a pool prior to kernel launch.

James Brodman and I were discussing polymorphic memory resources and USM yesterday and he pointed me tohttps://github.com/intel/llvm/blob/sycl/sycl/test/usm/allocator_vector.cpp.  That isn’t everything you want by any means (and it is SYCL rather than OpenMP) but maybe it’s a start.

Intel is actively working on supporting a subset of the C++ STL in device code, but the focus is SYCL applications.  I don’t know what docs we’ve shared so far but if you are at the workshop, find one of the Intel folks and ask them to go through https://software.intel.com/en-us/oneapi-dpcpp-library-guide with you.  This website should be public in the near future, but perhaps there is an offline HTML version in the SDK already.  If not, hopefully Carlos et al. can help.

Jeff

From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces@lists.jlse.anl.gov<mailto:ecp-aurora-sdk-discuss-bounces@lists.jlse.anl.gov><mailto:ecp-aurora-sdk-discuss-bounces@lists.jlse.anl.gov>> on behalf of "Romano, Paul K. via Ecp-aurora-sdk-discuss" <ecp-aurora-sdk-discuss@lists.jlse.anl.gov<mailto:ecp-aurora-sdk-discuss@lists.jlse.anl.gov><mailto:ecp-aurora-sdk-discuss@lists.jlse.anl.gov>>
Reply-To: "Romano, Paul K." <promano@anl.gov<mailto:promano@anl.gov><mailto:promano@anl.gov>>
Date: Thursday, September 19, 2019 at 9:48 AM
To: "Ecp-aurora-sdk-discuss@lists.jlse.anl.gov<mailto:Ecp-aurora-sdk-discuss@lists.jlse.anl.gov><mailto:Ecp-aurora-sdk-discuss@lists.jlse.anl.gov>" <Ecp-aurora-sdk-discuss@lists.jlse.anl.gov<mailto:Ecp-aurora-sdk-discuss@lists.jlse.anl.gov><mailto:Ecp-aurora-sdk-discuss@lists.jlse.anl.gov>>
Subject: [Ecp-aurora-sdk-discuss] OMP target offload and C++ containers

I work on an application that makes heavy use of C++ containers (vectors, unordered_map, etc.) in areas/data structures of the code (good example here<https://github.com/openmc-dev/openmc/blob/develop/include/openmc/nuclide.h#L28-L97>) that will need to be offloaded and have a few general questions:

- Is it / will it be possible to use these containers on Gen9 / future devices using OpenMP target offload?
- If so, are methods that potentially allocate new memory (vector::push_back) permissible? If permissible, should I still avoid them?
- Will unified shared memory eventually be able to help?

Any suggestions for porting strategies for codes with complex data structures like this would be appreciated.

Thanks,
Paul
--
Ecp-aurora-sdk-discuss mailing list
Ecp-aurora-sdk-discuss@lists.jlse.anl.gov<mailto:Ecp-aurora-sdk-discuss@lists.jlse.anl.gov><mailto:Ecp-aurora-sdk-discuss@lists.jlse.anl.gov>
https://lists.jlse.anl.gov/mailman/listinfo/ecp-aurora-sdk-discuss