February 2020 - Ecp-aurora-sdk-discuss

Re: [Ecp-aurora-sdk-discuss] Building Pytorch on JLSE
by Christopher Knight 26 Feb '20

26 Feb '20

FYI, I just added back v1.0.3: /soft/compilers/ComputeCpp/ComputeCpp-CE-1.0.3-CentOS-x86_64 . chris > On Feb 26, 2020, at 2:06 PM, Rosales-fernandez, Carlos via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> wrote: > > Hi Hyunseung, > > It is possible that POC is only compatible with the 1.0.3 build. Let me reach out to the team that produced it and see how to best fix this problem. > > Carlos > > From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of ECP Aurora <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> > Reply-To: "Yoo, Hyunseung" <hsyoo(a)anl.gov> > Date: Wednesday, February 26, 2020 at 1:36 PM > To: ECP Aurora <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> > Subject: [Ecp-aurora-sdk-discuss] Building Pytorch on JLSE > > I am trying to build pytorch in JLSE, and learned there is a proof of concept version available, > > /soft/restricted/intel_dga/pytorchPOC/2019-10-21 > > There is very nice and neat instructions in INSTALL_IRIS_USER.txt, but seems like it needs to be updated. > > For example, build_gen9poc.sh sets, > > export COMPUTECPP_DIR=/soft/compilers/ComputeCpp/ComputeCpp-CE-1.0.3-CentOS-x86_64 > > which is no longer available. I found there are multiple updated versions; 1.1.6, 1.2.0, and 1.3.0 > > However, after I switched to /soft/compilers/ComputeCpp/current, (actually I tried all versions) > > I am stuck on python setup.py install step with an error as below, > > [ 7%] Built target mkldnn_sycl_sycl_stream_submit_cpu_primitive.cpp_20_ih > [ 7%] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wrappers.pb.cc.o > [ 7%] Built target mkldnn_sycl_sycl_stream_cpu_thunk.cpp_18_ih > [ 7%] Linking CXX static library ../../../lib/libprotobuf.a > [ 7%] Built target libprotobuf > make: *** [all] Error 2 > Failed to run 'bash ../tools/build_pytorch_libs.sh --use-nnpack --use-mkldnn --use-qnnpack caffe2’ > > Has anyone successfully built with newer version of ComputeCpp? > > Harry > -- > Ecp-aurora-sdk-discuss mailing list > Ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov > https://lists.jlse.anl.gov/mailman/listinfo/ecp-aurora-sdk-discuss

1 0

Re: [Ecp-aurora-sdk-discuss] Building Pytorch on JLSE
by Rosales-fernandez, Carlos 26 Feb '20

26 Feb '20

Hi Hyunseung, It is possible that POC is only compatible with the 1.0.3 build. Let me reach out to the team that produced it and see how to best fix this problem. Carlos From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of ECP Aurora <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Reply-To: "Yoo, Hyunseung" <hsyoo(a)anl.gov> Date: Wednesday, February 26, 2020 at 1:36 PM To: ECP Aurora <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Subject: [Ecp-aurora-sdk-discuss] Building Pytorch on JLSE I am trying to build pytorch in JLSE, and learned there is a proof of concept version available, /soft/restricted/intel_dga/pytorchPOC/2019-10-21 There is very nice and neat instructions in INSTALL_IRIS_USER.txt, but seems like it needs to be updated. For example, build_gen9poc.sh sets, export COMPUTECPP_DIR=/soft/compilers/ComputeCpp/ComputeCpp-CE-1.0.3-CentOS-x86_64 which is no longer available. I found there are multiple updated versions; 1.1.6, 1.2.0, and 1.3.0 However, after I switched to /soft/compilers/ComputeCpp/current, (actually I tried all versions) I am stuck on python setup.py install step with an error as below, [ 7%] Built target mkldnn_sycl_sycl_stream_submit_cpu_primitive.cpp_20_ih [ 7%] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wrappers.pb.cc.o [ 7%] Built target mkldnn_sycl_sycl_stream_cpu_thunk.cpp_18_ih [ 7%] Linking CXX static library ../../../lib/libprotobuf.a [ 7%] Built target libprotobuf make: *** [all] Error 2 Failed to run 'bash ../tools/build_pytorch_libs.sh --use-nnpack --use-mkldnn --use-qnnpack caffe2’ Has anyone successfully built with newer version of ComputeCpp? Harry

1 0

Building Pytorch on JLSE
by Yoo, Hyunseung 26 Feb '20

26 Feb '20

I am trying to build pytorch in JLSE, and learned there is a proof of concept version available, /soft/restricted/intel_dga/pytorchPOC/2019-10-21 There is very nice and neat instructions in INSTALL_IRIS_USER.txt, but seems like it needs to be updated. For example, build_gen9poc.sh sets, export COMPUTECPP_DIR=/soft/compilers/ComputeCpp/ComputeCpp-CE-1.0.3-CentOS-x86_64 which is no longer available. I found there are multiple updated versions; 1.1.6, 1.2.0, and 1.3.0 However, after I switched to /soft/compilers/ComputeCpp/current, (actually I tried all versions) I am stuck on python setup.py install step with an error as below, [ 7%] Built target mkldnn_sycl_sycl_stream_submit_cpu_primitive.cpp_20_ih [ 7%] Building CXX object third_party/protobuf/cmake/CMakeFiles/libprotobuf.dir/__/src/google/protobuf/wrappers.pb.cc.o [ 7%] Built target mkldnn_sycl_sycl_stream_cpu_thunk.cpp_18_ih [ 7%] Linking CXX static library ../../../lib/libprotobuf.a [ 7%] Built target libprotobuf make: *** [all] Error 2 Failed to run 'bash ../tools/build_pytorch_libs.sh --use-nnpack --use-mkldnn --use-qnnpack caffe2’ Has anyone successfully built with newer version of ComputeCpp? Harry

1 0

C/C++/Fortran compiler update in JLSE
by Rosales-fernandez, Carlos 25 Feb '20

25 Feb '20

Hi all, I have pushed a minor update to the OpemMP offload compilers in JLSE. You will find a new version 2020.02.24 that brings in bug fixes for the C and C++ compilers (Fortran build is identical to the one we deployed last week). There’s been multiple bugfixes and I do not have a detailed list at this point, but let me know if you need specifics and I will contact the compiler team to get them for you. This will now be the default “omp” compiler module, but all older versions remain in the system. Regards, Carlos

1 0

Re: [Ecp-aurora-sdk-discuss] difference between dpcpp in CNDA and intel_dga
by Williams, Timothy J. 25 Feb '20

25 Feb '20

I followed up on this with the systems people. There was an issue with ACL setup at play. We believe a permissions correction is in place now, so please try it again and confirm whether you now find the correct clang++ wrapper from PATH setup. Thanks, Tim ------- Timothy J. Williams, Ph.D. --- Argonne National Laboratory --- tjwilliams(a)anl.gov<mailto:[email protected]> ------- From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of Charles Leggett via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Reply-To: Charles Leggett <cgleggett(a)lbl.gov> Date: Tuesday, February 25, 2020 at 2:39 PM To: "Moore, Roy" <roy.moore(a)intel.com> Cc: "ecp-aurora-sdk-discuss(a)jlse.anl.gov" <ecp-aurora-sdk-discuss(a)jlse.anl.gov> Subject: Re: [Ecp-aurora-sdk-discuss] difference between dpcpp in CNDA and intel_dga Ahhh, OK. There's something very strange going on with the PATH resolution after setting up the intel_dga dpcpp module: > export PATH=/soft/restricted/intel_dga/modulefiles/dpcpp/2020.01.30.005_wrapper:/soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin:/bin:/usr/bin > which clang++ /soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin/clang++ > ls -l /soft/restricted/intel_dga/modulefiles/dpcpp/2020.01.30.005_wrapper total 8 4 -rwxr-x---. 1 rosales intel-dga 188 Jan 30 21:04 clang++* 4 -rwxr-x---. 1 rosales intel-dga 196 Jan 30 21:04 dpcpp* I don't seem to be a member of the intel-dga group (which is odd?): > groups cleggett cleggett : jlse ecp-aurora-sdk jlse-confluence aesp but I can still execute the wrapper script? > /soft/restricted/intel_dga/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ --version Intel(R) oneAPI DPC++ Compiler 2021.1-beta04 (2020.1.0.0122) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin However it's not picking up clang++ from the wrapper directory, even though that one's first in the PATH, but rather from the linux/bin one. So maybe a groups/permission issue? cheers, Charles. On Tue, Feb 25, 2020 at 12:01 PM Moore, Roy <roy.moore(a)intel.com<mailto:[email protected]>> wrote: Charles, /soft/restricted/CNDA/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ Is a wrapper script that adds a newer version of GCC for the toolchain to use, there is also a similar wrapper under intel_dga at /soft/restricted/intel_dga/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ Cheers, Roy -- Roy Moore IAGS / CPDP – Exascale Performance Codesign From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of Charles Leggett via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Reply-To: Charles Leggett <cgleggett(a)lbl.gov<mailto:[email protected]>> Date: Tuesday, February 25, 2020 at 1:40 PM To: "ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]>" <ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]>> Subject: [Ecp-aurora-sdk-discuss] difference between dpcpp in CNDA and intel_dga what is the difference between the versions of the dpcpp in the intel_dga and CNDA areas, ie between /soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin/clang++ and /soft/restricted/CNDA/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ I'm seeing different behavior when using them - the intel_dga one is picking stuff up from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/, whereas the CNDA is getting something more recent, so the intel_dga one is causing problems with c++17 code. cheers, Charles. -- Charles Leggett <CGLeggett(a)lbl.gov<mailto:[email protected]>> Lawrence Berkeley National Lab 1 Cyclotron Road, MS 50F-1650, Berkeley, CA 94720 tel: 510.495.2930 -- Charles Leggett <CGLeggett(a)lbl.gov<mailto:[email protected]>> Lawrence Berkeley National Lab 1 Cyclotron Road, MS 50F-1650, Berkeley, CA 94720 tel: 510.495.2930

1 0

Re: [Ecp-aurora-sdk-discuss] difference between dpcpp in CNDA and intel_dga
by Charles Leggett 25 Feb '20

25 Feb '20

Ahhh, OK. There's something very strange going on with the PATH resolution after setting up the intel_dga dpcpp module: > export PATH=/soft/restricted/intel_dga/modulefiles/dpcpp/2020.01.30.005_wrapper:/soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin:/bin:/usr/bin > which clang++ /soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin/clang++ > ls -l /soft/restricted/intel_dga/modulefiles/dpcpp/2020.01.30.005_wrapper total 8 4 -rwxr-x---. 1 rosales intel-dga 188 Jan 30 21:04 clang++* 4 -rwxr-x---. 1 rosales intel-dga 196 Jan 30 21:04 dpcpp* I don't seem to be a member of the intel-dga group (which is odd?): > groups cleggett cleggett : jlse ecp-aurora-sdk jlse-confluence aesp but I can still execute the wrapper script? > /soft/restricted/intel_dga/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ --version Intel(R) oneAPI DPC++ Compiler 2021.1-beta04 (2020.1.0.0122) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin However it's not picking up clang++ from the wrapper directory, even though that one's first in the PATH, but rather from the linux/bin one. So maybe a groups/permission issue? cheers, Charles. On Tue, Feb 25, 2020 at 12:01 PM Moore, Roy <roy.moore(a)intel.com> wrote: > Charles, > > > > /soft/restricted/CNDA/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ > > Is a wrapper script that adds a newer version of GCC for the toolchain to > use, there is also a similar wrapper under intel_dga at > > > > /soft/restricted/intel_dga/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ > > > > Cheers, > > > > Roy > > > > > > -- > > Roy Moore > > IAGS / CPDP – Exascale Performance Codesign > > > > > > *From: *Ecp-aurora-sdk-discuss < > ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of Charles > Leggett via Ecp-aurora-sdk-discuss < > ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> > *Reply-To: *Charles Leggett <cgleggett(a)lbl.gov> > *Date: *Tuesday, February 25, 2020 at 1:40 PM > *To: *"ecp-aurora-sdk-discuss(a)jlse.anl.gov" < > ecp-aurora-sdk-discuss(a)jlse.anl.gov> > *Subject: *[Ecp-aurora-sdk-discuss] difference between dpcpp in CNDA and > intel_dga > > > > > > what is the difference between the versions of the dpcpp in the intel_dga > and CNDA areas, ie between > > > > > /soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin/clang++ > > > > and > > > > /soft/restricted/CNDA/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ > > > > I'm seeing different behavior when using them - the intel_dga one is > picking stuff up from > /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/, > whereas the CNDA is getting something more recent, so the intel_dga one is > causing problems with c++17 code. > > > > cheers, Charles. > > -- > > Charles Leggett <CGLeggett(a)lbl.gov> > Lawrence Berkeley National Lab > 1 Cyclotron Road, MS 50F-1650, Berkeley, CA 94720 > > tel: 510.495.2930 > -- Charles Leggett <CGLeggett(a)lbl.gov> Lawrence Berkeley National Lab 1 Cyclotron Road, MS 50F-1650, Berkeley, CA 94720 tel: 510.495.2930

1 0

Re: [Ecp-aurora-sdk-discuss] difference between dpcpp in CNDA and intel_dga
by Moore, Roy 25 Feb '20

25 Feb '20

Charles, /soft/restricted/CNDA/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ Is a wrapper script that adds a newer version of GCC for the toolchain to use, there is also a similar wrapper under intel_dga at /soft/restricted/intel_dga/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ Cheers, Roy -- Roy Moore IAGS / CPDP – Exascale Performance Codesign From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of Charles Leggett via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Reply-To: Charles Leggett <cgleggett(a)lbl.gov> Date: Tuesday, February 25, 2020 at 1:40 PM To: "ecp-aurora-sdk-discuss(a)jlse.anl.gov" <ecp-aurora-sdk-discuss(a)jlse.anl.gov> Subject: [Ecp-aurora-sdk-discuss] difference between dpcpp in CNDA and intel_dga what is the difference between the versions of the dpcpp in the intel_dga and CNDA areas, ie between /soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin/clang++ and /soft/restricted/CNDA/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ I'm seeing different behavior when using them - the intel_dga one is picking stuff up from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/, whereas the CNDA is getting something more recent, so the intel_dga one is causing problems with c++17 code. cheers, Charles. -- Charles Leggett <CGLeggett(a)lbl.gov<mailto:[email protected]>> Lawrence Berkeley National Lab 1 Cyclotron Road, MS 50F-1650, Berkeley, CA 94720 tel: 510.495.2930

1 0

difference between dpcpp in CNDA and intel_dga
by Charles Leggett 25 Feb '20

25 Feb '20

what is the difference between the versions of the dpcpp in the intel_dga and CNDA areas, ie between /soft/restricted/intel_dga/sdk/2020.1.30.5/compiler/latest/linux/bin/clang++ and /soft/restricted/CNDA/modulefiles/dpcpp/2020.01.30.005_wrapper/clang++ I'm seeing different behavior when using them - the intel_dga one is picking stuff up from /usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../include/c++/4.8.5/, whereas the CNDA is getting something more recent, so the intel_dga one is causing problems with c++17 code. cheers, Charles. -- Charles Leggett <CGLeggett(a)lbl.gov> Lawrence Berkeley National Lab 1 Cyclotron Road, MS 50F-1650, Berkeley, CA 94720 tel: 510.495.2930

1 0

Re: [Ecp-aurora-sdk-discuss] problems with sycl::intel::experimental::printf
by Rosales-fernandez, Carlos 25 Feb '20

25 Feb '20

Charles, Quick reply here so the list is aware. I can reproduce your issue in JLSE. A single call works, but you have this inside a loop and it looks like after some number of iterations the printf numbers are incorrect – then it segfaults. We will follow this up with your reproducer and file an issue against the dpcpp compiler. Carlos From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov> on behalf of ECP Aurora <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov> Reply-To: Charles Leggett <cgleggett(a)lbl.gov> Date: Tuesday, February 25, 2020 at 9:46 AM To: "ecp-aurora-sdk-discuss(a)jlse.anl.gov" <ecp-aurora-sdk-discuss(a)jlse.anl.gov> Subject: [Ecp-aurora-sdk-discuss] problems with sycl::intel::experimental::printf I'm trying to print out some counters inside a SyCL kernel using the new sycl::intel::experimental::printf, but after a while it starts printing out blanks instead of the ints, then segfaults. I have seen this on both beta04 on ubuntu 18.04, and on the iris nodes with the latest sdk. I can point you to some example code that demonstrates this. cheers, Charles. -- Charles Leggett <CGLeggett(a)lbl.gov<mailto:[email protected]>> Lawrence Berkeley National Lab 1 Cyclotron Road, MS 50F-1650, Berkeley, CA 94720 tel: 510.495.2930

1 0

Re: [Ecp-aurora-sdk-discuss] problems with sycl::intel::experimental::printf
by Applencourt, Thomas 25 Feb '20

25 Feb '20

If you provide a reproducer, I can try to debug it. In my naive examples, stream are working. I try them on the IntelDevCloud and in JLSE. Regards, Thomas Thomas Applencourt Leadership Computing Facility Argonne National Laboratory Building 240 Office - 1.D.18 9700 S Cass Av., Lemont Il, 60439 (630) 252-0036 ________________________________ From: Charles Leggett <cgleggett(a)lbl.gov> Sent: Tuesday, February 25, 2020 10:05 To: Applencourt, Thomas <tapplencourt(a)anl.gov> Cc: Rosales-fernandez, Carlos via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov>; ecp-aurora-sdk-discuss(a)jlse.anl.gov <ecp-aurora-sdk-discuss(a)jlse.anl.gov> Subject: Re: [Ecp-aurora-sdk-discuss] problems with sycl::intel::experimental::printf I have never been able to make sycl::stream work - it segfault right away. cheers, Charles. On Tue, Feb 25, 2020 at 7:59 AM Applencourt, Thomas <tapplencourt(a)anl.gov<mailto:[email protected]>> wrote: I'm sure you know but as a workaround you can use the sycl::stream (https://github.com/alcf-perfengr/sycltrain/blob/master/9_sycl_of_hell/1_my_…) Thomas Applencourt Leadership Computing Facility Argonne National Laboratory Building 240 Office - 1.D.18 9700 S Cass Av., Lemont Il, 60439 (630) 252-0036 ________________________________ From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of Rosales-fernandez, Carlos via Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Sent: Tuesday, February 25, 2020 09:55 To: Charles Leggett <cgleggett(a)lbl.gov<mailto:[email protected]>>; ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]> <ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]>> Subject: Re: [Ecp-aurora-sdk-discuss] problems with sycl::intel::experimental::printf Charles, I don’t believe the printf still works in every case. I tried this for printing a constant and it works OK in JLSE. If you send a reproducer for your failed jobs so we can look into it in more details: #include <CL/sycl.hpp> using namespace sycl; int main() { // Creating SYCL queue queue Q; #ifdef __SYCL_DEVICE_ONLY__ #define CONSTANT __attribute__((opencl_constant)) #else #define CONSTANT #endif Q.submit([](handler &CGH) { CGH.single_task([=]() { static const CONSTANT char Format[] = "%f\n"; intel::experimental::printf(Format, 33.4f); }); }); return 0; } From: Ecp-aurora-sdk-discuss <ecp-aurora-sdk-discuss-bounces(a)lists.jlse.anl.gov<mailto:[email protected]>> on behalf of ECP Aurora <ecp-aurora-sdk-discuss(a)lists.jlse.anl.gov<mailto:[email protected]>> Reply-To: Charles Leggett <cgleggett(a)lbl.gov<mailto:[email protected]>> Date: Tuesday, February 25, 2020 at 9:46 AM To: "ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]>" <ecp-aurora-sdk-discuss(a)jlse.anl.gov<mailto:[email protected]>> Subject: [Ecp-aurora-sdk-discuss] problems with sycl::intel::experimental::printf I'm trying to print out some counters inside a SyCL kernel using the new sycl::intel::experimental::printf, but after a while it starts printing out blanks instead of the ints, then segfaults. I have seen this on both beta04 on ubuntu 18.04, and on the iris nodes with the latest sdk. I can point you to some example code that demonstrates this. cheers, Charles. -- Charles Leggett <CGLeggett(a)lbl.gov<mailto:[email protected]>> Lawrence Berkeley National Lab 1 Cyclotron Road, MS 50F-1650, Berkeley, CA 94720 tel: 510.495.2930 -- Charles Leggett <CGLeggett(a)lbl.gov<mailto:[email protected]>> Lawrence Berkeley National Lab 1 Cyclotron Road, MS 50F-1650, Berkeley, CA 94720 tel: 510.495.2930

1 0