Thanks for letting me know.  I'll try to remember this link line in case someone else asks about it in the future.

Best,

Jeff

From: "Banerjee,Tania" <tmishra@ufl.edu>
Date: Monday, February 20, 2017 at 4:59 PM
To: "Hammond, Jeff R" <jeff.r.hammond@intel.com>, "intel-nda@jlse.anl.gov" <intel-nda@jlse.anl.gov>
Subject: Re: [JLSE-Intel-NDA] ScaLAPACK on KNL?

Thanks Jeff. It worked for me when I dynamically linked the blacs library. So my compile line looked as:

mpiicc -mkl ex.c /soft/compilers/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64/libmkl_scalapack_lp64.a -Wl,-Bstatic -Wl,--start-group /soft/compilers/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64/libmkl_intel_lp64.a /soft/compilers/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64/libmkl_core.a /soft/compilers/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64/libmkl_intel_thread.a -Wl,--end-group -Wl,-Bdynamic  -o mult -lmkl_blacs_intelmpi_lp64

Best regards,

Tania

---

From: Hammond, Jeff R <jeff.r.hammond@intel.com>
Sent: Monday, February 20, 2017 7:01 PM
To: Banerjee,Tania; intel-nda@jlse.anl.gov
Subject: Re: [JLSE-Intel-NDA] ScaLAPACK on KNL?

You might post on the Intel Forum about this.  It's going to be at least two weeks before I have time to do any debugging on JLSE.

Jeff

From: "Banerjee,Tania" <tmishra@ufl.edu>
Date: Monday, February 20, 2017 at 3:59 PM
To: "Hammond, Jeff R" <jeff.r.hammond@intel.com>, "intel-nda@jlse.anl.gov" <intel-nda@jlse.anl.gov>
Subject: Re: [JLSE-Intel-NDA] ScaLAPACK on KNL?

Hi Jeff,

Thanks for the links. So, its the MPI wrapper that needs to be resolved. I tried to follow the option that talks about adding -Wl,-Bstatic/dynamic and currently have this:

MKL_STUFF = ${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a \

-Wl,-Bstatic -Wl,--start-group \

  ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a \

  ${MKLROOT}/lib/intel64/libmkl_core.a \

  ${MKLROOT}/lib/intel64/libmkl_intel_thread.a \

  ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_lp64.a \

-Wl,--end-group -Wl,-Bdynamic

Still I am not able to get the blacs_library* symbols below to appear with my executable:

21	00000000036a5f60 b blacs_library_loader_lock

22	00000000036a5f64 b blacs_library_lock_flag

Beyond that it still is confusing.

Thanks,

Tania

---

From: Hammond, Jeff R <jeff.r.hammond@intel.com>
Sent: Monday, February 20, 2017 6:25 PM
To: Banerjee,Tania; intel-nda@jlse.anl.gov
Subject: Re: [JLSE-Intel-NDA] ScaLAPACK on KNL?

I have not seen that one before but I have a good idea where it comes from. 

Have you looked at https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/590302 or https://software.intel.com/en-us/articles/using-intel-mkl-mpi-wrapper-with-the-intel-mkl-cluster-functions yet?

Jeff

From: "Banerjee,Tania" <tmishra@ufl.edu>
Date: Monday, February 20, 2017 at 3:19 PM
To: "Hammond, Jeff R" <jeff.r.hammond@intel.com>, "intel-nda@jlse.anl.gov" <intel-nda@jlse.anl.gov>
Subject: Re: [JLSE-Intel-NDA] ScaLAPACK on KNL?

Hi Jeff,

Thanks a lot for your help.

I am trying to figure it out, but currently I am getting an error:

Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.

when I run my program.

Thanks,

Tania

---

From: Hammond, Jeff R <jeff.r.hammond@intel.com>
Sent: Monday, February 20, 2017 5:35 PM
To: Banerjee,Tania; intel-nda@jlse.anl.gov
Subject: Re: [JLSE-Intel-NDA] ScaLAPACK on KNL?

Yes.  It is in $MKLROOT if you source the Intel toolchain script (e.g. /soft/compilers/intel/bin/compilervars.sh).

You can link single-threaded ScaLAPACK with -mkl=cluster or link it manually as follows (example is ScaLAPACK with threaded BLAS/LAPACK):

MKL_STUFF = ${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a \

-Wl,--start-group \

  ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a \

  ${MKLROOT}/lib/intel64/libmkl_core.a \

  ${MKLROOT}/lib/intel64/libmkl_intel_thread.a \

  ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_lp64.a \

-Wl,--end-group

FFTW_INCL = -I${MKLROOT}/include/fftw/

FFTW_LIBS = ${MKL_STUFF}

BLAS_LIBS = ${MKL_STUFF}

SCALAPACK_LIBS = ${MKL_STUFF}

Change lp64 to ilp64 as appropriate.

Jeff

From: <intel-nda-bounces@lists.jlse.anl.gov> on behalf of "Banerjee,Tania" <tmishra@ufl.edu>
Date: Monday, February 20, 2017 at 2:12 PM
To: "intel-nda@jlse.anl.gov" <intel-nda@jlse.anl.gov>
Subject: [JLSE-Intel-NDA] ScaLAPACK on KNL?

Hi,

Is MKL ScaLAPACK installed on JLSE for use on KNL?

Thanks,

Tania