Re-ran stream in Quadrant mode and I was able to get 435 GB/s with 1 thread per core, and a very slightly better 440 GB/s with 2 threads per core.
In SNC-4 mode I saw 311 GB/s with 1 thread and 442 GB/s with 2 threads. All2All hit 425 GB/s with 1 thread per core.

A peak of around 440 GB/s is basically what we should be expecting to see for a Bin 3 part, so the MCDRAM seems to be capable of delivering
the expected bandwidth. However, SNC-4 is an outlier in requiring 128 threads, and gives unexpected low (73 GB/s) bandwidth from one of the
clusters (vs 112 GB/s from each of the other three).

-Scott

On 4/26/16 3:56 PM, Harms, Kevin wrote:
Meeting with Avinash and Intel - 2016/4/26

Discussion:
  - We indicated that we are able to get 420,000 MB/s from stream in Flat All2All and Quadrant modes since the recent BIOS update
  - Intel would like us to investigate the Quadrant model in Flat mode because they think we should get 440,000 MB/s
  - Intel examined our NUMA output from the system and believes that we are not running a newer firmware because the logical processes should be nicely ordered.
  - Intel said they only expect at 5-6% variance between NUMA clusters.
  - Vitali present his HACC results on KNL (which were all good and have improved since A0)
  - Irv from Intel joined to help debug our BIOS setup.
    - He indicated that our BIOS is provided by PCSD division which does there own packaging and he doesn't know what they have.
    - Intel development runs BIOS created by him.
    - After some discussion, we need someone from Intel to help us debug the issue
    - Avinash will work to assign someone to help us
  - It appears Intel development has their own development boards and do not use Adams Pass boards

Action Items:
  - Argonne: Run our stream test cases in Flat-Quadrant configuration and try to achieve 440,000 MB/s using 1 or 2 threads per core
  - Intel: Assign someone to help investigate the BIOS version/configuration on our box
  - Argonne/Intel - setup a meeting in two weeks to discuss progress. Scott will present his NEK results.


kevin

  



_______________________________________________
Intel-nda mailing list
Intel-nda@lists.jlse.anl.gov
https://lists.jlse.anl.gov/mailman/listinfo/intel-nda