Re-ran stream in Quadrant mode and I was able to get 435 GB/s with 1 thread per core, and a very slightly better 440 GB/s with 2 threads per core. In SNC-4 mode I saw 311 GB/s with 1 thread and 442 GB/s with 2 threads. All2All hit 425 GB/s with 1 thread per core. A peak of around 440 GB/s is basically what we should be expecting to see for a Bin 3 part, so the MCDRAM seems to be capable of delivering the expected bandwidth. However, SNC-4 is an outlier in requiring 128 threads, and gives unexpected low (73 GB/s) bandwidth from one of the clusters (vs 112 GB/s from each of the other three). -Scott On 4/26/16 3:56 PM, Harms, Kevin wrote:
Meeting with Avinash and Intel - 2016/4/26
Discussion: - We indicated that we are able to get 420,000 MB/s from stream in Flat All2All and Quadrant modes since the recent BIOS update - Intel would like us to investigate the Quadrant model in Flat mode because they think we should get 440,000 MB/s - Intel examined our NUMA output from the system and believes that we are not running a newer firmware because the logical processes should be nicely ordered. - Intel said they only expect at 5-6% variance between NUMA clusters. - Vitali present his HACC results on KNL (which were all good and have improved since A0) - Irv from Intel joined to help debug our BIOS setup. - He indicated that our BIOS is provided by PCSD division which does there own packaging and he doesn't know what they have. - Intel development runs BIOS created by him. - After some discussion, we need someone from Intel to help us debug the issue - Avinash will work to assign someone to help us - It appears Intel development has their own development boards and do not use Adams Pass boards
Action Items: - Argonne: Run our stream test cases in Flat-Quadrant configuration and try to achieve 440,000 MB/s using 1 or 2 threads per core - Intel: Assign someone to help investigate the BIOS version/configuration on our box - Argonne/Intel - setup a meeting in two weeks to discuss progress. Scott will present his NEK results.
kevin
_______________________________________________ Intel-nda mailing list [email protected] https://lists.jlse.anl.gov/mailman/listinfo/intel-nda