[JLSE-Intel-NDA] KNL SNC-4 mode performance irregularity
On our KNL B0 I'm seeing some irregularities in the MCDRAM performance when running Stream in SNC-4 mode. I'm running Stream with 16 threads all bound to either domain 0, 1, 2, or 3 and memory bound to just one of the four MCDRAM domains (4,5,6,7). Below is a table of the results showing memory bandwidth in MB/sec, where the thread domain used is specified on the left the MCDRAM domain on the top of the table. Domain 4 never achieves more than 73 GB/s while the other domains (5,6,7) all reach 112 GB/s. MCDRAM NUMA Domain 4 5 6 7 0 73,306 84,053 107,483 55,978 Thread Domain 1 64,837 112,380 55,521 77,765 2 63,575 67,508 111,569 77,957 3 65,506 102,203 76,131 111,617 numactl -H shows that nodes 0,1,2,3 are associated with the DDR and 16 cores each while nodes 4,5,6,7 are MCDRAM nodes with no associated processors. However the following nodes are closest: Node 4 -> Node 0 Node 5 -> Node 1 Node 6 -> Node 2 Node 7 -> Node 3
participants (1)
-
Scott Parker