On our KNL B0 I'm seeing some irregularities in the MCDRAM performance when running Stream in SNC-4 mode.
I'm running Stream with 16 threads all bound to either domain 0, 1, 2, or 3 and memory bound to just one of the
four MCDRAM domains (4,5,6,7). Below is a table of the results showing memory bandwidth in MB/sec, where
the thread domain used is specified on the left the MCDRAM domain on the top of the table.

Domain 4 never achieves more than 73 GB/s while the other domains (5,6,7) all reach 112 GB/s.





MCDRAM NUMA Domain



4 5 6 7

0 73,306 84,053 107,483 55,978
Thread Domain
1 64,837 112,380 55,521 77,765

2 63,575 67,508 111,569 77,957

3 65,506 102,203 76,131 111,617


numactl -H shows that nodes 0,1,2,3 are associated with the DDR and 16 cores each
while nodes 4,5,6,7 are MCDRAM nodes with no associated processors. However the
following nodes are closest:
    Node 4 -> Node 0
    Node 5 -> Node 1
    Node 6 -> Node 2
    Node 7 -> Node 3