Hello, FYI: I have confirmed with the memkind developers that the policy is broken at this moment and documentation is not reflecting the boundary behaviour of the hbw_malloc for out-of-memory conditions. Vitali On 05/03/2016 09:07 AM, Vitali A. Morozov wrote:

Thanks for reporting it, Marta - I have been distracted yesterday and failed sending a message on this issue.

We are facing with tho issues.

First of, we see a non-intuitive behaviour of the default policy HBW_POLICY_PREFERRED. As documented

"If insufficient memory is available from the high bandwidth NUMA node closest at allocation time, fall back to standard memory (default) with the smallest NUMA distance."

This brings two questions.

What happens if there is no HBW memory on the system? The expected answer - hbw_malloc (and its Fortran equivalent ATTRIBUTES FASTMEM) should behave equivalently to malloc.

What happens on KNL in cache mode, where the is no memory other then DDR memory? The expected answer - hbw_malloc (and its Fortran equivalent) should behave equivalently to malloc.

Marta and I have found that on both Xeon and KNL in cache mode both Fortran and C codes failed to allocate a small chunk of memory with hbw_malloc interface. We think this is a bug and it need to be reported and fixed.

We found a workaround. Additionally to the mentioned conditional compilation (which is not an appropriate way to fix it), the user may define

MEMKIND_HBW_NODE=0

This seems to direct all memory requests to numa node 0, and this exactly what hbw_malloc is expected to do if it runs out of memory on other numa nodes.

Secondly, we seems to fail setting a policy from Fortran (not that the policy works as follows from the first report, but still). In C, the function "hbw_set_policy" changes the policy and the following "hbw_get_policy" returns a new policy. In Fortran the equivalent

ierr = hbw_set_policy( %VAL(1) )

returns an error and the following

ierr = hbw_get_policy()

returns the default policy as if the previous function is ignored.

The correspondent codes are in JLSE servers in /home/morozov/KNL/HSCD-intel/test5 for those of you who want to verify the reports.

Best,

Vitali

On 05/02/2016 10:19 PM, Garcia Martinez, Marta wrote:

...

Hi,

Vitali and I are using FASTMEM in a subroutine and it works well in flat mode. However, trying to run the same test in cache mode is causing problems (issues with not enough memory) at runtime (not compilation).

!DIR$ ATTRIBUTES FASTMEM :: var

A turnaround for the moment is to enclose that between an if defined declaration like this:

!DEC$ IF DEFINED(name) !DIR$ ATTRIBUTES FASTMEM :: var !DEC$ ENDIF

and then compile with the variable ‘name'.

Marta

_______________________________________________ Intel-nda mailing list [email protected] https://lists.jlse.anl.gov/mailman/listinfo/intel-nda