On 11/01/2016 10:00 AM, Luke Van Roekel wrote:

Thanks Rob, no luck with the mpi_info object and our computing folks are not willing to set the necessary option. Do you know why this is specific to intel-mpi? Openmpi has no issue. Our HPC people thought the MPI_File_write_at_all always requires a file lock, but openmpi seems to be fine.

I don't think the collectives (routines ending in _all) will trigger a lock request. ROMIO makes a lock request to guard against false sharing when performing a data sieving write it's called data sieving but it's actually a read-modify-write and we need to protect against two processes doing partial updates of the same region. OpenMPI has a different MPI-IO implementation as of I think 1.10. I don't know what optimizations they implement or how they avoid false sharing. ==rob

On Tue, Nov 1, 2016 at 8:42 AM, Rob Latham <[email protected] <mailto:[email protected]>> wrote:

On 10/31/2016 11:47 PM, Luke Van Roekel wrote:

Hello,

I've been trying to compile and run a very simple mpi test on our cluster with intel-mpi and openmpi. The test program is below. When I run with openmpi everything is fine. When I run with intel-mpi, I receive the following error

This requires fcntl(2) to be implemented. As of 8/25/2011 it is not. Generic MPICH Message: File locking failed in ADIOI_Set_lock(fd 6,cmd F_SETLKW/7,type F_WRLCK/1,whence 0) with return value FFFFFFFF and errno 26.

- If the file system is NFS, you need to use NFS version 3, ensure that the lockd daemon is running on all the machines, and mount the directory with the 'noac' option (no attribute caching).

- If the file system is LUSTRE, ensure that the directory is mounted with the 'flock' option.

ADIOI_Set_lock:: Function not implemented

ADIOI_Set_lock:offset 0, length 4

your site administrator needs to enable fcntl locking with the 'flock' mount option .

You can try disabling data sieving: you would create an MPI_Info object and add the key "romio_ds_write" with the value "disable"

==rob

Any thoughts on how to proceed? The size/format of the file read in seems to make no difference.

Regards, Luke

#include <stdio.h>

#include <stdlib.h>

#include <mpi.h>

int main(int argc, char **argv) {

int buf, err;

MPI_File fh;

MPI_Status status;

MPI_Init(&argc, &argv);

if (argc != 2) {

printf("Usage: %s filename\n", argv[0]);

MPI_Finalize();

return 1;

}

err = MPI_File_open(MPI_COMM_WORLD, argv[1], MPI_MODE_CREATE |

MPI_MODE_RDWR, MPI_INFO_NULL, &fh);

if (err != MPI_SUCCESS) printf("Error: MPI_File_open()\n");

err = MPI_File_write_all(fh, &buf, 1, MPI_INT, &status);

if (err != MPI_SUCCESS) printf("Error: MPI_File_write_all()\n");

MPI_File_close(&fh);

MPI_Finalize();

return 0;

}

_______________________________________________ discuss mailing list [email protected] <mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss <https://lists.mpich.org/mailman/listinfo/discuss>

_______________________________________________ discuss mailing list [email protected] <mailto:[email protected]> To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss <https://lists.mpich.org/mailman/listinfo/discuss>

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss