May 2015 - commits - mailman3.cels.anl.gov

[mpich] MPICH primary repository branch, master, updated. v3.2b2-35-g13b72b4
by noreply＠mpich.org 19 May '15

19 May '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 13b72b4d12c61b201a8349c18638e3481668cd7f (commit) from beda01ce47143311c03aeb61791b12d4d6e6f999 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/13b72b4d12c61b201a8349c18638e3481… commit 13b72b4d12c61b201a8349c18638e3481668cd7f Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> Date: Sun May 17 21:36:56 2015 -0500 dev/sock: fix build error with usleep Defines usleep when needed. This build error was discovered via the random config tests in jenkins. Signed-off-by: Rob Latham <robl(a)mcs.anl.gov> diff --git a/src/util/wrappers/mpiu_process_wrappers.h b/src/util/wrappers/mpiu_process_wrappers.h index e571279..7675c1b 100644 --- a/src/util/wrappers/mpiu_process_wrappers.h +++ b/src/util/wrappers/mpiu_process_wrappers.h @@ -35,6 +35,9 @@ #elif defined (USE_USLEEP_FOR_YIELD) #ifdef HAVE_UNISTD_H #include <unistd.h> + #if defined (NEEDS_USLEEP_DECL) + int usleep(useconds_t usec); + #endif #endif #define MPIU_PW_Sched_yield() usleep(0) #elif defined (USE_SLEEP_FOR_YIELD) ----------------------------------------------------------------------- Summary of changes: src/util/wrappers/mpiu_process_wrappers.h | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b2-34-gbeda01c
by noreply＠mpich.org 16 May '15

16 May '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via beda01ce47143311c03aeb61791b12d4d6e6f999 (commit) via 8395e55f093b02c783a0e71cdb28421e76dc399f (commit) from 9ff9a4e6f2cca18eba33bb1e384d5c71d8b66303 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/beda01ce47143311c03aeb61791b12d4d… commit beda01ce47143311c03aeb61791b12d4d6e6f999 Author: Pavan Balaji <balaji(a)anl.gov> Date: Wed May 13 05:14:33 2015 -0500 remove the scif netmod. Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/src/mpi/errhan/errnames.txt b/src/mpi/errhan/errnames.txt index eff8eaa..694cc87 100644 --- a/src/mpi/errhan/errnames.txt +++ b/src/mpi/errhan/errnames.txt @@ -1587,42 +1587,6 @@ is too big (> MPIU_SHMW_GHND_SZ) **mpi_status_set_elements_x: MPI_Status_set_elements_x failed **mpi_status_set_elements_x %p %D %c: MPI_Status_set_elements_x(status=%p, datatype=%D, count=%c) -# SCIF netmod functions - -**scif_open:scif_open failed -**scif_open %s:scif_open failed (scif_open failed with error '%s') -**scif_bind:scif_bind failed -**scif_bind %s:scif_bind failed (scif_bind failed with error '%s') -**scif_listen:scif_listen failed -**scif_listen %s:scif_listen failed (scif_listen failed with error '%s') -**scif_connect:scif_connect failed -**scif_connect %s:scif_connect failed (scif_connect failed with error '%s') -**scif_accept:scif_accept failed -**scif_accept %s:scif_accept failed (scif_accept failed with error '%s') -**scif_send:scif_send failed -**scif_send %s:scif_send failed (scif_send failed with error '%s') -**scif_recv:scif_recv failed -**scif_recv %s:scif_recv failed (scif_recv failed with error '%s') -**scif_get_nodeIDs:scif_get_nodeIDs failed -**scif_get_nodeIDs %s %d:scif_get_nodeIDs failed (scif_get_nodeIDs failed with error '%s', error number = %d) -**scif_init_shmsend:failed to init shared memory for send -**scif_init_shmsend %s:failed to init shared memory for send (scif_init_shmsend failed with error '%s') -**scif_init_shmrecv:failed to init shared memory for recv -**scif_init_shmrecv %s:failed to init shared memory for recv (scif_init_shmrecv failed with error '%s') -**argstr_missingnode:failed to get node from business card -**scif_writev:scif_writev failed -**scif_writev %s:scif_writev failed (scif_writev failed with error '%s') -**scif_read:scif_read failed -**scif_scif_read %s:scif_scif_read failed (scif_scif_read failed with error '%s') -**scif_readv:scif_readv failed -**scif_scif_readv %s:scif_scif_readv (scif_scif_readv failed with error '%s') -**scif_poll_send:scif_poll_send failed -**scif_poll_send %s:scif_poll_send failed (scif_poll_send failed with error '%s') -**poll error:poll error -**loadsendiov:sendiov failed -**MPIDI_PG_Get_vc:MPIDI_PG_Get_vc failed -**MPIDI_PG_Get_vc %s:MPIDI_PG_Get_vc failed (MPIDI_PG_Get_vc failed with error '%s') - # Datarep conversion function not supported by ROMIO (Temproary until implemented) **drconvnotsupported:Read and Write datarep conversions are currently not supported by MPI-IO diff --git a/src/mpid/ch3/channels/nemesis/netmod/Makefile.mk b/src/mpid/ch3/channels/nemesis/netmod/Makefile.mk index 1fc79f6..3aed298 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/Makefile.mk +++ b/src/mpid/ch3/channels/nemesis/netmod/Makefile.mk @@ -8,7 +8,6 @@ include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/tcp/Makefile.mk include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/none/Makefile.mk -include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/scif/Makefile.mk include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/portals4/Makefile.mk include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/ib/Makefile.mk include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/mxm/Makefile.mk diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/Makefile.mk b/src/mpid/ch3/channels/nemesis/netmod/scif/Makefile.mk deleted file mode 100644 index 67c10c5..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/Makefile.mk +++ /dev/null @@ -1,21 +0,0 @@ -## -*- Mode: Makefile; -*- -## vim: set ft=automake : -## -## (C) 2012 by Argonne National Laboratory. -## See COPYRIGHT in top-level directory. -## - -if BUILD_NEMESIS_NETMOD_SCIF - -mpi_core_sources += \ - src/mpid/ch3/channels/nemesis/netmod/scif/scif_finalize.c \ - src/mpid/ch3/channels/nemesis/netmod/scif/scif_init.c \ - src/mpid/ch3/channels/nemesis/netmod/scif/scif_send.c \ - src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.c \ - src/mpid/ch3/channels/nemesis/netmod/scif/scifsm.c - -noinst_HEADERS += \ - src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.h \ - src/mpid/ch3/channels/nemesis/netmod/scif/scif_impl.h - -endif BUILD_NEMESIS_NETMOD_SCIF diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/cross_values.txt b/src/mpid/ch3/channels/nemesis/netmod/scif/cross_values.txt deleted file mode 100644 index 09423c6..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/cross_values.txt +++ /dev/null @@ -1,16 +0,0 @@ -# The Fortran related cross compilation values. -# This file is generated with mpich/maint/fcrosscompile/configure -# with CC/F77/FC set to "icc/ifort -mmic". -CROSS_F77_SIZEOF_INTEGER="4" -CROSS_F77_SIZEOF_REAL="4" -CROSS_F77_SIZEOF_DOUBLE_PRECISION="8" -CROSS_F77_TRUE_VALUE="-1" -CROSS_F77_FALSE_VALUE="0" -CROSS_F90_ADDRESS_KIND="8" -CROSS_F90_OFFSET_KIND="8" -CROSS_F90_INTEGER_KIND="4" -CROSS_F90_REAL_MODEL=" 6 , 37" -CROSS_F90_DOUBLE_MODEL=" 15 , 307" -CROSS_F90_INTEGER_MODEL=" 9" -CROSS_F90_ALL_INTEGER_MODELS=" 2 , 1, 4 , 2, 9 , 4, 18 , 8," -CROSS_F90_INTEGER_MODEL_MAP=" { 2 , 1 , 1 }, { 4 , 2 , 2 }, { 9 , 4 , 4 }, { 18 , 8 , 8 }," diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/scif_finalize.c b/src/mpid/ch3/channels/nemesis/netmod/scif/scif_finalize.c deleted file mode 100644 index ea1ed87..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/scif_finalize.c +++ /dev/null @@ -1,37 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Portions of this code were written by Intel Corporation. - * Copyright (C) 2011-2012 Intel Corporation. Intel provides this material - * to Argonne National Laboratory subject to Software Grant and Corporate - * Contributor License Agreement dated February 8, 2012. - */ - -#include "scif_impl.h" - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_finalize -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_finalize(void) -{ - int i, ret; - scifconn_t *it_sc; - - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_FINALIZE); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_FINALIZE); - - for (i = 0; i < MPID_nem_scif_nranks; ++i) { - it_sc = &MPID_nem_scif_conns[i]; - if (it_sc->fd == -1) { - continue; /* no connection */ - } - if (scif_close(it_sc->fd) == 0) - it_sc->fd = -1; - } - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_FINALIZE); - return MPI_SUCCESS; -} diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/scif_impl.h b/src/mpid/ch3/channels/nemesis/netmod/scif/scif_impl.h deleted file mode 100644 index b52e89a..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/scif_impl.h +++ /dev/null @@ -1,89 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Portions of this code were written by Intel Corporation. - * Copyright (C) 2011-2012 Intel Corporation. Intel provides this material - * to Argonne National Laboratory subject to Software Grant and Corporate - * Contributor License Agreement dated February 8, 2012. - */ - -#ifndef SCIF_IMPL_H -#define SCIF_IMPL_H - -#include "mpid_nem_impl.h" -#include <sys/types.h> -#include <scif.h> -#include <errno.h> -#include "scifrw.h" - -typedef GENERIC_Q_DECL(struct MPID_Request) reqq_t; - -typedef struct { - int fd; - struct scif_portID addr; - MPIDI_VC_t *vc; - shmchan_t csend, crecv; -} scifconn_t; - -/* The vc provides a generic buffer in which network modules can store - private fields This removes all dependencies from the VC struction - on the network module, facilitating dynamic module loading. */ -typedef struct { - scifconn_t *sc; - reqq_t send_queue; - int terminate; -} MPID_nem_scif_vc_area; - -/* macro for scif private in VC */ -#define VC_SCIF(vc) ((MPID_nem_scif_vc_area *)vc->ch.netmod_area.padding) - -typedef struct { - uint64_t seqno; -} scifmsg_t; - -#define RQ_SCIF(req) ((scifmsg_t *)(&(req)->ch.netmod_area.padding)) - -#define ASSIGN_SC_TO_VC(vc_scif_, sc_) do { \ - (vc_scif_)->sc = (sc_); \ - } while (0) - -/* functions */ -int MPID_nem_scif_init(MPIDI_PG_t * pg_p, int pg_rank, char **bc_val_p, int *val_max_sz_p); -int MPID_nem_scif_finalize(void); - -int MPID_nem_scif_get_business_card(int my_rank, char **bc_val_p, int *val_max_sz_p); -int MPID_nem_scif_connect_to_root(const char *business_card, MPIDI_VC_t * new_vc); -int MPID_nem_scif_vc_init(MPIDI_VC_t * vc); -int MPID_nem_scif_vc_destroy(MPIDI_VC_t * vc); -int MPID_nem_scif_vc_terminate(MPIDI_VC_t * vc); - -int MPID_nem_scif_iSendContig(MPIDI_VC_t * vc, MPID_Request * sreq, - void *hdr, MPIDI_msg_sz_t hdr_sz, - void *data, MPIDI_msg_sz_t data_sz); -int MPID_nem_scif_iStartContigMsg(MPIDI_VC_t * vc, void *hdr, - MPIDI_msg_sz_t hdr_sz, void *data, - MPIDI_msg_sz_t data_sz, MPID_Request ** sreq_ptr); -int MPID_nem_scif_SendNoncontig(MPIDI_VC_t * vc, MPID_Request * sreq, - void *header, MPIDI_msg_sz_t hdr_sz); - -int MPID_nem_scif_vc_terminated(MPIDI_VC_t * vc); -int MPID_nem_scif_connpoll(int in_blocking_poll); - -int MPID_nem_scif_error_out_send_queue(struct MPIDI_VC *vc, int req_errno); -int MPID_nem_scif_send_queued(MPIDI_VC_t * vc, reqq_t * send_queue); - -/* Keys for business cards */ -#define MPIDI_CH3I_PORT_KEY "port" -#define MPIDI_CH3I_HOST_DESCRIPTION_KEY "description" -#define MPIDI_CH3I_NODE_KEY "node" - -#define MPID_NEM_SCIF_RECV_MAX_PKT_LEN 1024 - -extern scifconn_t *MPID_nem_scif_conns; -extern int MPID_nem_scif_nranks; -extern char *MPID_nem_scif_recv_buf; -extern int MPID_nem_scif_myrank; - -#endif /* SCIF_IMPL_H */ diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/scif_init.c b/src/mpid/ch3/channels/nemesis/netmod/scif/scif_init.c deleted file mode 100644 index e047264..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/scif_init.c +++ /dev/null @@ -1,469 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Portions of this code were written by Intel Corporation. - * Copyright (C) 2011-2013 Intel Corporation. Intel provides this material - * to Argonne National Laboratory subject to Software Grant and Corporate - * Contributor License Agreement dated February 8, 2012. - */ - -#include "scif_impl.h" -#ifdef USE_PMI2_API -#include "pmi2.h" -#else -#include "pmi.h" -#endif - -MPID_nem_netmod_funcs_t MPIDI_nem_scif_funcs = { - MPID_nem_scif_init, - MPID_nem_scif_finalize, -#ifdef ENABLE_CHECKPOINTING - NULL, - NULL, - NULL, -#endif - MPID_nem_scif_connpoll, - MPID_nem_scif_get_business_card, - MPID_nem_scif_connect_to_root, - MPID_nem_scif_vc_init, - MPID_nem_scif_vc_destroy, - MPID_nem_scif_vc_terminate, - NULL /* anysource iprobe */ -}; - -int MPID_nem_scif_nranks; -scifconn_t *MPID_nem_scif_conns; -char *MPID_nem_scif_recv_buf; -int MPID_nem_scif_myrank; - -static int listen_fd; -static int listen_port; - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_post_init -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_nem_scif_post_init(void) -{ - int mpi_errno = MPI_SUCCESS, pmi_errno; - MPIDI_PG_t *my_pg = MPIDI_Process.my_pg; - - int my_rank = MPIDI_CH3I_my_rank; - int i; - MPIDI_VC_t *vc; - scifconn_t *sc; - MPIDI_CH3I_VC *vc_ch; - MPID_nem_scif_vc_area *vc_scif; - size_t s; - int ret; - off_t offset; - int peer_rank; - - MPIDI_STATE_DECL(MPID_NEM_SCIF_POST_INIT); - MPIDI_FUNC_ENTER(MPID_NEM_SCIF_POST_INIT); - - for (i = 0; i < MPID_nem_scif_nranks; i++) { - - vc = &my_pg->vct[i]; - vc_ch = &vc->ch; - - if (vc->pg_rank == MPID_nem_scif_myrank || vc_ch->is_local) { - continue; - } - - /* restore some value which might be rewrited during MPID_nem_vc_init() */ - - vc->sendNoncontig_fn = MPID_nem_scif_SendNoncontig; - vc_ch->iStartContigMsg = MPID_nem_scif_iStartContigMsg; - vc_ch->iSendContig = MPID_nem_scif_iSendContig; - - } - - fn_exit: - MPIDI_FUNC_EXIT(MPID_NEM_SCIF_POST_INIT); - return mpi_errno; - fn_fail: - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_init -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_init(MPIDI_PG_t * pg_p, int pg_rank, char **bc_val_p, int *val_max_sz_p) -{ - int mpi_errno = MPI_SUCCESS; - int pmi_errno; - int ret; - int i; - MPIU_CHKPMEM_DECL(2); - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_INIT); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_INIT); - - /* first make sure that our private fields in the vc fit into the - * area provided */ - MPIU_Assert(sizeof(MPID_nem_scif_vc_area) <= MPID_NEM_VC_NETMOD_AREA_LEN); - - MPID_nem_scif_nranks = pg_p->size; - MPID_nem_scif_myrank = pg_rank; - - /* set up listener socket */ - { - listen_fd = scif_open(); - MPIU_ERR_CHKANDJUMP1(listen_fd == -1, mpi_errno, MPI_ERR_OTHER, - "**scif_open", "**scif_open %s", MPIU_Strerror(errno)); - - listen_port = scif_bind(listen_fd, 0); - MPIU_ERR_CHKANDJUMP1(listen_port == -1, mpi_errno, MPI_ERR_OTHER, - "**scif_bind", "**scif_bind %s", MPIU_Strerror(errno)); - - ret = scif_listen(listen_fd, MPID_nem_scif_nranks); - MPIU_ERR_CHKANDJUMP1(ret == -1, mpi_errno, MPI_ERR_OTHER, - "**scif_listen", "**scif_listen %s", MPIU_Strerror(errno)); - } - - /* create business card */ - mpi_errno = MPID_nem_scif_get_business_card(pg_rank, bc_val_p, val_max_sz_p); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - - MPIU_CHKPMEM_MALLOC(MPID_nem_scif_conns, scifconn_t *, - MPID_nem_scif_nranks * sizeof(scifconn_t), mpi_errno, "connection table"); - memset(MPID_nem_scif_conns, 0, MPID_nem_scif_nranks * sizeof(scifconn_t)); - for (i = 0; i < MPID_nem_scif_nranks; ++i) - MPID_nem_scif_conns[i].fd = -1; - - MPIU_CHKPMEM_MALLOC(MPID_nem_scif_recv_buf, char *, - MPID_NEM_SCIF_RECV_MAX_PKT_LEN, mpi_errno, "SCIF temporary buffer"); - MPIU_CHKPMEM_COMMIT(); - mpi_errno = MPID_nem_register_initcomp_cb(MPID_nem_scif_post_init); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - pmi_errno = PMI_Barrier(); - MPIU_ERR_CHKANDJUMP1(pmi_errno != PMI_SUCCESS, mpi_errno, MPI_ERR_OTHER, "**pmi_barrier", - "**pmi_barrier %d", pmi_errno); - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_INIT); - return mpi_errno; - fn_fail: - MPIU_CHKPMEM_REAP(); - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_get_business_card -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_get_business_card(int my_rank, char **bc_val_p, int *val_max_sz_p) -{ - int mpi_errno = MPI_SUCCESS; - int str_errno = MPIU_STR_SUCCESS; - int ret; - char hostname[512]; - uint16_t self; - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_GET_BUSINESS_CARD); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_GET_BUSINESS_CARD); - - hostname[sizeof(hostname) - 1] = 0; - gethostname(hostname, sizeof(hostname) - 1); - str_errno = - MPIU_Str_add_string_arg(bc_val_p, val_max_sz_p, MPIDI_CH3I_HOST_DESCRIPTION_KEY, hostname); - if (str_errno) { - MPIU_ERR_CHKANDJUMP(str_errno == MPIU_STR_NOMEM, mpi_errno, MPI_ERR_OTHER, "**buscard_len"); - MPIU_ERR_SETANDJUMP(mpi_errno, MPI_ERR_OTHER, "**buscard"); - } - - str_errno = MPIU_Str_add_int_arg(bc_val_p, val_max_sz_p, MPIDI_CH3I_PORT_KEY, listen_port); - if (str_errno) { - MPIU_ERR_CHKANDJUMP(str_errno == MPIU_STR_NOMEM, mpi_errno, MPI_ERR_OTHER, "**buscard_len"); - MPIU_ERR_SETANDJUMP(mpi_errno, MPI_ERR_OTHER, "**buscard"); - } - - ret = scif_get_nodeIDs(NULL, 0, &self); - MPIU_ERR_CHKANDJUMP2(ret == -1, mpi_errno, MPI_ERR_OTHER, - "**scif_get_nodeIDs", "**scif_get_nodeIDs %s %d", - MPIU_Strerror(errno), errno); - str_errno = MPIU_Str_add_int_arg(bc_val_p, val_max_sz_p, MPIDI_CH3I_NODE_KEY, self); - if (str_errno) { - MPIU_ERR_CHKANDJUMP(str_errno == MPIU_STR_NOMEM, mpi_errno, MPI_ERR_OTHER, "**buscard_len"); - MPIU_ERR_SETANDJUMP(mpi_errno, MPI_ERR_OTHER, "**buscard"); - } - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_GET_BUSINESS_CARD); - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_connect_to_root -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_connect_to_root(const char *business_card, MPIDI_VC_t * new_vc) -{ - return MPI_SUCCESS; -} - -#undef FUNCNAME -#define FUNCNAME scif_addr_from_bc -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -static int scif_addr_from_bc(const char *business_card, uint16_t * addr, uint16_t * port) -{ - int mpi_errno = MPI_SUCCESS; - int ret; - int tmp; - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_GET_ADDR_PORT_FROM_BC); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_GET_ADDR_PORT_FROM_BC); - - ret = MPIU_Str_get_int_arg(business_card, MPIDI_CH3I_PORT_KEY, &tmp); - *port = (uint16_t) tmp; - /* MPIU_STR_FAIL is not a valid MPI error code so we store the - * result in ret instead of mpi_errno. */ - MPIU_ERR_CHKANDJUMP(ret != MPIU_STR_SUCCESS, mpi_errno, MPI_ERR_OTHER, "**argstr_missingport"); - - ret = MPIU_Str_get_int_arg(business_card, MPIDI_CH3I_NODE_KEY, &tmp); - *addr = (uint16_t) tmp; - MPIU_ERR_CHKANDJUMP(ret != MPIU_STR_SUCCESS, mpi_errno, MPI_ERR_OTHER, "**argstr_missingnode"); - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_GET_ADDR_PORT_FROM_BC); - return mpi_errno; - fn_fail: - MPIU_DBG_MSG_FMT(NEM_SOCK_DET, VERBOSE, (MPIU_DBG_FDEST, "failure. mpi_errno = %d", mpi_errno)); - goto fn_exit; -} - -static int get_addr(MPIDI_VC_t * vc, struct scif_portID *addr) -{ - int mpi_errno = MPI_SUCCESS; - char *bc; - int pmi_errno; - int val_max_sz; - MPIU_CHKLMEM_DECL(1); - - /* Allocate space for the business card */ - pmi_errno = PMI_KVS_Get_value_length_max(&val_max_sz); - MPIU_ERR_CHKANDJUMP1(pmi_errno, mpi_errno, MPI_ERR_OTHER, "**fail", "**fail %d", pmi_errno); - MPIU_CHKLMEM_MALLOC(bc, char *, val_max_sz, mpi_errno, "bc"); - - mpi_errno = vc->pg->getConnInfo(vc->pg_rank, bc, val_max_sz, vc->pg); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - - mpi_errno = scif_addr_from_bc(bc, &addr->node, &addr->port); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - - fn_exit: - MPIU_CHKLMEM_FREEALL(); - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_vc_init -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_vc_init(MPIDI_VC_t * vc) -{ - int mpi_errno = MPI_SUCCESS; - MPIDI_CH3I_VC *vc_ch; - MPID_nem_scif_vc_area *vc_scif; - int ret; - size_t s; - scifconn_t *sc; - off_t offset; - int peer_rank; - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_VC_INIT); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_VC_INIT); - - /* do the connection */ - if (vc->pg_rank < MPID_nem_scif_myrank) { - vc_ch = &vc->ch; - vc_scif = VC_SCIF(vc); - vc->sendNoncontig_fn = MPID_nem_scif_SendNoncontig; - vc_ch->iStartContigMsg = MPID_nem_scif_iStartContigMsg; - vc_ch->iSendContig = MPID_nem_scif_iSendContig; - - vc_ch->pkt_handler = NULL; // pkt_handlers; - vc_ch->num_pkt_handlers = 0; // MPIDI_NEM_SCIF_PKT_NUM_TYPES; - vc_ch->next = NULL; - vc_ch->prev = NULL; - - ASSIGN_SC_TO_VC(vc_scif, NULL); - vc_scif->send_queue.head = vc_scif->send_queue.tail = NULL; - vc_scif->sc = sc = &MPID_nem_scif_conns[vc->pg_rank]; - vc_scif->terminate = 0; - sc->vc = vc; - - sc->fd = scif_open(); - MPIU_ERR_CHKANDJUMP1(sc->fd == -1, mpi_errno, MPI_ERR_OTHER, - "**scif_open", "**scif_open %s", MPIU_Strerror(errno)); - mpi_errno = get_addr(vc, &sc->addr); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - ret = scif_connect(sc->fd, &sc->addr); - MPIU_ERR_CHKANDJUMP1(ret == -1, mpi_errno, MPI_ERR_OTHER, - "**scif_connect", "**scif_connect %s", MPIU_Strerror(errno)); - s = scif_send(sc->fd, &MPID_nem_scif_myrank, sizeof(MPID_nem_scif_myrank), SCIF_SEND_BLOCK); - MPIU_ERR_CHKANDJUMP1(s != sizeof(MPID_nem_scif_myrank), mpi_errno, MPI_ERR_OTHER, - "**scif_send", "**scif_send %s", MPIU_Strerror(errno)); - } - else { - struct scif_portID portID; - int fd; - // Can accept a connection from any peer, not necessary from vc->pg_rank. - // So we need to know the actual peer and adjust vc. - ret = scif_accept(listen_fd, &portID, &fd, SCIF_ACCEPT_SYNC); - MPIU_ERR_CHKANDJUMP1(ret, mpi_errno, MPI_ERR_OTHER, - "**scif_accept", "**scif_accept %s", MPIU_Strerror(errno)); - s = scif_recv(fd, &peer_rank, sizeof(peer_rank), SCIF_RECV_BLOCK); - MPIU_ERR_CHKANDJUMP1(s != sizeof(peer_rank), mpi_errno, MPI_ERR_OTHER, "**scif_recv", - "**scif_recv %s", MPIU_Strerror(errno)); - // check and adjust vc - if (peer_rank != vc->pg_rank) { - // get another vc - MPIDI_PG_Get_vc(MPIDI_Process.my_pg, peer_rank, &vc); - // check another new corresponds to actual peer_rank - MPIU_ERR_CHKANDJUMP1(peer_rank != vc->pg_rank, mpi_errno, MPI_ERR_OTHER, - "**MPIDI_PG_Get_vc", "**MPIDI_PG_Get_vc %s", - "wrong vc after accept"); - } - vc_ch = &vc->ch; - vc_scif = VC_SCIF(vc); - - vc->sendNoncontig_fn = MPID_nem_scif_SendNoncontig; - vc_ch->iStartContigMsg = MPID_nem_scif_iStartContigMsg; - vc_ch->iSendContig = MPID_nem_scif_iSendContig; - vc_ch->pkt_handler = NULL; // pkt_handlers; - vc_ch->num_pkt_handlers = 0; // MPIDI_NEM_SCIF_PKT_NUM_TYPES; - vc_ch->next = NULL; - vc_ch->prev = NULL; - ASSIGN_SC_TO_VC(vc_scif, NULL); - vc_scif->send_queue.head = vc_scif->send_queue.tail = NULL; - vc_scif->sc = sc = &MPID_nem_scif_conns[vc->pg_rank]; - vc_scif->terminate = 0; - sc->vc = vc; - sc->addr = portID; - sc->fd = fd; - } - MPIDI_CHANGE_VC_STATE(vc, ACTIVE); - ret = MPID_nem_scif_init_shmsend(&sc->csend, sc->fd, vc->pg_rank); - MPIU_ERR_CHKANDJUMP1(ret, mpi_errno, MPI_ERR_OTHER, - "**scif_init_shmsend", "**scif_init_shmsend %s", MPIU_Strerror(errno)); - - /* Exchange offsets */ - s = scif_send(sc->fd, &sc->csend.offset, sizeof(off_t), SCIF_SEND_BLOCK); - MPIU_ERR_CHKANDJUMP1(s != sizeof(off_t), mpi_errno, MPI_ERR_OTHER, - "**scif_send", "**scif_send %s", MPIU_Strerror(errno)); - s = scif_recv(sc->fd, &offset, sizeof(off_t), SCIF_RECV_BLOCK); - MPIU_ERR_CHKANDJUMP1(s != sizeof(off_t), mpi_errno, MPI_ERR_OTHER, - "**scif_recv", "**scif_recv %s", MPIU_Strerror(errno)); - - ret = MPID_nem_scif_init_shmrecv(&sc->crecv, sc->fd, offset, vc->pg_rank); - MPIU_ERR_CHKANDJUMP1(ret, mpi_errno, MPI_ERR_OTHER, - "**scif_init_shmrecv", "**scif_init_shmrecv %s", MPIU_Strerror(errno)); - - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_VC_INIT); - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_vc_destroy -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_vc_destroy(MPIDI_VC_t * vc) -{ - /* currently do nothing */ - return MPI_SUCCESS; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_vc_terminate -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_nem_scif_vc_terminate(MPIDI_VC_t * vc) -{ - int mpi_errno = MPI_SUCCESS; - int req_errno = MPI_SUCCESS; - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_VC_TERMINATE); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_VC_TERMINATE); - - if (vc->state != MPIDI_VC_STATE_CLOSED) { - /* VC is terminated as a result of a fault. Complete - * outstanding sends with an error and terminate connection - * immediately. */ - MPIU_ERR_SET1(req_errno, MPI_ERR_OTHER, "**comm_fail", "**comm_fail %d", vc->pg_rank); - mpi_errno = MPID_nem_scif_error_out_send_queue(vc, req_errno); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - mpi_errno = MPID_nem_scif_vc_terminated(vc); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - } - else { - MPID_nem_scif_vc_area *vc_scif = VC_SCIF(vc); - - vc_scif->terminate = 1; - /* VC is terminated as a result of the close protocol. Wait - * for sends to complete, then terminate. */ - - if (MPIDI_CH3I_Sendq_empty(vc_scif->send_queue)) { - /* The sendq is empty, so we can immediately terminate the - * connection. */ - mpi_errno = MPID_nem_scif_vc_terminated(vc); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - } - /* else: just return. We'll call vc_terminated() from the - * commrdy_handler once the sendq is empty. */ - } - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_VC_TERMINATE); - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_vc_terminated -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_vc_terminated(MPIDI_VC_t * vc) -{ - /* This is called when the VC is to be terminated once all queued - * sends have been sent. */ - int mpi_errno = MPI_SUCCESS; - MPIDI_STATE_DECL(MPID_NEM_SCIF_VC_TERMINATED); - - MPIDI_FUNC_ENTER(MPID_NEM_SCIF_VC_TERMINATED); - - mpi_errno = MPIDI_CH3U_Handle_connection(vc, MPIDI_VC_EVENT_TERMINATED); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - - fn_exit: - MPIDI_FUNC_EXIT(MPID_NEM_SCIF_VC_TERMINATED); - return mpi_errno; - fn_fail: - MPIU_DBG_MSG_FMT(NEM_SOCK_DET, VERBOSE, (MPIU_DBG_FDEST, "failure. mpi_errno = %d", mpi_errno)); - goto fn_exit; -} diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/scif_send.c b/src/mpid/ch3/channels/nemesis/netmod/scif/scif_send.c deleted file mode 100644 index 1c0d345..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/scif_send.c +++ /dev/null @@ -1,490 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Portions of this code were written by Intel Corporation. - * Copyright (C) 2011-2012 Intel Corporation. Intel provides this material - * to Argonne National Laboratory subject to Software Grant and Corporate - * Contributor License Agreement dated February 8, 2012. - */ - -#include "scif_impl.h" - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_send_queued -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_send_queued(MPIDI_VC_t * vc, reqq_t * send_queue) -{ - int mpi_errno = MPI_SUCCESS; - MPID_Request *sreq; - MPIDI_msg_sz_t offset; - MPID_IOV *iov; - int complete; - uint64_t seqno; - MPID_nem_scif_vc_area *vc_scif = VC_SCIF(vc); - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_SEND_QUEUED); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_SEND_QUEUED); - - MPIU_Assert(vc != NULL); - - if (MPIDI_CH3I_Sendq_empty(*send_queue)) { - if (vc_scif->terminate) - MPID_nem_scif_vc_terminated(vc); - goto fn_exit; - } - while (!MPIDI_CH3I_Sendq_empty(*send_queue)) { - sreq = MPIDI_CH3I_Sendq_head(*send_queue); - seqno = RQ_SCIF(sreq)->seqno; - complete = 0; - if (seqno) { - /* We were just waiting for DMA to finish */ - complete = MPID_nem_scif_chk_seqno(&vc_scif->sc->csend, seqno); - } - else if (MPID_nem_scif_poll_send(vc_scif->sc->fd, &vc_scif->sc->csend)) { - MPIU_DBG_MSG_P(CH3_CHANNEL, VERBOSE, "Sending %p", sreq); - - iov = &sreq->dev.iov[sreq->dev.iov_offset]; - - offset = MPID_nem_scif_writev(vc_scif->sc->fd, &vc_scif->sc->csend, - iov, sreq->dev.iov_count, &seqno); - MPIU_ERR_CHKANDJUMP1(offset <= 0, mpi_errno, MPI_ERR_OTHER, - "**scif_writev", "**scif_writev %s", MPIU_Strerror(errno)); - MPIU_DBG_MSG_D(CH3_CHANNEL, VERBOSE, "write " MPIDI_MSG_SZ_FMT, offset); - RQ_SCIF(sreq)->seqno = seqno; - complete = 1; - for (iov = &sreq->dev.iov[sreq->dev.iov_offset]; - iov < &sreq->dev.iov[sreq->dev.iov_offset + sreq->dev.iov_count]; ++iov) { - if (offset < iov->MPID_IOV_LEN) { - iov->MPID_IOV_BUF = (char *) iov->MPID_IOV_BUF + offset; - iov->MPID_IOV_LEN -= offset; - /* iov_count should be equal to the number of iov's remaining */ - sreq->dev.iov_count -= ((iov - sreq->dev.iov) - sreq->dev.iov_offset); - sreq->dev.iov_offset = iov - sreq->dev.iov; - complete = 0; - seqno = 0; - break; - } - offset -= iov->MPID_IOV_LEN; - } - if (seqno) - complete = 0; - } - if (!complete) { - /* writev couldn't write the entire iov, give up for now */ - break; - } - else { - /* sent whole message */ - int (*reqFn) (MPIDI_VC_t *, MPID_Request *, int *); - - RQ_SCIF(sreq)->seqno = 0; - reqFn = sreq->dev.OnDataAvail; - if (!reqFn) { - MPIU_Assert(MPIDI_Request_get_type(sreq) != MPIDI_REQUEST_TYPE_GET_RESP); - MPIDI_CH3U_Request_complete(sreq); - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, ".... complete"); - MPIDI_CH3I_Sendq_dequeue(send_queue, &sreq); - continue; - } - - complete = 0; - mpi_errno = reqFn(vc, sreq, &complete); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - - if (complete) { - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, ".... complete"); - MPIDI_CH3I_Sendq_dequeue(send_queue, &sreq); - continue; - } - sreq->dev.iov_offset = 0; - } - } - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_SEND_QUEUED); - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_iStartContigMsg -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_iStartContigMsg(MPIDI_VC_t * vc, void *hdr, MPIDI_msg_sz_t hdr_sz, - void *data, MPIDI_msg_sz_t data_sz, MPID_Request ** sreq_ptr) -{ - int mpi_errno = MPI_SUCCESS; - MPID_Request *sreq = NULL; - MPIDI_msg_sz_t offset = 0; - MPID_nem_scif_vc_area *vc_scif = VC_SCIF(vc); - scifconn_t *sc = vc_scif->sc; - uint64_t seqno = 0; - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_ISTARTCONTIGMSG); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_ISTARTCONTIGMSG); - - MPIU_Assert(hdr_sz <= sizeof(MPIDI_CH3_Pkt_t)); - - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "scif_iStartContigMsg"); - MPIDI_DBG_Print_packet((MPIDI_CH3_Pkt_t *) hdr); - - if (MPIDI_CH3I_Sendq_empty(vc_scif->send_queue) && - MPID_nem_scif_poll_send(sc->fd, &sc->csend)) { - MPID_IOV iov[2]; - - iov[0].MPID_IOV_BUF = hdr; - iov[0].MPID_IOV_LEN = sizeof(MPIDI_CH3_Pkt_t); - iov[1].MPID_IOV_BUF = data; - iov[1].MPID_IOV_LEN = data_sz; - - offset = MPID_nem_scif_writev(sc->fd, &sc->csend, iov, 2, &seqno); - MPIU_ERR_CHKANDJUMP1(offset <= 0, mpi_errno, MPI_ERR_OTHER, - "**scif_writev", "**scif_writev %s", MPIU_Strerror(errno)); - MPIU_DBG_MSG_FMT(CH3_CHANNEL, VERBOSE, - (MPIU_DBG_FDEST, "scif_send " MPIDI_MSG_SZ_FMT " fd=%d", - offset, sc->fd)); - - if (offset == sizeof(MPIDI_CH3_Pkt_t) + data_sz) { - if (!seqno) { - /* sent whole message */ - *sreq_ptr = NULL; - goto fn_exit; - } - } - } - - /* create and enqueue request */ - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "enqueuing"); - - /* create a request */ - sreq = MPID_Request_create(); - RQ_SCIF(sreq)->seqno = seqno; - MPIU_Assert(sreq != NULL); - MPIU_Object_set_ref(sreq, 2); - sreq->kind = MPID_REQUEST_SEND; - - sreq->dev.OnDataAvail = 0; - sreq->ch.vc = vc; - sreq->dev.iov_offset = 0; - - if (!seqno) { - if (offset < sizeof(MPIDI_CH3_Pkt_t)) { - sreq->dev.pending_pkt = *(MPIDI_CH3_Pkt_t *) hdr; - sreq->dev.iov[0].MPID_IOV_BUF = (char *) &sreq->dev.pending_pkt + offset; - sreq->dev.iov[0].MPID_IOV_LEN = sizeof(MPIDI_CH3_Pkt_t) - offset; - if (data_sz) { - sreq->dev.iov[1].MPID_IOV_BUF = data; - sreq->dev.iov[1].MPID_IOV_LEN = data_sz; - sreq->dev.iov_count = 2; - } - else - sreq->dev.iov_count = 1; - } - else { - sreq->dev.iov[0].MPID_IOV_BUF = (char *) data + (offset - sizeof(MPIDI_CH3_Pkt_t)); - sreq->dev.iov[0].MPID_IOV_LEN = data_sz - (offset - sizeof(MPIDI_CH3_Pkt_t)); - sreq->dev.iov_count = 1; - } - - MPIU_Assert(sreq->dev.iov_count >= 1 && sreq->dev.iov[0].MPID_IOV_LEN > 0); - } - - if (MPIDI_CH3I_Sendq_empty(vc_scif->send_queue)) { - /* this will be the first send on the queue: queue it and set - * the write flag on the pollfd */ - MPIDI_CH3I_Sendq_enqueue(&vc_scif->send_queue, sreq); - } - else { - /* there are other sends in the queue before this one: try to - * send from the queue */ - MPIDI_CH3I_Sendq_enqueue(&vc_scif->send_queue, sreq); - mpi_errno = MPID_nem_scif_send_queued(vc, &vc_scif->send_queue); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - } - - *sreq_ptr = sreq; - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_ISTARTCONTIGMSG); - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_iSendContig -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_iSendContig(MPIDI_VC_t * vc, MPID_Request * sreq, void *hdr, - MPIDI_msg_sz_t hdr_sz, void *data, MPIDI_msg_sz_t data_sz) -{ - int mpi_errno = MPI_SUCCESS; - MPIDI_msg_sz_t offset = 0; - MPID_nem_scif_vc_area *vc_scif = VC_SCIF(vc); - scifconn_t *sc = vc_scif->sc; - uint64_t seqno = 0; - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_ISENDCONTIGMSG); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_ISENDCONTIGMSG); - - MPIU_Assert(hdr_sz <= sizeof(MPIDI_CH3_Pkt_t)); - - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "scif_iSendContig"); - - MPIDI_DBG_Print_packet((MPIDI_CH3_Pkt_t *) hdr); - - if (MPIDI_CH3I_Sendq_empty(vc_scif->send_queue) && - MPID_nem_scif_poll_send(sc->fd, &sc->csend)) { - MPID_IOV iov[2]; - - iov[0].MPID_IOV_BUF = hdr; - iov[0].MPID_IOV_LEN = sizeof(MPIDI_CH3_Pkt_t); - iov[1].MPID_IOV_BUF = data; - iov[1].MPID_IOV_LEN = data_sz; - - offset = MPID_nem_scif_writev(sc->fd, &sc->csend, iov, 2, &seqno); - MPIU_ERR_CHKANDJUMP1(offset <= 0, mpi_errno, MPI_ERR_OTHER, - "**scif_writev", "**scif_writev %s", MPIU_Strerror(errno)); - MPIU_DBG_MSG_FMT(CH3_CHANNEL, VERBOSE, - (MPIU_DBG_FDEST, "scif_send " MPIDI_MSG_SZ_FMT " fd=%d", - offset, sc->fd)); - - if (offset == sizeof(MPIDI_CH3_Pkt_t) + data_sz) { - /* sent whole message */ - int (*reqFn) (MPIDI_VC_t *, MPID_Request *, int *); - - if (seqno) - goto enqueue_request; - reqFn = sreq->dev.OnDataAvail; - if (!reqFn) { - MPIU_Assert(MPIDI_Request_get_type(sreq) != MPIDI_REQUEST_TYPE_GET_RESP); - MPIDI_CH3U_Request_complete(sreq); - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, ".... complete"); - goto fn_exit; - } - else { - int complete = 0; - - mpi_errno = reqFn(vc, sreq, &complete); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - - if (complete) { - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, ".... complete"); - goto fn_exit; - } - - /* not completed: more to send */ - goto enqueue_request; - } - } - } - - /* save iov */ - if (offset < sizeof(MPIDI_CH3_Pkt_t)) { - sreq->dev.pending_pkt = *(MPIDI_CH3_Pkt_t *) hdr; - sreq->dev.iov[0].MPID_IOV_BUF = (char *) &sreq->dev.pending_pkt + offset; - sreq->dev.iov[0].MPID_IOV_LEN = sizeof(MPIDI_CH3_Pkt_t) - offset; - if (data_sz) { - sreq->dev.iov[1].MPID_IOV_BUF = data; - sreq->dev.iov[1].MPID_IOV_LEN = data_sz; - sreq->dev.iov_count = 2; - } - else - sreq->dev.iov_count = 1; - seqno = 0; - } - else { - sreq->dev.iov[0].MPID_IOV_BUF = (char *) data + (offset - sizeof(MPIDI_CH3_Pkt_t)); - sreq->dev.iov[0].MPID_IOV_LEN = data_sz - (offset - sizeof(MPIDI_CH3_Pkt_t)); - sreq->dev.iov_count = 1; - seqno = 0; - } - - enqueue_request: - /* enqueue request */ - RQ_SCIF(sreq)->seqno = seqno; - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "enqueuing"); - MPIU_Assert(seqno || (sreq->dev.iov_count >= 1 && sreq->dev.iov[0].MPID_IOV_LEN > 0)); - - sreq->ch.vc = vc; - sreq->dev.iov_offset = 0; - - if (MPIDI_CH3I_Sendq_empty(vc_scif->send_queue)) { - /* this will be the first send on the queue: queue it and set - * the write flag on the pollfd */ - MPIDI_CH3I_Sendq_enqueue(&vc_scif->send_queue, sreq); - } - else { - /* there are other sends in the queue before this one: try to - * send from the queue */ - MPIDI_CH3I_Sendq_enqueue(&vc_scif->send_queue, sreq); - mpi_errno = MPID_nem_scif_send_queued(vc, &vc_scif->send_queue); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - } - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_ISENDCONTIGMSG); - return mpi_errno; - fn_fail: - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_SendNoncontig -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_scif_SendNoncontig(MPIDI_VC_t * vc, MPID_Request * sreq, void *header, - MPIDI_msg_sz_t hdr_sz) -{ - int mpi_errno = MPI_SUCCESS; - int iov_n; - MPID_IOV iov[MPID_IOV_LIMIT]; - MPID_IOV *iov_p; - MPIDI_msg_sz_t offset = 0; - int complete; - uint64_t seqno = 0; - MPID_nem_scif_vc_area *vc_scif = VC_SCIF(vc); - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_SENDNONCONTIG); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_SENDNONCONTIG); - - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "scif_SendNoncontig"); - MPIU_Assert(hdr_sz <= sizeof(MPIDI_CH3_Pkt_t)); - - iov[0].MPID_IOV_BUF = header; - iov[0].MPID_IOV_LEN = sizeof(MPIDI_CH3_Pkt_t); - - iov_n = MPID_IOV_LIMIT - 1; - - mpi_errno = MPIDI_CH3U_Request_load_send_iov(sreq, &iov[1], &iov_n); - MPIU_ERR_CHKANDJUMP(mpi_errno, mpi_errno, MPI_ERR_OTHER, "**loadsendiov"); - - iov_n += 1; - offset = 0; - - if (MPIDI_CH3I_Sendq_empty(vc_scif->send_queue) && - MPID_nem_scif_poll_send(vc_scif->sc->fd, &vc_scif->sc->csend)) { - offset = - MPID_nem_scif_writev(vc_scif->sc->fd, &vc_scif->sc->csend, iov, iov_n, &seqno); - MPIU_ERR_CHKANDJUMP1(offset <= 0, mpi_errno, MPI_ERR_OTHER, - "**scif_writev", "**scif_writev %s", MPIU_Strerror(errno)); - - MPIU_DBG_MSG_D(CH3_CHANNEL, VERBOSE, "scif_send noncontig " MPIDI_MSG_SZ_FMT, offset); - } - - if (offset < iov[0].MPID_IOV_LEN) { - /* header was not yet sent, save it in req */ - sreq->dev.pending_pkt = *(MPIDI_CH3_Pkt_t *) header; - iov[0].MPID_IOV_BUF = (MPID_IOV_BUF_CAST) & sreq->dev.pending_pkt; - iov[0].MPID_IOV_LEN = sizeof(MPIDI_CH3_Pkt_t); - } - - /* check if whole iov was sent, and save any unsent portion of - * iov */ - sreq->dev.iov_count = 0; - complete = 1; - for (iov_p = &iov[0]; iov_p < &iov[iov_n]; ++iov_p) { - if (offset < iov_p->MPID_IOV_LEN) { - sreq->dev.iov[sreq->dev.iov_count].MPID_IOV_BUF = - (MPID_IOV_BUF_CAST) ((char *) iov_p->MPID_IOV_BUF + offset); - sreq->dev.iov[sreq->dev.iov_count].MPID_IOV_LEN = iov_p->MPID_IOV_LEN - offset; - offset = 0; - ++sreq->dev.iov_count; - complete = 0; - seqno = 0; - } - else - offset -= iov_p->MPID_IOV_LEN; - } - if (seqno) - complete = MPID_nem_scif_chk_seqno(&vc_scif->sc->csend, seqno); - - if (complete) { - /* sent whole iov */ - int (*reqFn) (MPIDI_VC_t *, MPID_Request *, int *); - - reqFn = sreq->dev.OnDataAvail; - if (!reqFn) { - MPIDI_CH3U_Request_complete(sreq); - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, ".... complete"); - goto fn_exit; - } - - complete = 0; - mpi_errno = reqFn(vc, sreq, &complete); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - - if (complete) { - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, ".... complete"); - goto fn_exit; - } - seqno = 0; - } - - /* enqueue request */ - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "enqueuing"); - MPIU_Assert(seqno || (sreq->dev.iov_count >= 1 && sreq->dev.iov[0].MPID_IOV_LEN > 0)); - - RQ_SCIF(sreq)->seqno = seqno; - sreq->ch.vc = vc; - sreq->dev.iov_offset = 0; - - if (MPIDI_CH3I_Sendq_empty(vc_scif->send_queue)) { - /* this will be the first send on the queue: queue it and set - * the write flag on the pollfd */ - MPIDI_CH3I_Sendq_enqueue(&vc_scif->send_queue, sreq); - } - else { - /* there are other sends in the queue before this one: try to - * send from the queue */ - MPIDI_CH3I_Sendq_enqueue(&vc_scif->send_queue, sreq); - mpi_errno = MPID_nem_scif_send_queued(vc, &vc_scif->send_queue); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - } - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_SENDNONCONTIG); - return mpi_errno; - fn_fail: - MPID_Request_release(sreq); - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_error_out_send_queue -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_nem_scif_error_out_send_queue(struct MPIDI_VC *const vc, int req_errno) -{ - int mpi_errno = MPI_SUCCESS; - MPID_Request *req; - MPID_nem_scif_vc_area *const vc_scif = VC_SCIF(vc); - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_ERROR_OUT_SEND_QUEUE); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_ERROR_OUT_SEND_QUEUE); - - /* we don't call onDataAvail or onFinal handlers because this is - * an error condition and we just want to mark them as complete */ - - /* send queue */ - while (!MPIDI_CH3I_Sendq_empty(vc_scif->send_queue)) { - MPIDI_CH3I_Sendq_dequeue(&vc_scif->send_queue, &req); - req->status.MPI_ERROR = req_errno; - - MPIDI_CH3U_Request_complete(req); - } - - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_ERROR_OUT_SEND_QUEUE); - return mpi_errno; -} diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.c b/src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.c deleted file mode 100644 index c1536d2..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.c +++ /dev/null @@ -1,472 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Portions of this code were written by Intel Corporation. - * Copyright (C) 2011-2012 Intel Corporation. Intel provides this material - * to Argonne National Laboratory subject to Software Grant and Corporate - * Contributor License Agreement dated February 8, 2012. - */ - -#include <stdio.h> -#include <stdlib.h> -#include <stdint.h> -#include <scif.h> -#include <assert.h> -#include <sys/uio.h> -#include <string.h> -#include "scifrw.h" -#if defined HAVE_UNISTD_H -#include <unistd.h> -#endif - -extern int MPID_nem_scif_myrank; - -int MPID_nem_scif_init_shmsend(shmchan_t * csend, int ep, int rank) -{ - int retval = 0; -/* FIXME: correctly detect getpagesize */ -#if defined HAVE_GETPAGESIZE || 1 - int pagesize = getpagesize(); -#else -#error need getpagesize -#endif - off_t offset; - void *p; - - assert(csend->buflen == 0); - retval = posix_memalign(&p, pagesize, SHMBUFSIZE); - if (retval != 0) - goto fn_exit; - offset = scif_register(ep, p, SHMBUFSIZE, 0, SCIF_PROT_READ | SCIF_PROT_WRITE, 0); - if (offset == SCIF_REGISTER_FAILED) { - retval = errno; - free(p); - goto fn_exit; - } - csend->bufp = p; - csend->buflen = SHMBUFSIZE; - csend->offset = offset; - csend->seqno = 10 * MPID_nem_scif_myrank; - csend->lseqno = (uint64_t *) csend->bufp; - csend->rseqno = csend->lseqno + 1; - *csend->lseqno = *csend->rseqno = csend->seqno; - csend->curp = csend->bufp + 2 * sizeof(uint64_t); - csend->pos = -1; - csend->reg = 0; - csend->rank = rank; - csend->dma_count = 0; - csend->dma_chdseqno = 0; - - fn_exit: - return retval; -} - -int MPID_nem_scif_init_shmrecv(shmchan_t * crecv, int ep, off_t offs, int rank) -{ - int retval = 0; - - crecv->bufp = scif_mmap(0, SHMBUFSIZE, SCIF_PROT_READ | SCIF_PROT_WRITE, 0, ep, offs); - if (crecv->bufp == SCIF_MMAP_FAILED) { - retval = errno; - goto fn_exit; - } - crecv->buflen = SHMBUFSIZE; - crecv->pos = -1; - crecv->offset = offs; - crecv->seqno = 10 * rank; - crecv->lseqno = (uint64_t *) crecv->bufp; - crecv->rseqno = crecv->lseqno + 1; - crecv->curp = crecv->bufp + 2 * sizeof(uint64_t); - crecv->reg = 0; - crecv->dmalen = 0; - crecv->dmaoffset = 0; - crecv->dmastart = -1; - crecv->dmaend = 0; - crecv->rank = rank; - fn_exit: - return retval; -} - -void MPID_nem_scif_unregmem(int ep, shmchan_t * c) -{ - regmem_t *rp = c->reg, *prev = c->reg; - uint64_t lseqno = c->dma_chdseqno; - - while (rp && c->dma_count) { - if (rp->seqno <= lseqno) { - scif_unregister(ep, rp->offset, rp->size); - prev->next = rp->next; - if (c->reg == rp) { - c->reg = rp->next; - } - free(rp); - --c->dma_count; - rp = prev->next; - } - else { - prev = rp; - rp = rp->next; - } - } -} - -static regmem_t *regmem(int ep, shmchan_t * c, void *addr, size_t len) -{ - regmem_t *rp; - uint64_t base, size; -/* FIXME: correctly detect getpagesize */ -#if defined HAVE_GETPAGESIZE || 1 - int pagesize = getpagesize(); -#else -#error need getpagesize -#endif - - /* - * FIXME: if the user unmaps this address and then it gets - * remapped to a different physical address, then this address - * won't be pinned anymore. Hopefully we'll just get a dma - * failure. - */ - - base = (uint64_t) addr & ~(pagesize - 1); - size = ((uint64_t) addr + len + pagesize - 1) & ~(pagesize - 1); - size -= base; - - rp = malloc(sizeof(regmem_t)); - rp->base = (char *) base; - rp->size = size; - rp->offset = scif_register(ep, (void *) base, size, 0, SCIF_PROT_READ | SCIF_PROT_WRITE, 0); - if (rp->offset == SCIF_REGISTER_FAILED) { - fprintf(stderr, "regmem failed: errno %d base 0x%lx size %ld\n", errno, base, size); - free(rp); - return 0; - } - rp->next = c->reg; - c->reg = rp; - return rp; -} - -static int dma_read(int ep, shmchan_t * c, void *recv_buf, off_t raddr, size_t msglen, - size_t buflen, int *did_dma) -{ - regmem_t *rp; - int retval = 0; - off_t locoffs; - size_t newbufsiz; - size_t firstcacheline; - size_t rem; - int mark; -/* FIXME: correctly detect getpagesize */ -#if defined HAVE_GETPAGESIZE || 1 - int pagesize = getpagesize(); -#else -#error need getpagesize -#endif - - /* if we have any left over from the last DMA */ - if (c->dmastart >= 0) { - rem = c->dmaend - c->dmastart - c->pos; - if (rem > buflen) - rem = buflen; - memcpy(recv_buf, c->dmabuf + c->dmastart + c->pos, rem); - if (c->dmastart + c->pos + rem >= c->dmaend) - c->dmastart = -1; - goto fn_exit; - } - - /* see if we can DMA into the destination */ - if (buflen >= msglen && - ((off_t) recv_buf & (CACHE_LINESIZE - 1)) == ((raddr + c->pos) & (CACHE_LINESIZE - 1))) { - - retval = scif_vreadfrom(ep, recv_buf, buflen, raddr + c->pos, 0); - if (retval < 0) { - fprintf(stderr, "scif_vreadfrom #1 returns %d, errno %d\n", retval, errno); - fprintf(stderr, "recv_buf: %p raddr: 0x%lx buflen: %ld\n", - recv_buf, raddr + c->pos, buflen); - } - scif_fence_mark(ep, SCIF_FENCE_INIT_SELF, &mark); - scif_fence_wait(ep, mark); - *did_dma = 1; - goto fn_exit; - } - - /* DMA into a local buffer and copy */ - newbufsiz = (msglen + pagesize + pagesize - 1) & ~(pagesize - 1); - if (newbufsiz < DMABUFSIZE) - newbufsiz = DMABUFSIZE; - if (newbufsiz > c->dmalen) { - void *p; - off_t offset; - if (c->dmalen) { - free(c->dmabuf); - } - retval = posix_memalign(&p, pagesize, newbufsiz); - if (retval != 0) - goto fn_exit; - - c->dmabuf = p; - c->dmalen = newbufsiz; - c->dmaoffset = 0; - c->dmastart = -1; - c->dmaend = 0; - } - - /* get the alignment right */ - firstcacheline = (raddr + (CACHE_LINESIZE - 1)) & ~(CACHE_LINESIZE - 1); - c->dmastart = CACHE_LINESIZE - (firstcacheline - raddr); - c->dmaend = c->dmastart + msglen; - if (c->dmaend > c->dmalen) { - fprintf(stderr, "%d: botch len c->dmastart %ld c->dmaend %ld c->dmalen %ld\n", - MPID_nem_scif_myrank, c->dmastart, c->dmaend, c->dmalen); - assert(0); - } - locoffs = c->dmaoffset + c->dmastart; - assert(c->pos == 0); - retval = scif_vreadfrom(ep, c->dmabuf + c->dmastart, msglen, raddr, 0); - *did_dma = 1; - scif_fence_mark(ep, SCIF_FENCE_INIT_SELF, &mark); - scif_fence_wait(ep, mark); - if (retval < 0) - fprintf(stderr, "scif_readfrom #2 returns %d, errno %d\n", retval, errno); - rem = c->dmaend - c->dmastart; - if (rem > buflen) - rem = buflen; - memcpy(recv_buf, c->dmabuf + c->dmastart, rem); - if (c->dmastart + rem >= c->dmaend) - c->dmastart = -1; - - fn_exit: - return retval; -} - -static ssize_t getmsg(int ep, shmchan_t * c, void *recv_buf, size_t len, int *did_dma) -{ - int64_t msglen; - int largemsg = 0; - size_t tlen; - int err; - ssize_t retval; - - msglen = *(int64_t *) c->curp; - - if (msglen == -1) { - /* go to beginning of buffer */ - c->curp = c->bufp + 2 * sizeof(uint64_t); - msglen = *(int64_t *) c->curp; - } - if (msglen < 0) { - largemsg = 1; - msglen = -msglen; - } - tlen = msglen - c->pos; - if (tlen > len) - tlen = len; - if (!largemsg) - memcpy(recv_buf, c->curp + sizeof(int64_t) + c->pos, tlen); - else { - err = dma_read(ep, c, recv_buf, ((off_t *) c->curp)[1], msglen, tlen, did_dma); - if (err) { - retval = -1; - goto fn_exit; - } - } - c->pos += tlen; - assert(c->pos <= msglen); - retval = tlen; - if (c->pos == msglen) { - /* Done with this message */ - c->pos = -1; - if (!largemsg) - c->curp += (sizeof(int64_t) + ((msglen + 7) & ~7)); - else - c->curp += (2 * sizeof(int64_t)); - } - fn_exit: - return retval; -} - -/* Read at most one message */ - -static ssize_t do_scif_read(int ep, shmchan_t * c, void *recv_buf, size_t len, int *did_dma) -{ - ssize_t retval = 0; - size_t nread = 0; - uint64_t rseqno; - - while (nread < len) { - if (c->pos >= 0) { - /* partial message chunk left */ - retval = getmsg(ep, c, (char *) recv_buf + nread, len - nread, did_dma); - if (retval < 0) { - nread = -1; - goto fn_exit; - } - nread += retval; - continue; - } - /* Check if we have a message */ - rseqno = *c->rseqno; - if (rseqno <= c->seqno) { - goto fn_exit; - } - /* Message is available */ - ++c->seqno; - c->pos = 0; - retval = getmsg(ep, c, (char *) recv_buf + nread, len - nread, did_dma); - if (retval < 0) { - nread = -1; - goto fn_exit; - } - nread += retval; - } - - fn_exit: - return nread; -} - -ssize_t MPID_nem_scif_read(int ep, shmchan_t * c, void *recv_buf, size_t len) -{ - int did_dma = 0; - ssize_t retval; - int mark; - - retval = do_scif_read(ep, c, recv_buf, len, &did_dma); - if (retval > 0 && did_dma) { - scif_fence_mark(ep, SCIF_FENCE_INIT_SELF, &mark); - scif_fence_wait(ep, mark); - } - /* Tell the other side we're done */ - *c->lseqno = c->seqno; - if (retval <= 0) - fprintf(stderr, "%2d: %2d: %08ld: %08ld: readv err %ld\n", - MPID_nem_scif_myrank, c->rank, c->seqno, *c->rseqno, retval); - return retval; -} - -ssize_t MPID_nem_scif_readv(int ep, shmchan_t * c, const struct iovec * iov, int iov_cnt) -{ - ssize_t retval = 0; - size_t nread = 0; - int did_dma = 0; - int mark; - int i; - - for (i = 0; i < iov_cnt; ++i) { - retval = do_scif_read(ep, c, iov[i].iov_base, iov[i].iov_len, &did_dma); - if (retval < 0) { - nread = retval; - goto fn_exit; - } - if (retval == 0) - break; - nread += retval; - if (retval < iov[i].iov_len) - break; - } - if (retval > 0 && did_dma) { - scif_fence_mark(ep, SCIF_FENCE_INIT_SELF, &mark); - scif_fence_wait(ep, mark); - } - fn_exit: - /* Tell the other side we're done */ - *c->lseqno = c->seqno; - if (nread <= 0) - fprintf(stderr, "%5d: %08ld: readv err %ld\n", getpid(), c->seqno, nread); - return nread; -} - -int MPID_nem_scif_poll_send(int ep, shmchan_t * csend) -{ - size_t avail; - int retval = 1; - uint64_t lseqno; - - avail = csend->buflen - (csend->curp - csend->bufp) - sizeof(uint64_t); - if (avail >= SMALLMSG) - goto fn_exit; /* room for a message */ - lseqno = *csend->lseqno; - if (lseqno == csend->seqno) - goto fn_exit; /* remote side has consumed everything */ - assert(lseqno < csend->seqno); - retval = 0; - fn_exit: - return retval; -} - -ssize_t MPID_nem_scif_writev(int ep, shmchan_t * c, const struct iovec * iov, int iov_cnt, - uint64_t * seqno) -{ - size_t nwritten = 0; - int did_dma = 0; - int i; - int mark; - regmem_t *rp; - - for (i = 0; i < iov_cnt; ++i) { - size_t len; - int smallmsg = 0; - size_t iovlen; - size_t avail; - - iovlen = iov[i].iov_len; - if (iovlen == 0) - continue; - avail = c->buflen - (c->curp - c->bufp) - sizeof(uint64_t); - if (iovlen >= SMALLMSG) - len = 2 * sizeof(int64_t); - else { - smallmsg = 1; - len = sizeof(int64_t) + ((iovlen + 7) & ~7); - } - if (len > avail) { - uint64_t lseqno = *c->lseqno; - *(int64_t *) c->curp = ~0l; - if (lseqno != c->seqno) { - if (nwritten == 0) - fprintf(stderr, "%5d: %08ld: lseqno %08ld len %ld writev reset buffer\n", - getpid(), c->seqno, lseqno, len); - assert(lseqno < c->seqno); - goto fn_exit; - } - /* Remote side has consumed everything, we can reset the buffer */ - c->curp = c->bufp + 2 * sizeof(uint64_t); - } - if (smallmsg) { - *(int64_t *) c->curp = iovlen; - memcpy(c->curp + sizeof(int64_t), iov[i].iov_base, iovlen); - } - else { - did_dma = 1; - rp = regmem(ep, c, iov[i].iov_base, iovlen); - if (rp == 0) { - nwritten = -1; - goto fn_exit; - } - *(int64_t *) c->curp = -iovlen; - ((uint64_t *) c->curp)[1] = rp->offset + ((char *) iov[i].iov_base - rp->base); - } - c->curp += len; - nwritten += iovlen; - ++c->seqno; - - if (did_dma) { - rp->seqno = c->seqno; - ++c->dma_count; - } - } - fn_exit: -#if !defined(__MIC__) - _mm_mfence(); -#endif - /* Tell the other side it's available */ - *c->rseqno = c->seqno; - if (did_dma) - *seqno = c->seqno; - else - *seqno = 0; - if (nwritten <= 0) - fprintf(stderr, "%5d: %08ld: writev err %ld\n", getpid(), c->seqno, nwritten); - return nwritten; -} diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.h b/src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.h deleted file mode 100644 index f371fc0..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.h +++ /dev/null @@ -1,88 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Portions of this code were written by Intel Corporation. - * Copyright (C) 2011-2012 Intel Corporation. Intel provides this material - * to Argonne National Laboratory subject to Software Grant and Corporate - * Contributor License Agreement dated February 8, 2012. - */ - -#ifndef SCIFRW_H -#define SCIFRW_H - -#define SHMBUFSIZE (64*1024) -#define DMABUFSIZE (1024*1024) -#define SMALLMSG (4096) - -#define NPOLLS (100) -#define CACHE_LINESIZE (64) - -/* registered memory table entry */ -typedef struct regmem { - char *base; - size_t size; - off_t offset; /* for DMA */ - uint64_t seqno; /* sender sequence number, */ - /* used for unreg */ - struct regmem *next; -} regmem_t; - -/* Shared memory channel */ -typedef struct { - uint64_t seqno; /* my current sequence number */ - off_t offset; /* for DMA */ - /* Sequence numbers. rseqno is remote for receiver, lseqno is - * remote for sender. - */ - volatile uint64_t *rseqno; - volatile uint64_t *lseqno; - char *bufp; /* the mmap'd buffer */ - size_t buflen; /* and its size */ - char *curp; /* where I am currently */ - ssize_t pos; /* position in current packet (receive only) */ - char *dmabuf; /* buffer for large DMA */ - size_t dmalen; /* and its size */ - off_t dmaoffset; /* offset for DMA buffer */ - ssize_t dmastart; /* start of the read in the DMA buffer */ - ssize_t dmaend; /* end of the read in the DMA buffer */ - regmem_t *reg; /* registration list */ - int rank; /* mostly for debugging */ - int dma_count; /* count of DMA ops (send only) */ - int dma_chdseqno; /* last checked segno */ -} shmchan_t; - -int MPID_nem_scif_init_shmsend(shmchan_t * csend, int ep, int rank); -int MPID_nem_scif_init_shmrecv(shmchan_t * crecv, int ep, off_t offs, int rank); - -ssize_t MPID_nem_scif_read(int ep, shmchan_t * c, void *recv_buf, size_t len); -ssize_t MPID_nem_scif_readv(int ep, shmchan_t * c, const struct iovec *iov, int iov_cnt); -ssize_t MPID_nem_scif_writev(int ep, shmchan_t * c, const struct iovec *iov, int iov_cnt, - uint64_t * seqno); - -int MPID_nem_scif_poll_send(int ep, shmchan_t * csend); - -static inline int MPID_nem_scif_poll_recv(shmchan_t * crecv) -{ - uint64_t rseqno = *crecv->rseqno; - return crecv->pos != -1 || crecv->seqno < rseqno; -} - - -static inline int MPID_nem_scif_chk_seqno(shmchan_t * csend, int seqno) -{ - return *csend->lseqno >= seqno; -} - -static inline int MPID_nem_scif_chk_dma_unreg(shmchan_t * csend) -{ - uint64_t lseqno = *csend->lseqno; - - if (lseqno != csend->dma_chdseqno) { - csend->dma_chdseqno = lseqno; - return csend->dma_count; - } - return 0; -} -#endif diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/scifsm.c b/src/mpid/ch3/channels/nemesis/netmod/scif/scifsm.c deleted file mode 100644 index e2dd96b..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/scifsm.c +++ /dev/null @@ -1,201 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Portions of this code were written by Intel Corporation. - * Copyright (C) 2011-2012 Intel Corporation. Intel provides this material - * to Argonne National Laboratory subject to Software Grant and Corporate - * Contributor License Agreement dated February 8, 2012. - */ - -#include "scif_impl.h" - -#undef FUNCNAME -#define FUNCNAME MPID_nem_scif_recv_handler -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -static int MPID_nem_scif_recv_handler(scifconn_t * const sc) -{ - const int sc_fd = sc->fd; - shmchan_t *const sc_ch = &sc->crecv; - MPIDI_VC_t *const sc_vc = sc->vc; - int mpi_errno = MPI_SUCCESS; - ssize_t bytes_recvd; - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_SCIF_RECV_HANDLER); - - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_SCIF_RECV_HANDLER); - - if (sc_vc->ch.recv_active == NULL) { - /* receive a new message */ - bytes_recvd = - MPID_nem_scif_read(sc_fd, sc_ch, MPID_nem_scif_recv_buf, - MPID_NEM_SCIF_RECV_MAX_PKT_LEN); - MPIU_ERR_CHKANDJUMP1(bytes_recvd <= 0, mpi_errno, MPI_ERR_OTHER, - "**scif_read", "**scif_scif_read %s", MPIU_Strerror(errno)); - - MPIU_DBG_MSG_FMT(CH3_CHANNEL, VERBOSE, - (MPIU_DBG_FDEST, - "New recv " MPIDI_MSG_SZ_FMT - " (fd=%d, vc=%p, sc=%p)", bytes_recvd, sc_fd, sc_vc, sc)); - - mpi_errno = MPID_nem_handle_pkt(sc_vc, MPID_nem_scif_recv_buf, bytes_recvd); - if (mpi_errno) - MPIU_ERR_POP_LABEL(mpi_errno, fn_noncomm_fail); - } - else { - /* there is a pending receive, receive it directly into the - * user buffer */ - MPIDI_CH3I_VC *const sc_vc_ch = &sc_vc->ch; - MPID_Request *const rreq = sc_vc_ch->recv_active; - MPID_IOV *iov = &rreq->dev.iov[rreq->dev.iov_offset]; - int (*reqFn) (MPIDI_VC_t *, MPID_Request *, int *); - - MPIU_Assert(rreq->dev.iov_count > 0); - MPIU_Assert(rreq->dev.iov_offset >= 0); - MPIU_Assert(rreq->dev.iov_count + rreq->dev.iov_offset <= MPID_IOV_LIMIT); - - bytes_recvd = MPID_nem_scif_readv(sc_fd, sc_ch, iov, rreq->dev.iov_count); - MPIU_ERR_CHKANDJUMP1(bytes_recvd <= 0, mpi_errno, MPI_ERR_OTHER, - "**scif_readv", "**scif_scif_readv %s", MPIU_Strerror(errno)); - - MPIU_DBG_MSG_D(CH3_CHANNEL, VERBOSE, "Cont recv %ld", bytes_recvd); - - /* update the iov */ - for (iov = &rreq->dev.iov[rreq->dev.iov_offset]; - iov < &rreq->dev.iov[rreq->dev.iov_offset + rreq->dev.iov_count]; ++iov) { - if (bytes_recvd < iov->MPID_IOV_LEN) { - iov->MPID_IOV_BUF = (char *) iov->MPID_IOV_BUF + bytes_recvd; - iov->MPID_IOV_LEN -= bytes_recvd; - rreq->dev.iov_count = - &rreq->dev.iov[rreq->dev.iov_offset + rreq->dev.iov_count] - iov; - rreq->dev.iov_offset = iov - rreq->dev.iov; - MPIU_DBG_MSG_D(CH3_CHANNEL, VERBOSE, "bytes_recvd = %ld", (long int) bytes_recvd); - MPIU_DBG_MSG_D(CH3_CHANNEL, VERBOSE, "iov len = %ld", (long int) iov->MPID_IOV_LEN); - MPIU_DBG_MSG_D(CH3_CHANNEL, VERBOSE, "iov_offset = %d", rreq->dev.iov_offset); - goto fn_exit; - } - bytes_recvd -= iov->MPID_IOV_LEN; - } - - /* the whole iov has been received */ - - reqFn = rreq->dev.OnDataAvail; - if (!reqFn) { - MPIU_Assert(MPIDI_Request_get_type(rreq) != MPIDI_REQUEST_TYPE_GET_RESP); - MPIDI_CH3U_Request_complete(rreq); - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "...complete"); - sc_vc_ch->recv_active = NULL; - } - else { - int complete = 0; - - mpi_errno = reqFn(sc_vc, rreq, &complete); - if (mpi_errno) - MPIU_ERR_POP_LABEL(mpi_errno, fn_noncomm_fail); - - if (complete) { - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "...complete"); - sc_vc_ch->recv_active = NULL; - } - else { - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "...not complete"); - } - } - } - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_SCIF_RECV_HANDLER); - return mpi_errno; - fn_fail: /* comm related failures jump here */ - { - MPIU_ERR_SET1(mpi_errno, MPI_ERR_OTHER, "**comm_fail", "**comm_fail %d", sc_vc->pg_rank); - } - fn_noncomm_fail: /* NON-comm related failures jump here */ - goto fn_exit; - -} - -#undef FUNCNAME -#define FUNCNAME state_commrdy_handler -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -static int state_commrdy_handler(int is_readable, int is_writeable, scifconn_t * const sc) -{ - int mpi_errno = MPI_SUCCESS; - MPIDI_VC_t *sc_vc = sc->vc; - MPID_nem_scif_vc_area *sc_vc_scif = VC_SCIF(sc_vc); - MPIDI_STATE_DECL(MPID_STATE_STATE_COMMRDY_HANDLER); - - MPIDI_FUNC_ENTER(MPID_STATE_STATE_COMMRDY_HANDLER); - - if (is_readable) { - mpi_errno = MPID_nem_scif_recv_handler(sc); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - } - if (is_writeable && !MPIDI_CH3I_Sendq_empty(sc_vc_scif->send_queue)) { - mpi_errno = MPID_nem_scif_send_queued(sc_vc, &sc_vc_scif->send_queue); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - /* check to see if this VC is waiting for outstanding sends to - * finish in order to terminate */ - if (MPIDI_CH3I_Sendq_empty(sc_vc_scif->send_queue) && sc_vc_scif->terminate == 1) { - /* The sendq is empty, so we can immediately terminate - * the connection. */ - mpi_errno = MPID_nem_scif_vc_terminated(sc_vc); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - } - } - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_STATE_COMMRDY_HANDLER); - return mpi_errno; - fn_fail: - goto fn_exit; -} - -int MPID_nem_scif_connpoll(int in_blocking_poll) -{ - int mpi_errno = MPI_SUCCESS, i; - static int npolls = 0; - - /* poll the real scif channels */ - for (i = 0; i < MPID_nem_scif_nranks; ++i) { - int is_readable; - int is_writeable; - - if (i == MPID_nem_scif_myrank) - continue; - scifconn_t *it_sc = &MPID_nem_scif_conns[i]; - if (it_sc->fd == -1) - continue; /* no connection */ - if (MPID_nem_scif_chk_dma_unreg(&it_sc->csend)) { - MPID_nem_scif_unregmem(it_sc->fd, &it_sc->csend); - } - is_readable = MPID_nem_scif_poll_recv(&it_sc->crecv); - is_writeable = MPID_nem_scif_poll_send(it_sc->fd, &it_sc->csend); - MPIU_ERR_CHKANDJUMP1(is_writeable == -1, mpi_errno, MPI_ERR_OTHER, - "**scif_poll_send", "**scif_poll_send %s", MPIU_Strerror(errno)); - mpi_errno = state_commrdy_handler(is_readable, is_writeable, it_sc); - if (mpi_errno) - MPIU_ERR_POP(mpi_errno); - if (npolls >= NPOLLS) { - struct pollfd fds; - fds.fd = it_sc->fd; - fds.events = POLLIN; - fds.revents = 0; - MPIU_ERR_CHKANDJUMP1(poll(&fds, 1, 0) < 0, mpi_errno, MPI_ERR_OTHER, - "**poll", "**poll %s", MPIU_Strerror(errno)); - MPIU_ERR_CHKANDJUMP(fds.revents & POLLERR, mpi_errno, MPI_ERR_OTHER, "**poll"); - } - } - if (npolls++ >= NPOLLS) - npolls = 0; - - fn_exit: - return mpi_errno; - fn_fail: - MPIU_DBG_MSG_FMT(NEM_SOCK_DET, VERBOSE, (MPIU_DBG_FDEST, "failure. mpi_errno = %d", mpi_errno)); - goto fn_exit; -} diff --git a/src/mpid/ch3/channels/nemesis/netmod/scif/subconfigure.m4 b/src/mpid/ch3/channels/nemesis/netmod/scif/subconfigure.m4 deleted file mode 100644 index 7a314fd..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/scif/subconfigure.m4 +++ /dev/null @@ -1,23 +0,0 @@ -[#] start of __file__ -dnl MPICH_SUBCFG_AFTER=src/mpid/ch3/channels/nemesis - -AC_DEFUN([PAC_SUBCFG_PREREQ_]PAC_SUBCFG_AUTO_SUFFIX,[ - AM_COND_IF([BUILD_CH3_NEMESIS],[ - for net in $nemesis_networks ; do - AS_CASE([$net],[scif],[build_nemesis_netmod_scif=yes]) - done - ]) - AM_CONDITIONAL([BUILD_NEMESIS_NETMOD_SCIF],[test "X$build_nemesis_netmod_scif" = "Xyes"]) - - # check if getpagesize is available - AC_CHECK_FUNCS(getpagesize) -])dnl - -AC_DEFUN([PAC_SUBCFG_BODY_]PAC_SUBCFG_AUTO_SUFFIX,[ -AM_COND_IF([BUILD_NEMESIS_NETMOD_SCIF],[ - AC_MSG_NOTICE([RUNNING CONFIGURE FOR ch3:nemesis:scif]) - PAC_SET_HEADER_LIB_PATH(scif) -])dnl end AM_COND_IF(BUILD_NEMESIS_NETMOD_SCIF,...) -])dnl end _BODY - -[#] end of __file__ http://git.mpich.org/mpich.git/commitdiff/8395e55f093b02c783a0e71cdb28421e7… commit 8395e55f093b02c783a0e71cdb28421e76dc399f Author: Pavan Balaji <balaji(a)anl.gov> Date: Tue May 12 13:17:43 2015 -0500 remove the newmad netmod. Signed-off-by: Ken Raffenetti <raffenet(a)mcs.anl.gov> diff --git a/src/mpid/ch3/channels/nemesis/netmod/Makefile.mk b/src/mpid/ch3/channels/nemesis/netmod/Makefile.mk index c55c980..1fc79f6 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/Makefile.mk +++ b/src/mpid/ch3/channels/nemesis/netmod/Makefile.mk @@ -8,7 +8,6 @@ include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/tcp/Makefile.mk include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/none/Makefile.mk -include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/newmad/Makefile.mk include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/scif/Makefile.mk include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/portals4/Makefile.mk include $(top_srcdir)/src/mpid/ch3/channels/nemesis/netmod/ib/Makefile.mk diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/Makefile.mk b/src/mpid/ch3/channels/nemesis/netmod/newmad/Makefile.mk deleted file mode 100644 index 097b26f..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/Makefile.mk +++ /dev/null @@ -1,26 +0,0 @@ -## -*- Mode: Makefile; -*- -## vim: set ft=automake : -## -## (C) 2012 by Argonne National Laboratory. -## See COPYRIGHT in top-level directory. -## - -if BUILD_NEMESIS_NETMOD_NEWMAD - -mpi_core_sources += \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_finalize.c \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_init.c \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_poll.c \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_send.c \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_register.c \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_test.c \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_cancel.c \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_probe.c \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_alloc.c - -noinst_HEADERS += \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_impl.h \ - src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_extended_interface.h - -endif BUILD_NEMESIS_NETMOD_NEWMAD - diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_alloc.c b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_alloc.c deleted file mode 100644 index cdfa1a7..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_alloc.c +++ /dev/null @@ -1,133 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#include "newmad_impl.h" - -#define NUM_REQ (1024) - -static MPID_nem_newmad_internal_req_t *MPID_nem_newmad_internal_req_queue_head; - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_internal_req_queue_init -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_internal_req_queue_init(void) -{ - MPID_nem_newmad_internal_req_t *curr_req; - MPID_nem_newmad_internal_req_t *next_req; - int mpi_errno = MPI_SUCCESS; - int index; - - MPID_nem_newmad_internal_req_queue_head = (MPID_nem_newmad_internal_req_t *)MPIU_Malloc(sizeof(MPID_nem_newmad_internal_req_t)); - curr_req = MPID_nem_newmad_internal_req_queue_head; - - for(index = 0 ; index < (NUM_REQ - 1) ; index++) - { - next_req = (MPID_nem_newmad_internal_req_t *)MPIU_Malloc(sizeof(MPID_nem_newmad_internal_req_t)); - curr_req->next = next_req; - curr_req = next_req; - } - curr_req->next = NULL; - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_internal_req_queue_destroy -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_internal_req_queue_destroy(void) -{ - MPID_nem_newmad_internal_req_t *curr_req; - MPID_nem_newmad_internal_req_t *req_to_del; - int mpi_errno = MPI_SUCCESS; - - curr_req = MPID_nem_newmad_internal_req_queue_head; - - if(curr_req->next == NULL) - { - MPIU_Free(curr_req); - } - else - { - do - { - req_to_del = curr_req; - curr_req = curr_req->next; - MPIU_Free(req_to_del); - } - while(curr_req != NULL); - } - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_internal_req_dequeue -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_internal_req_dequeue(MPID_nem_newmad_internal_req_t **param_req) -{ - int mpi_errno = MPI_SUCCESS; - - *param_req = MPID_nem_newmad_internal_req_queue_head; - MPID_nem_newmad_internal_req_queue_head = (*param_req)->next; - (*param_req)->next = NULL; - - if(MPID_nem_newmad_internal_req_queue_head == NULL) - { - MPID_nem_newmad_internal_req_t *curr_req; - MPID_nem_newmad_internal_req_t *next_req; - int index; - - MPID_nem_newmad_internal_req_queue_head = (MPID_nem_newmad_internal_req_t *)MPIU_Malloc(sizeof(MPID_nem_newmad_internal_req_t)); - curr_req = MPID_nem_newmad_internal_req_queue_head; - for(index = 0 ; index < (NUM_REQ - 1) ; index++) - { - next_req = (MPID_nem_newmad_internal_req_t *)MPIU_Malloc(sizeof(MPID_nem_newmad_internal_req_t)); - curr_req->next = next_req; - curr_req = next_req; - } - curr_req->next = NULL; - } - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_internal_req_enqueue -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_internal_req_enqueue(MPID_nem_newmad_internal_req_t *req) -{ - int mpi_errno = MPI_SUCCESS; - - MPIU_Assert(MPID_nem_newmad_internal_req_queue_head != NULL); - req->next = MPID_nem_newmad_internal_req_queue_head; - MPID_nem_newmad_internal_req_queue_head = req; - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_cancel.c b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_cancel.c deleted file mode 100644 index 7e479b9..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_cancel.c +++ /dev/null @@ -1,70 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#include "newmad_impl.h" -#include "my_papi_defs.h" - -#undef FUNCNAME -#define FUNCNAME MPID_nem_mx_cancel_send -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_cancel_send(MPIDI_VC_t *vc, MPID_Request *sreq) -{ - nm_sr_request_t *nmad_req = NULL; - int mpi_errno = MPI_SUCCESS; - int ret; - - nmad_req = &(REQ_FIELD(sreq,newmad_req)); - ret = nm_sr_scancel(mpid_nem_newmad_session,nmad_req); - - if (ret == NM_ESUCCESS) - { - MPIR_STATUS_SET_CANCEL_BIT(sreq->status, TRUE); - (VC_FIELD(vc,pending_sends)) -= 1; - } - else - { - MPIR_STATUS_SET_CANCEL_BIT(sreq->status, FALSE); - } - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_cancel_recv -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_cancel_recv(MPIDI_VC_t *vc, MPID_Request *rreq) -{ - nm_sr_request_t *nmad_req = NULL; - int mpi_errno = MPI_SUCCESS; - int ret; - - nmad_req = &(REQ_FIELD(rreq,newmad_req)); - ret = nm_sr_rcancel(mpid_nem_newmad_session,nmad_req); - - if (ret == NM_ESUCCESS) - { - MPIR_STATUS_SET_CANCEL_BIT(rreq->status, TRUE); - } - else - { - MPIR_STATUS_SET_CANCEL_BIT(rreq->status, FALSE); - } - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_extended_interface.h b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_extended_interface.h deleted file mode 100644 index 3e5afa7..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_extended_interface.h +++ /dev/null @@ -1,75 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#ifndef NMNAD_EXTENDED_INTERFACE_H -#define NMNAD_EXTENDED_INTERFACE_H - -#include "mpid_nem_impl.h" - -static inline int nm_sr_irecv_with_ref_tagged(nm_session_t p_session, - nm_gate_t p_gate, - nm_tag_t tag, nm_tag_t mask, - void *data, uint32_t len, - nm_sr_request_t *p_request, - void *ref) -{ - nm_sr_recv_init(p_session, p_request); - nm_sr_request_set_ref(p_session, p_request, ref); - nm_sr_recv_unpack_data(p_session, p_request, data, len); - if(ref != NULL) - nm_sr_request_set_completion_queue(p_session, p_request); - const int err = nm_sr_recv_irecv(p_session, p_request, p_gate, tag, mask); - return err; -} - -static inline int nm_sr_irecv_iov_with_ref_tagged(nm_session_t p_session, - nm_gate_t p_gate, - nm_tag_t tag, nm_tag_t mask, - struct iovec *iov, int num_entries, - nm_sr_request_t *p_request, void *ref) -{ - nm_sr_recv_init(p_session, p_request); - nm_sr_request_set_ref(p_session, p_request, ref); - nm_sr_recv_unpack_iov(p_session, p_request, iov, num_entries); - if(ref != NULL) - nm_sr_request_set_completion_queue(p_session, p_request); - const int err = nm_sr_recv_irecv(p_session, p_request, p_gate, tag, mask); - return err; -} - - -static inline int nm_sr_issend_iov(nm_session_t p_session, - nm_gate_t p_gate, nm_tag_t tag, - const struct iovec *iov, int num_entries, - nm_sr_request_t *p_request) -{ - nm_sr_send_init(p_session, p_request); - nm_sr_send_pack_iov(p_session, p_request, iov, num_entries); - const int err = nm_sr_send_issend(p_session, p_request, p_gate, tag); - return err; -} - -static inline int nm_sr_issend_iov_with_ref(nm_session_t p_session, - nm_gate_t p_gate, nm_tag_t tag, - const struct iovec *iov, int num_entries, - nm_sr_request_t *p_request, - void*ref) -{ - nm_sr_send_init(p_session, p_request); - nm_sr_request_set_ref(p_session, p_request, ref); - nm_sr_send_pack_iov(p_session, p_request, iov, num_entries); - if(ref != NULL) - nm_sr_request_set_completion_queue(p_session, p_request); - const int err = nm_sr_send_issend(p_session, p_request, p_gate, tag); - return err; -} - - -#endif diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_finalize.c b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_finalize.c deleted file mode 100644 index 7faf49b..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_finalize.c +++ /dev/null @@ -1,46 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#include "newmad_impl.h" - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_finalize -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int -MPID_nem_newmad_finalize() -{ - int mpi_errno = MPI_SUCCESS; - - nm_session_destroy(mpid_nem_newmad_session); - - MPID_nem_newmad_internal_req_queue_destroy(); - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_ckpt_shutdown -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int -MPID_nem_newmad_ckpt_shutdown () -{ - int mpi_errno = MPI_SUCCESS; - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_impl.h b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_impl.h deleted file mode 100644 index 089e52d..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_impl.h +++ /dev/null @@ -1,214 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#ifndef NEWMAD_MODULE_IMPL_H -#define NEWMAD_MODULE_IMPL_H -#ifdef USE_PMI2_API -#include "pmi2.h" -#else -#include "pmi.h" -#endif -#include <sys/types.h> -#include <sys/socket.h> -#include <stdint.h> -#include <nm_public.h> -#include <nm_sendrecv_interface.h> -#include <nm_sampling.h> -#include "mpid_nem_impl.h" - -int MPID_nem_newmad_init (MPIDI_PG_t *pg_p, int pg_rank, char **bc_val_p, int *val_max_sz_p); -int MPID_nem_newmad_finalize (void); -int MPID_nem_newmad_poll(int in_blocking_progress); -int MPID_nem_newmad_get_business_card (int my_rank, char **bc_val_p, int *val_max_sz_p); -int MPID_nem_newmad_connect_to_root (const char *business_card, MPIDI_VC_t *new_vc); -int MPID_nem_newmad_vc_init (MPIDI_VC_t *vc); -int MPID_nem_newmad_vc_destroy(MPIDI_VC_t *vc); -int MPID_nem_newmad_vc_terminate (MPIDI_VC_t *vc); - -/* alternate interface */ -int MPID_nem_newmad_iSendContig(MPIDI_VC_t *vc, MPID_Request *sreq, void *hdr, MPIDI_msg_sz_t hdr_sz, - void *data, MPIDI_msg_sz_t data_sz); -int MPID_nem_newmad_iStartContigMsg(MPIDI_VC_t *vc, void *hdr, MPIDI_msg_sz_t hdr_sz, void *data, - MPIDI_msg_sz_t data_sz, MPID_Request **sreq_ptr); -int MPID_nem_newmad_SendNoncontig(MPIDI_VC_t *vc, MPID_Request *sreq, void *header, MPIDI_msg_sz_t hdr_sz); - -/* Direct Routines */ -int MPID_nem_newmad_directSend(MPIDI_VC_t *vc, const void * buf, int count, MPI_Datatype datatype, int dest, int tag, - MPID_Comm * comm, int context_offset, MPID_Request **sreq_p); -int MPID_nem_newmad_directSsend(MPIDI_VC_t *vc, const void * buf, int count, MPI_Datatype datatype, int dest, int tag, - MPID_Comm * comm, int context_offset,MPID_Request **sreq_p); -int MPID_nem_newmad_directRecv(MPIDI_VC_t *vc, MPID_Request *rreq); -int MPID_nem_newmad_cancel_send(MPIDI_VC_t *vc, MPID_Request *sreq); -int MPID_nem_newmad_cancel_recv(MPIDI_VC_t *vc, MPID_Request *rreq); -int MPID_nem_newmad_probe(MPIDI_VC_t *vc, int source, int tag, MPID_Comm *comm, - int context_offset, MPI_Status *status); -int MPID_nem_newmad_iprobe(MPIDI_VC_t *vc, int source, int tag, MPID_Comm *comm, - int context_offset, int *flag, MPI_Status *status); -int MPID_nem_newmad_improbe(MPIDI_VC_t *vc, int source, int tag, MPID_Comm *comm, int context_offset, int *flag, MPID_Request **message, MPI_Status *status); -/* Any source management */ -void MPID_nem_newmad_anysource_posted(MPID_Request *rreq); -int MPID_nem_newmad_anysource_matched(MPID_Request *rreq); -int MPID_nem_newmad_anysource_iprobe(int tag, MPID_Comm *comm, int context_offset, int *flag, MPI_Status *status); -int MPID_nem_newmad_anysource_improbe(int tag, MPID_Comm *comm, int context_offset, int *flag, - MPID_Request **message,MPI_Status *status); -/* Callbacks for events */ -void MPID_nem_newmad_get_adi_msg(nm_sr_event_t event, const nm_sr_event_info_t*info); - -/* Dtype management */ -int MPID_nem_newmad_process_sdtype(MPID_Request **sreq_p, MPI_Datatype datatype, MPID_Datatype * dt_ptr, const void *buf, - int count, MPIDI_msg_sz_t data_sz, struct iovec **newmad_iov, int *num_iov, int first_taken); -int MPID_nem_newmad_process_rdtype(MPID_Request **rreq_p, MPID_Datatype * dt_ptr, MPIDI_msg_sz_t data_sz,struct iovec **newmad_iov, - int *num_iov); - -/* Connection management*/ -int MPID_nem_newmad_send_conn_info (MPIDI_VC_t *vc); - -#define MPID_NEM_NMAD_MAX_SIZE 64 -typedef nm_gate_t mpid_nem_newmad_p_gate_t; - -/* The current max size for the whole structure is 128 bytes */ -typedef struct -{ - char url[MPID_NEM_NMAD_MAX_SIZE]; - mpid_nem_newmad_p_gate_t p_gate; - int pending_sends; -} MPID_nem_newmad_vc_area; -/* accessor macro to private fields in VC */ -#define VC_FIELD(vcp, field) (((MPID_nem_newmad_vc_area *)vcp->ch.netmod_area.padding)->field) - -/* The req provides a generic buffer in which network modules can store - private fields This removes all dependencies from the req structure - on the network module, facilitating dynamic module loading. */ -typedef struct -{ - nm_sr_request_t newmad_req; - struct iovec *iov; - int iov_to_delete; - int deltmpbuf; -} MPID_nem_newmad_req_area; - -#define TMP_DEL_VALUE 111 - -/* accessor macro to private fields in REQ */ -#define REQ_FIELD(reqp, field) (((MPID_nem_newmad_req_area *)((reqp)->ch.netmod_area.padding))->field) - -/* The begining of this structure is the same as MPID_Request */ -struct MPID_nem_newmad_internal_req -{ - MPIU_OBJECT_HEADER; /* adds (unused) handle and ref_count fields */ - MPID_Request_kind_t kind; /* used */ - MPIDI_CH3_Pkt_t pending_pkt; - MPIDI_VC_t *vc; - void *tmpbuf; - MPIDI_msg_sz_t tmpbuf_sz; - nm_sr_request_t newmad_req; - struct MPID_nem_newmad_internal_req *next; -}; - -typedef struct MPID_nem_newmad_internal_req MPID_nem_newmad_internal_req_t; - -typedef union -{ - MPID_nem_newmad_internal_req_t nem_newmad_req; - MPID_Request mpi_req; -} MPID_nem_newmad_unified_req_t ; - -/* Internal Reqs management */ -int MPID_nem_newmad_internal_req_queue_init(void); -int MPID_nem_newmad_internal_req_queue_destroy(void); -int MPID_nem_newmad_internal_req_dequeue(MPID_nem_newmad_internal_req_t **req); -int MPID_nem_newmad_internal_req_enqueue(MPID_nem_newmad_internal_req_t *req); - -#if CH3_RANK_BITS == 16 -#define NBITS_TAG 32 -typedef int32_t Nmad_Nem_tag_t; -#elif CH3_RANK_BITS == 32 -#define NBITS_TAG 16 -typedef int16_t Nmad_Nem_tag_t; -#endif /* CH3_RANK_BITS */ - -#define NBITS_RANK CH3_RANK_BITS -#define NBITS_CTXT 16 -#define NBITS_PGRANK (sizeof(int)*8) - -#define NEM_NMAD_MATCHING_BITS (NBITS_TAG+NBITS_RANK+NBITS_CTXT) -#define SHIFT_TAG (NBITS_RANK+NBITS_CTXT) -#define SHIFT_RANK (NBITS_CTXT) -#define SHIFT_PGRANK (NBITS_CTXT) -#define SHIFT_CTXT (0) - -#define NEM_NMAD_MAX_TAG ((UINT64_C(1)<<NBITS_TAG) -1) -#define NEM_NMAD_MAX_RANK ((UINT64_C(1)<<NBITS_RANK) -1) -#define NEM_NMAD_MAX_CTXT ((UINT64_C(1)<<NBITS_CTXT) -1) -#define NEM_NMAD_MAX_PGRANK ((UINT64_C(1)<<NBITS_PGRANK)-1) - -#define NEM_NMAD_TAG_MASK (NEM_NMAD_MAX_TAG <<SHIFT_TAG ) -#define NEM_NMAD_RANK_MASK (NEM_NMAD_MAX_RANK<<SHIFT_RANK) -#define NEM_NMAD_CTXT_MASK (NEM_NMAD_MAX_CTXT<<SHIFT_CTXT) -#define NEM_NMAD_PGRANK_MASK (NEM_NMAD_MAX_PGRANK<<SHIFT_PGRANK) - -#define NEM_NMAD_MATCH_FULL_MASK (UINT64_C(0xffffffffffffffff)) -#define NEM_NMAD_MATCH_EMPTY_MASK (UINT64_C(0x0000000000000000)) - -#define NEM_NMAD_SET_TAG(_match, _tag) do { \ - MPIU_Assert((_tag >= 0)&&(_tag <= (NEM_NMAD_MAX_TAG))); \ - ((_match) |= (((nm_tag_t)((_tag)&(NEM_NMAD_MAX_TAG))) << SHIFT_TAG)); \ - }while(0) -#define NEM_NMAD_SET_SRC(_match, _src) do { \ - MPIU_Assert((_src >= 0)&&(_src<=(NEM_NMAD_MAX_RANK)));\ - ((_match) |= (((nm_tag_t)(_src)) << SHIFT_RANK)); \ - }while(0) -#define NEM_NMAD_SET_CTXT(_match, _ctxt) do { \ - MPIU_Assert((_ctxt >= 0)&&(_ctxt<=(NEM_NMAD_MAX_CTXT)));\ - ((_match) |= (((nm_tag_t)(_ctxt)) << SHIFT_CTXT)); \ - }while(0) -#define NEM_NMAD_SET_PGRANK(_match, _pg_rank) do { \ - ((_match) |= (((nm_tag_t)(_pg_rank)) << SHIFT_PGRANK));\ - }while(0) -#define NEM_NMAD_SET_ANYSRC(_match) do{ \ - ((_match) &= ~NEM_NMAD_RANK_MASK); \ - }while(0) -#define NEM_NMAD_SET_ANYTAG(_match) do{ \ - ((_match) &= ~NEM_NMAD_TAG_MASK); \ - }while(0) - - -#define NEM_NMAD_MATCH_GET_TAG(_match, _tag) do{ \ - ((_tag) = ((Nmad_Nem_tag_t)(((_match) & NEM_NMAD_TAG_MASK) >> SHIFT_TAG)));\ - }while(0) -#define NEM_NMAD_MATCH_GET_RANK(_match, _rank) do{ \ - ((_rank) = ((MPIR_Rank_t)(((_match) & NEM_NMAD_RANK_MASK) >> SHIFT_RANK)));\ - }while(0) -#define NEM_NMAD_MATCH_GET_CTXT(_match, _ctxt) do{ \ - ((_ctxt) = ((MPIR_Context_id_t)(((_match) & NEM_NMAD_CTXT_MASK) >> SHIFT_CTXT)));\ - }while(0) -#define NEM_NMAD_MATCH_GET_PGRANK(_match, _pg_rank) do{ \ - ((_pg_rank) = ((int)(((_match) & NEM_NMAD_PGRANK_MASK) >> SHIFT_PGRANK))); \ - }while(0) - -#define NEM_NMAD_INTRA_CTXT (0x0000000c) -#define NEM_NMAD_SET_MATCH(_match,_tag,_rank,_context ) do{ \ - MPIU_Assert((_tag >= 0) &&(_tag <= (NEM_NMAD_MAX_TAG))); \ - MPIU_Assert((_rank >= 0) &&(_rank<=(NEM_NMAD_MAX_RANK))); \ - MPIU_Assert((_context >= 0)&&(_context<=(NEM_NMAD_MAX_CTXT))); \ - (_match)=((((nm_tag_t)(_tag)) << SHIFT_TAG) \ - |(((nm_tag_t)(_rank)) << SHIFT_RANK) \ - |(((nm_tag_t)(_context))<< SHIFT_CTXT)); \ - }while(0) -#define NEM_NMAD_DIRECT_MATCH(_match,_tag,_rank,_context) NEM_NMAD_SET_MATCH(_match,_tag,_rank,_context) -#define NEM_NMAD_ADI_MATCH(_match) NEM_NMAD_SET_MATCH(_match,0,0,NEM_NMAD_INTRA_CTXT) - -extern nm_session_t mpid_nem_newmad_session; - -#define NMAD_IOV_MAX_DEPTH (256) /* NM_SO_PREALLOC_IOV_LEN */ -/*#define DEBUG*/ - -#endif //NEWMAD_MODULE_IMPL_H - diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_init.c b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_init.c deleted file mode 100644 index 0256900..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_init.c +++ /dev/null @@ -1,314 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#include "newmad_impl.h" - -MPID_nem_netmod_funcs_t MPIDI_nem_newmad_funcs = { - MPID_nem_newmad_init, - MPID_nem_newmad_finalize, - MPID_nem_newmad_poll, - MPID_nem_newmad_get_business_card, - MPID_nem_newmad_connect_to_root, - MPID_nem_newmad_vc_init, - MPID_nem_newmad_vc_destroy, - MPID_nem_newmad_vc_terminate, - MPID_nem_newmad_anysource_iprobe, - MPID_nem_newmad_anysource_improbe -}; - -static MPIDI_Comm_ops_t comm_ops = { - MPID_nem_newmad_directRecv, /* recv_posted */ - - MPID_nem_newmad_directSend, /* send */ - MPID_nem_newmad_directSend, /* rsend */ - MPID_nem_newmad_directSsend, /* ssend */ - MPID_nem_newmad_directSend, /* isend */ - MPID_nem_newmad_directSend, /* irsend */ - MPID_nem_newmad_directSsend, /* issend */ - - NULL, /* send_init */ - NULL, /* bsend_init */ - NULL, /* rsend_init */ - NULL, /* ssend_init */ - NULL, /* startall */ - - MPID_nem_newmad_cancel_send,/* cancel_send */ - MPID_nem_newmad_cancel_recv, /* cancel_recv */ - - MPID_nem_newmad_probe, /* probe */ - MPID_nem_newmad_iprobe, /* iprobe */ - MPID_nem_newmad_improbe /* improbe */ -}; - -#define MPIDI_CH3I_URL_KEY "url_id" - -static int mpid_nem_newmad_myrank; -static const char *label="mpich"; -static const char *local_session_url = NULL; -nm_session_t mpid_nem_newmad_session; - -#undef FUNCNAME -#define FUNCNAME init_mad -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -static int init_mad( MPIDI_PG_t *pg_p ) -{ - int mpi_errno = MPI_SUCCESS; - char *dummy_argv[2] = {"mpich",NULL}; - int dummy_argc = 1; - int ret; - - MPID_nem_newmad_internal_req_queue_init(); - - ret = nm_session_create(&mpid_nem_newmad_session, label); - MPIU_Assert( ret == NM_ESUCCESS); - - ret = nm_session_init(mpid_nem_newmad_session, &dummy_argc,dummy_argv, &local_session_url); - MPIU_Assert( ret == NM_ESUCCESS); - - ret = nm_sr_monitor(mpid_nem_newmad_session, NM_SR_EVENT_RECV_UNEXPECTED, - &MPID_nem_newmad_get_adi_msg); - MPIU_Assert( ret == NM_ESUCCESS); - - ret = nm_sr_init(mpid_nem_newmad_session); - if(ret != NM_ESUCCESS) { - fprintf(stdout,"nm_sr_init return err = %d\n", ret); - } - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_init_completed -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int -MPID_nem_newmad_init_completed(void) -{ - - int mpi_errno = MPI_SUCCESS ; - /* - int ret; - - ret = nm_sr_monitor(mpid_nem_newmad_session, NM_SR_EVENT_RECV_UNEXPECTED, - &MPID_nem_newmad_get_adi_msg); - MPIU_Assert( ret == NM_ESUCCESS); - */ -fn_exit: - return mpi_errno; -fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_init -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int -MPID_nem_newmad_init (MPIDI_PG_t *pg_p, int pg_rank, - char **bc_val_p, int *val_max_sz_p) -{ - int mpi_errno = MPI_SUCCESS ; - int index; - - /* - fprintf(stdout,"Size of MPID_nem_mad_module_vc_area_internal_t : %i | size of nm_sr_request_t :%i | Size of req_area : %i\n", - sizeof(MPID_nem_newmad_vc_area_internal_t),sizeof(nm_sr_request_t), sizeof(MPID_nem_newmad_req_area)); - */ - - /* - MPIU_Assert( sizeof(MPID_nem_newmad_vc_area_internal_t) <= MPID_NEM_VC_NETMOD_AREA_LEN); - MPIU_Assert( sizeof(MPID_nem_newmad_req_area) <= MPID_NEM_REQ_NETMOD_AREA_LEN); - */ - - - if (sizeof(MPID_nem_newmad_vc_area) > MPID_NEM_VC_NETMOD_AREA_LEN) - { - fprintf(stdout,"===========================================================\n"); - fprintf(stdout,"=== Error : Newmad data structure size is too long ===\n"); - fprintf(stdout,"=== VC netmod area is %4i | Nmad struct size is %4i ===\n", - MPID_NEM_VC_NETMOD_AREA_LEN, sizeof(MPID_nem_newmad_vc_area)); - fprintf(stdout,"===========================================================\n"); - exit(0); - } - - if (sizeof(MPID_nem_newmad_req_area) > MPID_NEM_REQ_NETMOD_AREA_LEN) - { - fprintf(stdout,"===========================================================\n"); - fprintf(stdout,"=== Error : Newmad data structure size is too long ===\n"); - fprintf(stdout,"=== Req netmod area is %4i | Nmad struct size is %4i ===\n", - MPID_NEM_REQ_NETMOD_AREA_LEN, sizeof(MPID_nem_newmad_req_area)); - fprintf(stdout,"===========================================================\n"); - exit(0); - } - - mpid_nem_newmad_myrank = pg_rank; - init_mad(pg_p); - - mpi_errno = MPID_nem_newmad_get_business_card(pg_rank,bc_val_p, val_max_sz_p); - if (mpi_errno) MPIU_ERR_POP (mpi_errno); - - mpi_errno = MPIDI_CH3I_Register_anysource_notification(MPID_nem_newmad_anysource_posted, MPID_nem_newmad_anysource_matched); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - - mpi_errno = MPID_nem_register_initcomp_cb(MPID_nem_newmad_init_completed); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_get_business_card -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int -MPID_nem_newmad_get_business_card (int my_rank, char **bc_val_p, int *val_max_sz_p) -{ - int mpi_errno = MPI_SUCCESS; - int str_errno = MPIU_STR_SUCCESS; - - str_errno = MPIU_Str_add_binary_arg (bc_val_p, val_max_sz_p, MPIDI_CH3I_URL_KEY, local_session_url, strlen(local_session_url)); - if (str_errno) { - MPIU_ERR_CHKANDJUMP(str_errno == MPIU_STR_NOMEM, mpi_errno, MPI_ERR_OTHER, "**buscard_len"); - MPIU_ERR_SETANDJUMP(mpi_errno, MPI_ERR_OTHER, "**buscard"); - } - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_get_from_bc -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int -MPID_nem_newmad_get_from_bc (const char *business_card, char *url) -{ - int mpi_errno = MPI_SUCCESS; - int str_errno = MPIU_STR_SUCCESS; - int len; - - str_errno = MPIU_Str_get_binary_arg (business_card, MPIDI_CH3I_URL_KEY, url, - MPID_NEM_NMAD_MAX_SIZE, &len); - if (str_errno != MPIU_STR_SUCCESS) - { - /* FIXME: create a real error string for this */ - MPIU_ERR_CHKANDJUMP(str_errno, mpi_errno, MPI_ERR_OTHER, "**argstr_hostd"); - } - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_connect_to_root -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int -MPID_nem_newmad_connect_to_root (const char *business_card, MPIDI_VC_t *new_vc) -{ - int mpi_errno = MPI_SUCCESS; - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_vc_init -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int -MPID_nem_newmad_vc_init (MPIDI_VC_t *vc) -{ - MPIDI_CH3I_VC *vc_ch = &vc->ch; - char *business_card; - int mpi_errno = MPI_SUCCESS; - int val_max_sz; - int ret; - -#ifdef USE_PMI2_API - val_max_sz = PMI2_MAX_VALLEN; -#else - mpi_errno = PMI_KVS_Get_value_length_max(&val_max_sz); -#endif - business_card = (char *)MPIU_Malloc(val_max_sz); - mpi_errno = vc->pg->getConnInfo(vc->pg_rank, business_card,val_max_sz,vc->pg); - if (mpi_errno) MPIU_ERR_POP(mpi_errno); - - /* Very important */ - memset(VC_FIELD(vc, url),0,MPID_NEM_NMAD_MAX_SIZE); - VC_FIELD(vc,pending_sends) = 0; - - mpi_errno = MPID_nem_newmad_get_from_bc (business_card, VC_FIELD(vc, url)); - if (mpi_errno) MPIU_ERR_POP (mpi_errno); - - MPIU_Free(business_card); - - ret = nm_session_connect(mpid_nem_newmad_session, &(VC_FIELD(vc,p_gate)), VC_FIELD(vc, url)); - if (ret != NM_ESUCCESS) fprintf(stdout,"nm_session_connect returned ret = %d\n", ret); - - nm_gate_ref_set(VC_FIELD(vc, p_gate),(void*)vc); - MPIDI_CHANGE_VC_STATE(vc, ACTIVE); - - vc->eager_max_msg_sz = 32768; - vc->ready_eager_max_msg_sz = 32768; - vc->rndvSend_fn = NULL; - vc->sendNoncontig_fn = MPID_nem_newmad_SendNoncontig; - vc->comm_ops = &comm_ops; - - vc_ch->iStartContigMsg = MPID_nem_newmad_iStartContigMsg; - vc_ch->iSendContig = MPID_nem_newmad_iSendContig; - - fn_exit: - return mpi_errno; - fn_fail: - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_vc_destroy -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_vc_destroy(MPIDI_VC_t *vc) -{ - int mpi_errno = MPI_SUCCESS; - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_vc_terminate -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_vc_terminate (MPIDI_VC_t *vc) -{ - /* FIXME: Check to make sure that it's OK to terminate the - connection without making sure that all sends have been sent */ - - while((VC_FIELD(vc,pending_sends)) > 0) \ - MPID_nem_newmad_poll(FALSE); - - return MPIDI_CH3U_Handle_connection (vc, MPIDI_VC_EVENT_TERMINATED); -} - diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_poll.c b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_poll.c deleted file mode 100644 index 5a32515..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_poll.c +++ /dev/null @@ -1,596 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#include "newmad_impl.h" -#include "newmad_extended_interface.h" -#include "my_papi_defs.h" -#include "../mx/uthash.h" - -typedef struct mpid_nem_nmad_hash_struct { - MPID_Request *mpid_req_ptr; - nm_sr_request_t *nmad_req_ptr; - UT_hash_handle hh; -}mpid_nem_nmad_hash_t; - -static mpid_nem_nmad_hash_t *mpid_nem_nmad_asreqs = NULL; -#define MPID_MEM_NMAD_ADD_REQ_IN_HASH(_mpi_req,_nmad_req) do{ \ - mpid_nem_nmad_hash_t *s; \ - s = MPIU_Malloc(sizeof(mpid_nem_nmad_hash_t)); \ - s->mpid_req_ptr = (_mpi_req); \ - s->nmad_req_ptr = (_nmad_req); \ - HASH_ADD(hh, mpid_nem_nmad_asreqs, mpid_req_ptr, sizeof(MPID_Request*), s); \ - }while(0) -#define MPID_NEM_NMAD_GET_REQ_FROM_HASH(_mpi_req_ptr,_nmad_req) do{ \ - mpid_nem_nmad_hash_t *s; \ - HASH_FIND(hh, mpid_nem_nmad_asreqs, &(_mpi_req_ptr), sizeof(MPID_Request*), s); \ - if(s){HASH_DELETE(hh, mpid_nem_nmad_asreqs, s); (_nmad_req) = s->nmad_req_ptr; MPIU_Free(s);} else {(_nmad_req) = NULL;} \ - }while(0) - -static int MPID_nem_newmad_handle_rreq(MPID_Request *req, nm_tag_t match_info, size_t size); -static void MPID_nem_newmad_handle_sreq(MPID_Request *req); - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_get_adi_msg -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -void -MPID_nem_newmad_get_adi_msg(nm_sr_event_t event, const nm_sr_event_info_t*info) -{ - nm_tag_t match_info = info->recv_unexpected.tag; - MPIR_Context_id_t ctxt; - -#ifdef DEBUG - fprintf(stdout,"===> Getting ADI MSG match is %lx \n",match_info); -#endif - - NEM_NMAD_MATCH_GET_CTXT(match_info, ctxt); - if(ctxt == NEM_NMAD_INTRA_CTXT) - { - MPID_nem_newmad_internal_req_t *rreq; - mpid_nem_newmad_p_gate_t from = info->recv_unexpected.p_gate; - int length = info->recv_unexpected.len; - void *data; - - MPID_nem_newmad_internal_req_dequeue(&rreq); - rreq->kind = MPID_REQUEST_RECV; - rreq->vc = nm_gate_ref_get(from); - - if(length <= sizeof(MPIDI_CH3_Pkt_t)) - { - data = (char*)&(rreq->pending_pkt); - } - else - { - rreq->tmpbuf = MPIU_Malloc(length); - MPIU_Assert(rreq->tmpbuf); - rreq->tmpbuf_sz = length; - data = (char*)(rreq->tmpbuf); - } - - nm_sr_irecv_with_ref_tagged(mpid_nem_newmad_session, from, match_info, - NEM_NMAD_MATCH_FULL_MASK, data,length, &(rreq->newmad_req),(void *)rreq); - } - return; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_directRecv -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_directRecv(MPIDI_VC_t *vc, MPID_Request *rreq) -{ - int mpi_errno = MPI_SUCCESS; - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_NEWMAD_DIRECTRECV); - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_NEWMAD_DIRECTRECV); - - if (!vc->ch.is_local) - { - nm_tag_t match_info = 0; - nm_tag_t match_mask = NEM_NMAD_MATCH_FULL_MASK; - MPIR_Rank_t source = rreq->dev.match.parts.rank; - MPIR_Context_id_t context = rreq->dev.match.parts.context_id; - Nmad_Nem_tag_t tag = rreq->dev.match.parts.tag; - int ret; - MPIDI_msg_sz_t data_sz; - int dt_contig; - MPI_Aint dt_true_lb; - MPID_Datatype *dt_ptr; - - NEM_NMAD_DIRECT_MATCH(match_info,0,source,context); - if (tag != MPI_ANY_TAG) - { - NEM_NMAD_SET_TAG(match_info,tag); - } - else - { - NEM_NMAD_SET_ANYTAG(match_info); - NEM_NMAD_SET_ANYTAG(match_mask); - } - -#ifdef DEBUG - fprintf(stdout,"========> Posting Recv req %p (match is %lx) \n",rreq,match_info); -#endif - MPIDI_Datatype_get_info(rreq->dev.user_count,rreq->dev.datatype, dt_contig, data_sz, dt_ptr,dt_true_lb); - rreq->dev.OnDataAvail = NULL; - - if (dt_contig) - { - ret = nm_sr_irecv_with_ref_tagged(mpid_nem_newmad_session,VC_FIELD(vc,p_gate),match_info,match_mask, - (char*)(rreq->dev.user_buf) + dt_true_lb,data_sz, - &(REQ_FIELD(rreq,newmad_req)),(void*)rreq); - REQ_FIELD(rreq,iov) = NULL; - REQ_FIELD(rreq,iov_to_delete) = 0; - } - else - { - int num_seg = 0; - struct iovec *newmad_iov = (struct iovec *)MPIU_Malloc(NMAD_IOV_MAX_DEPTH*sizeof(struct iovec)); - struct iovec *newmad_iov_ptr = &(newmad_iov[0]); - MPID_nem_newmad_process_rdtype(&rreq,dt_ptr,data_sz,&newmad_iov_ptr,&num_seg); - MPIU_Assert(num_seg <= NMAD_IOV_MAX_DEPTH); -#ifdef DEBUG - { - int index; - for(index = 0; index < num_seg ; index++) - { - fprintf(stdout,"======================\n"); - fprintf(stdout,"RECV nmad_iov[%i]: [base %p][len %i]\n",index, - newmad_iov[index].iov_base,newmad_iov[index].iov_len); - } - } -#endif - ret = nm_sr_irecv_iov_with_ref_tagged(mpid_nem_newmad_session,VC_FIELD(vc,p_gate),match_info,match_mask, - newmad_iov,num_seg,&(REQ_FIELD(rreq,newmad_req)),(void*)rreq); - REQ_FIELD(rreq,iov) = newmad_iov; - REQ_FIELD(rreq,iov_to_delete) = 1; - } - } - else - { - /* Fixme : this might not work in the case of multiple netmods */ - memset((&(REQ_FIELD(rreq,newmad_req))),0,sizeof(nm_sr_request_t)); - } - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_NEWMAD_DIRECTRECV); - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_poll -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int -MPID_nem_newmad_poll(int in_blocking_poll) -{ - nm_sr_request_t *p_request = NULL; - nm_tag_t match_info = 0; - int mpi_errno = MPI_SUCCESS; - - nm_sr_send_success(mpid_nem_newmad_session, &p_request); - if (p_request != NULL) - { - MPID_nem_newmad_unified_req_t *ref; - MPID_Request *req; - MPID_Request_kind_t kind; - MPIR_Context_id_t ctxt; - - nm_sr_get_stag(mpid_nem_newmad_session,p_request, &match_info); - NEM_NMAD_MATCH_GET_CTXT(match_info, ctxt); - - nm_sr_get_ref(mpid_nem_newmad_session,p_request,(void *)&ref); - req = &(ref->mpi_req); - MPIU_Assert(req != NULL); - kind = req->kind; - - if(ctxt == NEM_NMAD_INTRA_CTXT) - { - if ((kind == MPID_REQUEST_SEND) || (kind == MPID_PREQUEST_SEND)) - { - MPID_nem_newmad_handle_sreq(req); - } - } - else - { - if ((kind == MPID_REQUEST_SEND) || (kind == MPID_PREQUEST_SEND)) - { - MPIU_Assert(MPIDI_Request_get_type(req) != MPIDI_REQUEST_TYPE_GET_RESP); - MPID_nem_newmad_handle_sreq(req); - } - } - } - - nm_sr_recv_success(mpid_nem_newmad_session, &p_request); - if (p_request != NULL) - { - MPID_nem_newmad_unified_req_t *ref; - MPID_Request *req; - MPID_Request_kind_t kind; - MPIR_Context_id_t ctxt; - size_t size; - - nm_sr_get_ref(mpid_nem_newmad_session,p_request,(void *)&ref); - req = &(ref->mpi_req); - MPIU_Assert(req != NULL); - kind = req->kind; - nm_sr_get_size(mpid_nem_newmad_session, p_request, &size); - nm_sr_get_rtag(mpid_nem_newmad_session,p_request, &match_info); - NEM_NMAD_MATCH_GET_CTXT(match_info, ctxt); - - if(ctxt == NEM_NMAD_INTRA_CTXT) - { - MPID_nem_newmad_internal_req_t *adi_req = &(ref->nem_newmad_req); - if (kind == MPID_REQUEST_RECV) - { - if (size <= sizeof(MPIDI_CH3_Pkt_t)) - { - MPID_nem_handle_pkt(adi_req->vc,(char *)&(adi_req->pending_pkt),(MPIDI_msg_sz_t)(size)); - } - else - { - MPID_nem_handle_pkt(adi_req->vc,(char *)(adi_req->tmpbuf),(MPIDI_msg_sz_t)(adi_req->tmpbuf_sz)); - MPIU_Free(adi_req->tmpbuf); - } - /* nm_core_disable_progression(mpid_nem_newmad_session->p_core); */ - MPID_nem_newmad_internal_req_enqueue(adi_req); - /* nm_core_enable_progression(mpid_nem_newmad_session->p_core); */ - } - else - { - MPIU_Assert(0); - } - } - else - { - if ((kind == MPID_REQUEST_RECV) || (kind == MPID_PREQUEST_RECV)) - { - int found = FALSE; - nm_sr_request_t *nmad_request = NULL; - MPIU_Assert(MPIDI_Request_get_type(req) != MPIDI_REQUEST_TYPE_GET_RESP); - MPIU_THREAD_CS_ENTER(MSGQUEUE,req); - MPID_NEM_NMAD_GET_REQ_FROM_HASH(req,nmad_request); - if(nmad_request != NULL) - { - MPIU_Assert(req->dev.match.parts.rank == MPI_ANY_SOURCE); - MPIU_Free(nmad_request); - } - found = MPIDI_CH3U_Recvq_DP(req); - if(found){ - MPID_nem_newmad_handle_rreq(req,match_info,size); - } - MPIU_THREAD_CS_EXIT(MSGQUEUE,req); - } - else - { - fprintf(stdout, ">>>>>>>>>>>>> ERROR: Wrong req type : %i (%p)\n",(int)kind,req); - MPIU_Assert(0); - } - } - } - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_handle_sreq -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -void -MPID_nem_newmad_handle_sreq(MPID_Request *req) -{ - int (*reqFn)(MPIDI_VC_t *, MPID_Request *, int *); -#ifdef DEBUG - fprintf(stdout,"========> Completing Send req %p \n",req); -#endif - (VC_FIELD(req->ch.vc,pending_sends)) -= 1; - if (((req->dev.datatype_ptr != NULL) && (req->dev.tmpbuf != NULL)) - || (REQ_FIELD(req,deltmpbuf) == TMP_DEL_VALUE)) - { - - MPIU_Free(req->dev.tmpbuf); - REQ_FIELD(req,deltmpbuf) = 0; - } - - reqFn = req->dev.OnDataAvail; - if (!reqFn){ - MPIDI_CH3U_Request_complete(req); - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, ".... complete"); - } - else{ - MPIDI_VC_t *vc = req->ch.vc; - int complete = 0; - reqFn(vc, req, &complete); - if(!complete) - { - MPIU_Assert(complete == TRUE); - } - } -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_new_handle_rreq -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -static int -MPID_nem_newmad_handle_rreq(MPID_Request *req, nm_tag_t match_info, size_t size) -{ - int mpi_errno = MPI_SUCCESS; - int complete = FALSE; - int dt_contig; - MPI_Aint dt_true_lb; - MPIDI_msg_sz_t userbuf_sz; - MPID_Datatype *dt_ptr; - MPIDI_msg_sz_t data_sz; - MPIDI_VC_t *vc = NULL; - -#ifdef DEBUG - fprintf(stdout,"========> Completing Recv req %p (match is %lx) \n",req,match_info); -#endif - - NEM_NMAD_MATCH_GET_RANK(match_info,req->status.MPI_SOURCE); - NEM_NMAD_MATCH_GET_TAG(match_info,req->status.MPI_TAG); - MPIR_STATUS_SET_COUNT(req->status, size); - req->dev.recv_data_sz = size; - - MPIDI_Datatype_get_info(req->dev.user_count, req->dev.datatype, dt_contig, userbuf_sz, dt_ptr, dt_true_lb); - - if (size <= userbuf_sz) { - data_sz = req->dev.recv_data_sz; - } - else - { - MPIU_DBG_MSG_FMT(CH3_OTHER,VERBOSE,(MPIU_DBG_FDEST, - "receive buffer too small; message truncated, msg_sz=" - MPIDI_MSG_SZ_FMT ", userbuf_sz=" - MPIDI_MSG_SZ_FMT, - req->dev.recv_data_sz, userbuf_sz)); - req->status.MPI_ERROR = MPIR_Err_create_code(MPI_SUCCESS, - MPIR_ERR_RECOVERABLE, FCNAME, __LINE__, MPI_ERR_TRUNCATE, - "**truncate", "**truncate %d %d %d %d", - req->status.MPI_SOURCE, req->status.MPI_TAG, - req->dev.recv_data_sz, userbuf_sz ); - MPIR_STATUS_SET_COUNT(req->status, userbuf_sz); - data_sz = userbuf_sz; - } - - if ((!dt_contig)&&(req->dev.tmpbuf != NULL)) - { - MPIDI_msg_sz_t last; - last = req->dev.recv_data_sz; - MPID_Segment_unpack( req->dev.segment_ptr, 0, &last, req->dev.tmpbuf); - MPIU_Free(req->dev.tmpbuf); - if (last != data_sz) { - MPIR_STATUS_SET_COUNT(req->status, last); - if (req->dev.recv_data_sz <= userbuf_sz) { - MPIU_ERR_SETSIMPLE(req->status.MPI_ERROR,MPI_ERR_TYPE,"**dtypemismatch"); - } - } - } - - if (REQ_FIELD(req,iov_to_delete) == 1) - MPIU_Free(REQ_FIELD(req,iov)); - - MPIDI_Comm_get_vc_set_active(req->comm, req->status.MPI_SOURCE, &vc); - MPIDI_CH3U_Handle_recv_req(vc, req, &complete); - MPIU_Assert(complete == TRUE); - - -#ifdef DEBUG - fprintf(stdout,"========> Completing Recv req %p done \n",req); -#endif - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_anysource_posted -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -void MPID_nem_newmad_anysource_posted(MPID_Request *rreq) -{ - /* This function is called whenever an anyource request has been - posted to the posted receive queue. */ - MPIR_Context_id_t context; - Nmad_Nem_tag_t tag; - nm_tag_t match_info = 0; - nm_tag_t match_mask = NEM_NMAD_MATCH_FULL_MASK; - nm_sr_request_t *newmad_req = MPIU_Malloc(sizeof(nm_sr_request_t)); - int num_seg = 1; - int ret; - MPIDI_msg_sz_t data_sz; - int dt_contig; - MPI_Aint dt_true_lb; - MPID_Datatype *dt_ptr; - struct iovec *newmad_iov = (struct iovec *)MPIU_Malloc(NMAD_IOV_MAX_DEPTH*sizeof(struct iovec)); - - tag = rreq->dev.match.parts.tag; - context = rreq->dev.match.parts.context_id; - NEM_NMAD_DIRECT_MATCH(match_info,0,0,context); - if (tag != MPI_ANY_TAG) - { - NEM_NMAD_SET_TAG(match_info,tag); - } - else - { - NEM_NMAD_SET_ANYTAG(match_info); - NEM_NMAD_SET_ANYTAG(match_mask); - } - NEM_NMAD_SET_ANYSRC(match_info); - NEM_NMAD_SET_ANYSRC(match_mask); - -#ifdef DEBUG - fprintf(stdout,"========> Any Source : Posting Recv req %p (nmad req is %p) (match is %lx) (mask is %lx) \n", - rreq,newmad_req,match_info,match_mask); -#endif - - MPIDI_Datatype_get_info(rreq->dev.user_count,rreq->dev.datatype, dt_contig, data_sz, dt_ptr,dt_true_lb); - rreq->dev.OnDataAvail = NULL; - - if (dt_contig) - { - newmad_iov[0].iov_base = (char*)(rreq->dev.user_buf) + dt_true_lb; - newmad_iov[0].iov_len = data_sz; - } - else - { - struct iovec *newmad_iov_ptr = &(newmad_iov[0]); - MPID_nem_newmad_process_rdtype(&rreq,dt_ptr,data_sz,&newmad_iov_ptr,&num_seg); - } - - ret = nm_sr_irecv_iov_with_ref_tagged(mpid_nem_newmad_session,NM_ANY_GATE,match_info,match_mask, - newmad_iov,num_seg,newmad_req,(void*)rreq); - REQ_FIELD(rreq,iov) = newmad_iov; - REQ_FIELD(rreq,iov_to_delete) = 1; - MPID_MEM_NMAD_ADD_REQ_IN_HASH(rreq,newmad_req); - /* - #ifdef DEBUG - fprintf(stdout,"========> Any Source : callback end \n"); - #endif - */ -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_anysource_matched -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_anysource_matched(MPID_Request *rreq) -{ - /* This function is called when an anysource request in the posted - receive queue is matched and dequeued */ - nm_sr_request_t *nmad_request = NULL; - int ret; - int matched = FALSE; - -#ifdef DEBUG - fprintf(stdout,"========> Any Source : MPID_nem_newmad_anysource_matched , req is %p\n",rreq); -#endif - - MPID_NEM_NMAD_GET_REQ_FROM_HASH(rreq,nmad_request); - - if(nmad_request != NULL) - { - -#ifdef DEBUG - fprintf(stdout,"========> Any Source nmad req found :%p \n",nmad_request); -#endif - ret = nm_sr_rcancel(mpid_nem_newmad_session,nmad_request); - if (ret != NM_ESUCCESS) - { - -#ifdef DEBUG - fprintf(stdout,"========> Any Source nmad req (%p) not cancelled \n",nmad_request); -#endif - size_t size; - nm_tag_t match_info; - MPIU_Assert(MPIDI_Request_get_type(rreq) != MPIDI_REQUEST_TYPE_GET_RESP); - ret = nm_sr_rwait(mpid_nem_newmad_session,nmad_request); - MPIU_Assert(ret == NM_ESUCCESS); - nm_sr_request_unset_completion_queue(mpid_nem_newmad_session,nmad_request); - nm_sr_get_rtag(mpid_nem_newmad_session,nmad_request,&match_info); - nm_sr_get_size(mpid_nem_newmad_session,nmad_request,&size); - MPID_nem_newmad_handle_rreq(rreq,match_info, size); - matched = TRUE; - } - else - { - MPID_Segment_free(rreq->dev.segment_ptr); - if (REQ_FIELD(rreq,iov_to_delete) == 1) - MPIU_Free(REQ_FIELD(rreq,iov)); - } - MPIU_Free(nmad_request); - } - return matched; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_process_rdtype -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_process_rdtype(MPID_Request **rreq_p, MPID_Datatype * dt_ptr, MPIDI_msg_sz_t data_sz, struct iovec *newmad_iov[], int *num_iov) -{ - MPID_Request *rreq = *rreq_p; - MPIDI_msg_sz_t last; - MPID_IOV *iov; - int n_iov = 0; - int mpi_errno = MPI_SUCCESS; - int index; - - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_NEWMAD_PROCESS_RDTYPE); - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_NEWMAD_PROCESS_RDTYPE); - - if (rreq->dev.segment_ptr == NULL) - { - rreq->dev.segment_ptr = MPID_Segment_alloc( ); - MPIU_ERR_CHKANDJUMP1((rreq->dev.segment_ptr == NULL), mpi_errno, MPI_ERR_OTHER, "**nomem", "**nomem %s", "MPID_Segment_alloc"); - } - MPID_Segment_init(rreq->dev.user_buf, rreq->dev.user_count, rreq->dev.datatype, rreq->dev.segment_ptr, 0); - rreq->dev.segment_first = 0; - rreq->dev.segment_size = data_sz; - last = rreq->dev.segment_size; - - MPID_Segment_count_contig_blocks(rreq->dev.segment_ptr,rreq->dev.segment_first,&last,&n_iov); - MPIU_Assert(n_iov > 0); - iov = MPIU_Malloc(n_iov*sizeof(MPID_IOV)); - - MPID_Segment_unpack_vector(rreq->dev.segment_ptr, rreq->dev.segment_first, &last,iov, &n_iov); - MPIU_Assert(last == rreq->dev.segment_size); - -#ifdef DEBUG - for(index = 0; index < n_iov ; index++) - { - fprintf(stdout,"======================\n"); - fprintf(stdout,"RECV iov[%i]: [base %p][len %i]\n",index, - iov[index].MPID_IOV_BUF,iov[index].MPID_IOV_LEN); - } -#endif - - if(n_iov <= NMAD_IOV_MAX_DEPTH) - { - for(index=0; index < n_iov ; index++) - { - (*newmad_iov)[index].iov_base = iov[index].MPID_IOV_BUF; - (*newmad_iov)[index].iov_len = iov[index].MPID_IOV_LEN; - } - rreq->dev.tmpbuf = NULL; - *num_iov = n_iov; - } - else - { - MPI_Aint packsize = 0; - MPIR_Pack_size_impl(rreq->dev.user_count, rreq->dev.datatype, &packsize); - rreq->dev.tmpbuf = MPIU_Malloc((size_t) packsize); - MPIU_Assert(rreq->dev.tmpbuf); - rreq->dev.tmpbuf_sz = packsize; - (*newmad_iov)[0].iov_base = (char *) rreq->dev.tmpbuf; - (*newmad_iov)[0].iov_len = (uint32_t) packsize; - *num_iov = 1 ; - } - MPIU_Free(iov); - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_NEWMAD_PROCESS_RDTYPE); - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - - diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_probe.c b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_probe.c deleted file mode 100644 index d275264..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_probe.c +++ /dev/null @@ -1,196 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#include "newmad_impl.h" -#include "my_papi_defs.h" - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_probe -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_probe(MPIDI_VC_t *vc, int source, int tag, MPID_Comm *comm, int context_offset, MPI_Status *status) -{ - nm_tag_t match_info = 0; - nm_tag_t match_mask = NEM_NMAD_MATCH_FULL_MASK; - nm_gate_t out_gate; - nm_gate_t in_gate; - nm_tag_t out_tag; - int size; - int mpi_errno = MPI_SUCCESS; - int ret; - - NEM_NMAD_SET_CTXT(match_info,comm->context_id + context_offset); - if( source == MPI_ANY_SOURCE) - { - NEM_NMAD_SET_ANYSRC(match_info); - NEM_NMAD_SET_ANYSRC(match_mask); - in_gate = NM_ANY_GATE; - } - else - { - NEM_NMAD_SET_SRC(match_info,source); - in_gate = VC_FIELD(vc,p_gate); - } - - if (tag != MPI_ANY_TAG) - { - NEM_NMAD_SET_TAG(match_info,tag); - } - else - { - NEM_NMAD_SET_ANYTAG(match_info); - NEM_NMAD_SET_ANYTAG(match_mask); - } - - do { - ret = nm_sr_probe(mpid_nem_newmad_session,in_gate,&out_gate, - match_info,match_mask,&out_tag,&size); - } - while (ret != NM_ESUCCESS); - - if (source != MPI_ANY_SOURCE) - status->MPI_SOURCE = source; - else - { - MPIDI_VC_t *vc; - int index; - vc = (MPIDI_VC_t *)nm_gate_ref_get(out_gate); - for(index = 0 ; index < comm->local_size ; index ++) - if (vc == comm->vcr[index]) - break; - status->MPI_SOURCE = index; - } - - if (tag != MPI_ANY_TAG) - status->MPI_TAG = tag; - else - NEM_NMAD_MATCH_GET_TAG(out_tag,status->MPI_TAG); - - MPIR_STATUS_SET_COUNT(*status, size); - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_iprobe -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_iprobe(MPIDI_VC_t *vc, int source, int tag, MPID_Comm *comm, int context_offset, int *flag, MPI_Status *status) -{ - nm_tag_t match_info = 0; - nm_tag_t match_mask = NEM_NMAD_MATCH_FULL_MASK; - nm_gate_t out_gate; - nm_gate_t in_gate; - nm_tag_t out_tag; - int size; - int mpi_errno = MPI_SUCCESS; - int ret; - - NEM_NMAD_SET_CTXT(match_info,comm->context_id + context_offset); - if( source == MPI_ANY_SOURCE) - { - NEM_NMAD_SET_ANYSRC(match_info); - NEM_NMAD_SET_ANYSRC(match_mask); - in_gate = NM_ANY_GATE; - } - else - { - NEM_NMAD_SET_SRC(match_info,source); - in_gate = VC_FIELD(vc,p_gate); - } - - if (tag != MPI_ANY_TAG) - { - NEM_NMAD_SET_TAG(match_info,tag); - } - else - { - NEM_NMAD_SET_ANYTAG(match_info); - NEM_NMAD_SET_ANYTAG(match_mask); - } - - ret = nm_sr_probe(mpid_nem_newmad_session,in_gate,&out_gate, - match_info,match_mask,&out_tag,&size); - if (ret == NM_ESUCCESS) - { - if (source != MPI_ANY_SOURCE) - status->MPI_SOURCE = source; - else - { - MPIDI_VC_t *vc; - int index; - vc = (MPIDI_VC_t *)nm_gate_ref_get(out_gate); - for(index = 0 ; index < comm->local_size ; index ++) - if (vc == comm->vcr[index]) - break; - status->MPI_SOURCE = index; - } - - if (tag != MPI_ANY_TAG) - status->MPI_TAG = tag; - else - NEM_NMAD_MATCH_GET_TAG(out_tag,status->MPI_TAG); - - MPIR_STATUS_SET_COUNT(*status, size); - *flag = TRUE; - } - else - *flag = FALSE; - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_improbe -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_improbe(MPIDI_VC_t *vc, int source, int tag, MPID_Comm *comm, int context_offset, int *flag, MPID_Request **message, MPI_Status *status) -{ - int mpi_errno = MPI_SUCCESS; - - /* not currently implemented for newmad */ - MPIU_ERR_SET(mpi_errno, MPI_ERR_INTERN, "**nomprobe"); - - fn_exit: - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_anysource_iprobe -#undef FCNAME -#define FCNAME MPIU_QUOTE(FUNCNAME) -int MPID_nem_newmad_anysource_iprobe(int tag, MPID_Comm *comm, int context_offset, int *flag, MPI_Status *status) -{ - return MPID_nem_newmad_iprobe(NULL, MPI_ANY_SOURCE, tag, comm, context_offset, flag, status); -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_anysource_improbe -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_anysource_improbe(int tag, MPID_Comm *comm, int context_offset, int *flag, MPID_Request **message, MPI_Status *status) -{ - return MPID_nem_newmad_improbe(NULL, MPI_ANY_SOURCE, tag, comm, context_offset,flag, message, status); -} - diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_register.c b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_register.c deleted file mode 100644 index 36695c5..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_register.c +++ /dev/null @@ -1,25 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#include "newmad_impl.h" - -int -MPID_nem_newmad_register_mem (void *p, int len) -{ - return 0; -} - -int -MPID_nem_newmad_deregister_mem (void *p, int len) -{ - return 0; -} - - diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_send.c b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_send.c deleted file mode 100644 index 23ee225..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_send.c +++ /dev/null @@ -1,420 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#include "newmad_impl.h" -#include "newmad_extended_interface.h" -#include "my_papi_defs.h" - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_iSendContig -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_iSendContig(MPIDI_VC_t *vc, MPID_Request *sreq, void *hdr, MPIDI_msg_sz_t hdr_sz, void *data, MPIDI_msg_sz_t data_sz) -{ - int mpi_errno = MPI_SUCCESS; - nm_tag_t match_info = 0; - struct iovec newmad_iov[2]; - int num_iov = 1; - - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_NEWMAD_ISENDCONTIG); - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_NEWMAD_ISENDCONTIG); - - MPIU_Assert(hdr_sz <= sizeof(MPIDI_CH3_Pkt_t)); - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "newmad_iSendContig"); - MPIDI_DBG_Print_packet((MPIDI_CH3_Pkt_t *)hdr); - - NEM_NMAD_ADI_MATCH(match_info); - -#ifdef DEBUG - fprintf(stdout,"iSendContig ========> Sending ADI msg for req %p (match is %lx) \n",sreq,match_info); -#endif - - MPIU_Memcpy(&(sreq->dev.pending_pkt),(char *)hdr,sizeof(MPIDI_CH3_Pkt_t)); - newmad_iov[0].iov_base = (char *)&(sreq->dev.pending_pkt); - newmad_iov[0].iov_len = sizeof(MPIDI_CH3_Pkt_t); - if(data_sz) - { - newmad_iov[1].iov_base = data; - newmad_iov[1].iov_len = data_sz; - num_iov += 1; - } - REQ_FIELD(sreq,iov) = NULL; - REQ_FIELD(sreq,iov_to_delete) = 0; - - nm_sr_isend_iov_with_ref(mpid_nem_newmad_session, VC_FIELD(vc, p_gate), match_info, - newmad_iov, num_iov, &(REQ_FIELD(sreq,newmad_req)),(void *)sreq); - (VC_FIELD(vc,pending_sends)) += 1; - sreq->ch.vc = vc; - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_NEWMAD_ISENDCONTIG); - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_iStartContigMsg -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_iStartContigMsg(MPIDI_VC_t *vc, void *hdr, MPIDI_msg_sz_t hdr_sz, void *data, MPIDI_msg_sz_t data_sz, MPID_Request **sreq_ptr) -{ - MPID_Request *sreq = NULL; - nm_tag_t match_info = 0; - struct iovec newmad_iov[2]; - int num_iov = 1; - int mpi_errno = MPI_SUCCESS; - - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_NEWMAD_ISTARTCONTIGMSG); - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_NEWMAD_ISTARTCONTIGMSG); - MPIU_Assert(hdr_sz <= sizeof(MPIDI_CH3_Pkt_t)); - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "newmad_iSendContig"); - MPIDI_DBG_Print_packet((MPIDI_CH3_Pkt_t *)hdr); - - /* create a request */ - sreq = MPID_Request_create(); - MPIU_Assert (sreq != NULL); - MPIU_Object_set_ref (sreq, 2); - sreq->kind = MPID_REQUEST_SEND; - sreq->dev.OnDataAvail = 0; - - NEM_NMAD_ADI_MATCH(match_info); -#ifdef DEBUG - fprintf(stdout,"StartcontigMsg ========> Sending ADI msg for req %p (match is %lx) \n",sreq,match_info); -#endif - - MPIU_Memcpy(&(sreq->dev.pending_pkt),(char *)hdr,sizeof(MPIDI_CH3_Pkt_t)); - newmad_iov[0].iov_base = (char *)&(sreq->dev.pending_pkt); - newmad_iov[0].iov_len = sizeof(MPIDI_CH3_Pkt_t); - if (data_sz) - { - newmad_iov[1].iov_base = (char *)data; - newmad_iov[1].iov_len = data_sz; - num_iov += 1; - } - REQ_FIELD(sreq,iov) = NULL; - REQ_FIELD(sreq,iov_to_delete) = 0; - nm_sr_isend_iov_with_ref(mpid_nem_newmad_session, VC_FIELD(vc, p_gate), match_info, - newmad_iov, num_iov, &(REQ_FIELD(sreq,newmad_req)),(void *)sreq); - (VC_FIELD(vc,pending_sends)) += 1; - sreq->ch.vc = vc; - - fn_exit: - *sreq_ptr = sreq; - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_NEWMAD_ISTARTCONTIGMSG); - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_SendNoncontig -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_SendNoncontig(MPIDI_VC_t *vc, MPID_Request *sreq, void *header, MPIDI_msg_sz_t hdr_sz) -{ - int mpi_errno = MPI_SUCCESS; - nm_tag_t match_info = 0; - struct iovec newmad_iov[2]; - int num_iov = 1; - MPIDI_msg_sz_t last; - MPIDI_msg_sz_t data_sz; - - /* - struct iovec *newmad_iov = (struct iovec *)MPIU_Malloc(NMAD_IOV_MAX_DEPTH*sizeof(struct iovec)); - MPIDI_msg_sz_t data_sz; - int dt_contig; - MPI_Aint dt_true_lb; - MPID_Datatype *dt_ptr; - */ - - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_NEWMAD_SENDNONCONTIGMSG); - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_NEWMAD_SENDNONCONTIGMSG); - MPIU_Assert(hdr_sz <= sizeof(MPIDI_CH3_Pkt_t)); - MPIU_DBG_MSG(CH3_CHANNEL, VERBOSE, "MPID_nem_newmad_iSendNoncontig"); - - NEM_NMAD_ADI_MATCH(match_info); - - MPIU_Memcpy(&(sreq->dev.pending_pkt),(char *)header,sizeof(MPIDI_CH3_Pkt_t)); - newmad_iov[0].iov_base = (char *)&(sreq->dev.pending_pkt); - newmad_iov[0].iov_len = sizeof(MPIDI_CH3_Pkt_t); - - data_sz = sreq->dev.segment_size - sreq->dev.segment_first; - last = sreq->dev.segment_size; - if (data_sz > 0) - { - sreq->dev.tmpbuf = MPIU_Malloc((size_t) data_sz); - REQ_FIELD(sreq,deltmpbuf) = TMP_DEL_VALUE; - MPID_Segment_pack(sreq->dev.segment_ptr,sreq->dev.segment_first, &last,(char *)(sreq->dev.tmpbuf)); - MPIU_Assert(last == sreq->dev.segment_size); - newmad_iov[1].iov_base = (char *)(sreq->dev.tmpbuf); - newmad_iov[1].iov_len = (uint32_t)last; - num_iov++; - } - - REQ_FIELD(sreq,iov) = NULL; - REQ_FIELD(sreq,iov_to_delete) = 0; - - /* - MPIDI_Datatype_get_info(sreq->dev.user_count,sreq->dev.datatype, dt_contig, data_sz, dt_ptr,dt_true_lb); - if(data_sz) - { - struct iovec *mad_iov_ptr = &(newmad_iov[0]); - MPID_nem_newmad_process_sdtype(&sreq,sreq->dev.datatype,dt_ptr,sreq->dev.user_buf, - sreq->dev.user_count,data_sz, &mad_iov_ptr,&num_iov,1); - } -#ifdef DEBUG - fprintf(stdout,"SendNonContig ========> Sending ADI msg for req %p (match is %lx) \n",sreq,match_info); -#endif - */ - - nm_sr_isend_iov_with_ref(mpid_nem_newmad_session, VC_FIELD(vc, p_gate), match_info, - newmad_iov, num_iov, &(REQ_FIELD(sreq,newmad_req)),(void*)sreq); - (VC_FIELD(vc,pending_sends)) += 1; - - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_NEWMAD_SENDNONCONTIGMSG); - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_directSend -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_directSend(MPIDI_VC_t *vc, const void * buf, int count, MPI_Datatype datatype, int rank, int tag, - MPID_Comm * comm, int context_offset, MPID_Request **sreq_p) -{ - MPID_Request *sreq = NULL; - nm_tag_t match_info = 0; - int num_iov = 0; - int mpi_errno = MPI_SUCCESS; - MPID_Datatype *dt_ptr; - int dt_contig; - MPIDI_msg_sz_t data_sz; - MPI_Aint dt_true_lb; - - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_NEWMAD_DIRECTSEND); - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_NEWMAD_DIRECTSEND); - - MPIDI_Datatype_get_info(count, datatype, dt_contig, data_sz, dt_ptr, dt_true_lb); - - MPIDI_Request_create_sreq(sreq, mpi_errno, goto fn_exit); - MPIDI_Request_set_type(sreq, MPIDI_REQUEST_TYPE_SEND); - sreq->partner_request = NULL; - MPIDI_VC_FAI_send_seqnum(vc, seqnum); - MPIDI_Request_set_seqnum(sreq, seqnum); - sreq->ch.vc = vc; - sreq->dev.OnDataAvail = NULL; - NEM_NMAD_DIRECT_MATCH(match_info,tag,comm->rank,comm->context_id + context_offset); -#ifdef DEBUG - fprintf(stdout,"Direct Send ========> Sending NON ADI msg for req %p (match is %lx) \n",sreq,match_info); -#endif - - if(data_sz) - { - struct iovec newmad_iov[NMAD_IOV_MAX_DEPTH]; - if (dt_contig) - { - newmad_iov[0].iov_base = (char*)(buf + dt_true_lb); - newmad_iov[0].iov_len = data_sz; - num_iov = 1; - } - else - { - struct iovec *mad_iov_ptr = &(newmad_iov[0]); - MPID_nem_newmad_process_sdtype(&sreq,datatype,dt_ptr,buf,count,data_sz,&mad_iov_ptr,&num_iov,0); - } - MPIU_Assert(num_iov <= NMAD_IOV_MAX_DEPTH); -#ifdef DEBUG - for(index = 0; index < num_iov ; index++) - { - fprintf(stdout,"======================\n"); - fprintf(stdout,"SEND nmad_iov[%i]: [base %p][len %i]\n",index, - newmad_iov[index].iov_base,newmad_iov[index].iov_len); - } -#endif - nm_sr_isend_iov_with_ref(mpid_nem_newmad_session, VC_FIELD(vc, p_gate), match_info, - newmad_iov, num_iov, &(REQ_FIELD(sreq,newmad_req)),(void*)sreq); - } - else - { - nm_sr_isend_with_ref(mpid_nem_newmad_session, VC_FIELD(vc, p_gate), match_info, - NULL, 0, &(REQ_FIELD(sreq,newmad_req)),(void*)sreq); - } - REQ_FIELD(sreq,iov) = NULL; - REQ_FIELD(sreq,iov_to_delete) = 0; - (VC_FIELD(vc,pending_sends)) += 1; - - fn_exit: - *sreq_p = sreq; - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_NEWMAD_DIRECTSEND); - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_directSsend -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_directSsend(MPIDI_VC_t *vc, const void * buf, int count, MPI_Datatype datatype, int rank, int tag, - MPID_Comm * comm, int context_offset, MPID_Request **sreq_p) -{ - MPID_Request *sreq = NULL; - nm_tag_t match_info = 0; - int num_iov = 0; - int mpi_errno = MPI_SUCCESS; - MPID_Datatype *dt_ptr; - int dt_contig; - MPIDI_msg_sz_t data_sz; - MPI_Aint dt_true_lb; - - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_NEWMAD_DIRECTSSEND); - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_NEWMAD_DIRECTSSEND); - - MPIDI_Datatype_get_info(count, datatype, dt_contig, data_sz, dt_ptr, dt_true_lb); - - MPIDI_Request_create_sreq(sreq, mpi_errno, goto fn_exit); - MPIDI_Request_set_type(sreq, MPIDI_REQUEST_TYPE_SEND); - sreq->partner_request = NULL; - MPIDI_VC_FAI_send_seqnum(vc, seqnum); - MPIDI_Request_set_seqnum(sreq, seqnum); - sreq->ch.vc = vc; - sreq->dev.OnDataAvail = NULL; - - NEM_NMAD_DIRECT_MATCH(match_info,tag,comm->rank,comm->context_id + context_offset); - if(data_sz) - { - struct iovec newmad_iov[NMAD_IOV_MAX_DEPTH]; - if (dt_contig) - { - newmad_iov[0].iov_base = (char*)(buf + dt_true_lb); - newmad_iov[0].iov_len = data_sz; - num_iov = 1; - } - else - { - struct iovec *mad_iov_ptr = &(newmad_iov[0]); - MPID_nem_newmad_process_sdtype(&sreq,datatype,dt_ptr,buf,count,data_sz,&mad_iov_ptr,&num_iov,0); - } - nm_sr_issend_iov(mpid_nem_newmad_session, VC_FIELD(vc, p_gate), match_info, - newmad_iov, num_iov, &(REQ_FIELD(sreq,newmad_req))); - } - else - { - nm_sr_issend(mpid_nem_newmad_session, VC_FIELD(vc, p_gate), match_info, - NULL, 0, &(REQ_FIELD(sreq,newmad_req))); - } - REQ_FIELD(sreq,iov) = NULL; - REQ_FIELD(sreq,iov_to_delete) = 0; - (VC_FIELD(vc,pending_sends)) += 1; - - fn_exit: - *sreq_p = sreq; - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_NEWMAD_DIRECTSSEND); - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - -#undef FUNCNAME -#define FUNCNAME MPID_nem_newmad_process_sdtype -#undef FCNAME -#define FCNAME MPIDI_QUOTE(FUNCNAME) -int MPID_nem_newmad_process_sdtype(MPID_Request **sreq_p, MPI_Datatype datatype, MPID_Datatype *dt_ptr, const void *buf, - int count, MPIDI_msg_sz_t data_sz, struct iovec *mad_iov[], int *num_iov, int first_taken) -{ - MPID_Request *sreq =*sreq_p; - MPID_IOV *iov; - MPIDI_msg_sz_t last; - int n_iov = 0; - int num_entries = NMAD_IOV_MAX_DEPTH - first_taken; - int mpi_errno = MPI_SUCCESS; - int index; - - MPIDI_STATE_DECL(MPID_STATE_MPID_NEM_NEWMAD_PROCESS_SDTYPE); - MPIDI_FUNC_ENTER(MPID_STATE_MPID_NEM_NEWMAD_PROCESS_SDTYPE); - - sreq->dev.segment_ptr = MPID_Segment_alloc( ); - MPIU_ERR_CHKANDJUMP1((sreq->dev.segment_ptr == NULL), mpi_errno, MPI_ERR_OTHER, "**nomem", "**nomem %s", "MPID_Segment_alloc"); - MPID_Segment_init(buf, count, datatype, sreq->dev.segment_ptr, 0); - sreq->dev.segment_first = 0; - sreq->dev.segment_size = data_sz; - last = sreq->dev.segment_size; - - MPID_Segment_count_contig_blocks(sreq->dev.segment_ptr,sreq->dev.segment_first,&last,&n_iov); - MPIU_Assert(n_iov > 0); - iov = MPIU_Malloc(n_iov*sizeof(MPID_IOV)); - - MPID_Segment_pack_vector(sreq->dev.segment_ptr, sreq->dev.segment_first, &last,iov, &n_iov); - MPIU_Assert(last == sreq->dev.segment_size); - -#ifdef DEBUG - for(index = 0; index < n_iov ; index++) - { - fprintf(stdout,"======================\n"); - fprintf(stdout,"SEND: iov[%i]: [base %p][len %i]\n",index, - iov[index].MPID_IOV_BUF,iov[index].MPID_IOV_LEN); - } -#endif - - if(n_iov <= num_entries) - { - for(index = 0; index < n_iov ; index++) - { - (*mad_iov)[first_taken+index].iov_base = iov[index].MPID_IOV_BUF; - (*mad_iov)[first_taken+index].iov_len = iov[index].MPID_IOV_LEN; - } - *num_iov = n_iov + first_taken; - } - else - { - int size_to_copy = 0; - int offset = 0; - int last_entry = num_entries - 1; - for(index = 0; index < n_iov ; index++) - { - if (index <= (last_entry-1)) - { - (*mad_iov)[first_taken+index].iov_base = iov[index].MPID_IOV_BUF; - (*mad_iov)[first_taken+index].iov_len = iov[index].MPID_IOV_LEN; - } - else - { - size_to_copy += iov[index].MPID_IOV_LEN; - } - } - sreq->dev.tmpbuf = MPIU_Malloc(size_to_copy); - MPIU_Assert(sreq->dev.tmpbuf); - REQ_FIELD(sreq,deltmpbuf) = TMP_DEL_VALUE; - for(index = last_entry; index < n_iov; index++) - { - MPIU_Memcpy((char *)(sreq->dev.tmpbuf) + offset, iov[index].MPID_IOV_BUF, iov[index].MPID_IOV_LEN); - offset += iov[index].MPID_IOV_LEN; - } - (*mad_iov)[NMAD_IOV_MAX_DEPTH-1].iov_base = sreq->dev.tmpbuf; - (*mad_iov)[NMAD_IOV_MAX_DEPTH-1].iov_len = size_to_copy; - *num_iov = NMAD_IOV_MAX_DEPTH; - } - MPIU_Free(iov); - fn_exit: - MPIDI_FUNC_EXIT(MPID_STATE_MPID_NEM_NEWMAD_PROCESS_SDTYPE); - return mpi_errno; - fn_fail: ATTRIBUTE((unused)) - goto fn_exit; -} - - diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_test.c b/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_test.c deleted file mode 100644 index 38fbb60..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_test.c +++ /dev/null @@ -1,17 +0,0 @@ -/* -*- Mode: C; c-basic-offset:4 ; indent-tabs-mode:nil ; -*- */ -/* - * (C) 2006 by Argonne National Laboratory. - * See COPYRIGHT in top-level directory. - * - * Copyright � 2006-2011 Guillaume Mercier, Institut Polytechnique de - * Bordeaux. All rights reserved. Permission is hereby granted to use, - * reproduce, prepare derivative works, and to redistribute to others. - */ - -#include "newmad_impl.h" - -int -MPID_nem_newmad_test() -{ - return 0; -} diff --git a/src/mpid/ch3/channels/nemesis/netmod/newmad/subconfigure.m4 b/src/mpid/ch3/channels/nemesis/netmod/newmad/subconfigure.m4 deleted file mode 100644 index ec603df..0000000 --- a/src/mpid/ch3/channels/nemesis/netmod/newmad/subconfigure.m4 +++ /dev/null @@ -1,44 +0,0 @@ -[#] start of __file__ -dnl MPICH_SUBCFG_AFTER=src/mpid/ch3/channels/nemesis - -AC_DEFUN([PAC_SUBCFG_PREREQ_]PAC_SUBCFG_AUTO_SUFFIX,[ - #check for NewMadeleine options - #AC_ARG_WITH(newmad, [--with-newmad=path - specify path where pm2 software can be found], - #if test "${with_newmad}" != "yes" -a "${with_newmad}" != "no" ; then - # LDFLAGS="$LDFLAGS `${with_newmad}/bin/pm2-config --flavor=$PM2_FLAVOR --libs`" - # CPPFLAGS="$CPPFLAGS `${with_newmad}/bin/pm2-config --flavor=$PM2_FLAVOR --cflags`" - #fi,) - - AM_COND_IF([BUILD_CH3_NEMESIS],[ - for net in $nemesis_networks ; do - AS_CASE([$net],[newmad],[build_nemesis_netmod_newmad=yes]) - done - ]) - AM_CONDITIONAL([BUILD_NEMESIS_NETMOD_NEWMAD],[test "X$build_nemesis_netmod_newmad" = "Xyes"]) - -])dnl - -AC_DEFUN([PAC_SUBCFG_BODY_]PAC_SUBCFG_AUTO_SUFFIX,[ -AM_COND_IF([BUILD_NEMESIS_NETMOD_NEWMAD],[ - AC_MSG_NOTICE([RUNNING CONFIGURE FOR ch3:nemesis:newmad]) - LDFLAGS="$LDFLAGS `pkg-config nmad --libs`" - CPPFLAGS="$CPPFLAGS `pkg-config nmad --cflags`" - WRAPPER_CFLAGS="$WRAPPER_CFLAGS `pkg-config nmad --cflags`" - AC_CHECK_HEADER([nm_public.h], , [ - AC_MSG_ERROR(['nm_public.h not found. Did you specify --with-newmad= ?']) - ]) - AC_CHECK_HEADER([nm_sendrecv_interface.h], , [ - AC_MSG_ERROR(['nm_sendrecv_interface.h not found. Did you specify --with-newmad= ?']) - ]) - AC_CHECK_LIB(nmad,nm_core_init, , [ - AC_MSG_ERROR(['nmad library not found. Did you specify --with-newmad= ?']) - ]) - AC_ARG_ENABLE(newmad-multirail, - [--enable-newmad-multirail - enables multirail support in newmad module],,enable_multi=no) - if test "$enable_multi" = "yes" ; then - AC_DEFINE(MPID_MAD_MODULE_MULTIRAIL, 1, [Define to enable multirail support in newmad module]) - fi - AC_DEFINE([ENABLE_COMM_OVERRIDES], 1, [define to add per-vc function pointers to override send and recv functions]) -])dnl end AM_COND_IF(BUILD_NEMESIS_NETMOD_NEWMAD,...) -])dnl end _BODY -[#] end of __file__ ----------------------------------------------------------------------- Summary of changes: src/mpi/errhan/errnames.txt | 36 -- src/mpid/ch3/channels/nemesis/netmod/Makefile.mk | 2 - .../ch3/channels/nemesis/netmod/newmad/Makefile.mk | 26 - .../channels/nemesis/netmod/newmad/newmad_alloc.c | 133 ----- .../channels/nemesis/netmod/newmad/newmad_cancel.c | 70 --- .../netmod/newmad/newmad_extended_interface.h | 75 --- .../nemesis/netmod/newmad/newmad_finalize.c | 46 -- .../channels/nemesis/netmod/newmad/newmad_impl.h | 214 ------- .../channels/nemesis/netmod/newmad/newmad_init.c | 314 ---------- .../channels/nemesis/netmod/newmad/newmad_poll.c | 596 -------------------- .../channels/nemesis/netmod/newmad/newmad_probe.c | 196 ------- .../nemesis/netmod/newmad/newmad_register.c | 25 - .../channels/nemesis/netmod/newmad/newmad_send.c | 420 -------------- .../channels/nemesis/netmod/newmad/newmad_test.c | 17 - .../channels/nemesis/netmod/newmad/subconfigure.m4 | 44 -- .../ch3/channels/nemesis/netmod/scif/Makefile.mk | 21 - .../channels/nemesis/netmod/scif/cross_values.txt | 16 - .../channels/nemesis/netmod/scif/scif_finalize.c | 37 -- .../ch3/channels/nemesis/netmod/scif/scif_impl.h | 89 --- .../ch3/channels/nemesis/netmod/scif/scif_init.c | 469 --------------- .../ch3/channels/nemesis/netmod/scif/scif_send.c | 490 ---------------- src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.c | 472 ---------------- src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.h | 88 --- src/mpid/ch3/channels/nemesis/netmod/scif/scifsm.c | 201 ------- .../channels/nemesis/netmod/scif/subconfigure.m4 | 23 - 25 files changed, 0 insertions(+), 4120 deletions(-) delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/Makefile.mk delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_alloc.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_cancel.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_extended_interface.h delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_finalize.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_impl.h delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_init.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_poll.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_probe.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_register.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_send.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/newmad_test.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/newmad/subconfigure.m4 delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/Makefile.mk delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/cross_values.txt delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/scif_finalize.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/scif_impl.h delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/scif_init.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/scif_send.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/scifrw.h delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/scifsm.c delete mode 100644 src/mpid/ch3/channels/nemesis/netmod/scif/subconfigure.m4 hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b2-32-g9ff9a4e
by noreply＠mpich.org 11 May '15

11 May '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 9ff9a4e6f2cca18eba33bb1e384d5c71d8b66303 (commit) via b3edcb539cee5705b14e59b48bd6e74623fdbd1f (commit) from 64b775637d138f297c87306901c0978b5e671b37 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/9ff9a4e6f2cca18eba33bb1e384d5c71d… commit 9ff9a4e6f2cca18eba33bb1e384d5c71d8b66303 Author: Rob Latham <robl(a)mcs.anl.gov> Date: Thu May 7 13:53:39 2015 -0500 avoid segfault when calling built-in error handler we were not correctly handling the case where the error handler MPI_ERRORS_ARE_FATAL was called. credit Lisandro Dalcin with finidng the issue. Signed-off-by: Junchao Zhang <jczhang(a)mcs.anl.gov> diff --git a/src/include/mpierror.h b/src/include/mpierror.h index 78c8612..5a68de7 100644 --- a/src/include/mpierror.h +++ b/src/include/mpierror.h @@ -178,6 +178,12 @@ void MPIR_Err_get_string( int, char *, int, MPIR_Err_get_class_string_func_t ); int MPIR_Err_set_msg( int code, const char *msg_string ); +/* This routine is called when there is a fatal error. Now public because file + * error handling is defined in a separate file from comm and win, but all + * three need to call it */ +void MPIR_Handle_fatal_error(struct MPID_Comm *comm_ptr, + const char fcname[], int errcode); + #define MPIR_ERR_CLASS_MASK 0x0000007f #define MPIR_ERR_CLASS_SIZE 128 #define MPIR_ERR_GET_CLASS(mpi_errno_) (mpi_errno_ & MPIR_ERR_CLASS_MASK) diff --git a/src/mpi/errhan/errutil.c b/src/mpi/errhan/errutil.c index fbe520f..ca3ff40 100644 --- a/src/mpi/errhan/errutil.c +++ b/src/mpi/errhan/errutil.c @@ -117,7 +117,6 @@ static int did_err_init = FALSE; /* helps us solve a bootstrapping problem */ with a valid value. */ static int checkValidErrcode( int, const char [], int * ); -static void handleFatalError( MPID_Comm *, const char [], int ); #if MPICH_ERROR_MSG_LEVEL >= MPICH_ERROR_MSG_ALL static int ErrGetInstanceString( int, char [], int ); @@ -249,7 +248,7 @@ int MPIR_Err_return_comm( MPID_Comm *comm_ptr, const char fcname[], { /* for whatever reason, we aren't initialized (perhaps error during MPI_Init) */ - handleFatalError(MPIR_Process.comm_world, fcname, errcode); + MPIR_Handle_fatal_error(MPIR_Process.comm_world, fcname, errcode); return MPI_ERR_INTERN; } /* --END ERROR HANDLING-- */ @@ -275,7 +274,7 @@ int MPIR_Err_return_comm( MPID_Comm *comm_ptr, const char fcname[], /* --BEGIN ERROR HANDLING-- */ if (MPIR_Err_is_fatal(errcode) || comm_ptr == NULL) { /* Calls MPID_Abort */ - handleFatalError( comm_ptr, fcname, errcode ); + MPIR_Handle_fatal_error( comm_ptr, fcname, errcode ); /* never get here */ } /* --END ERROR HANDLING-- */ @@ -292,7 +291,7 @@ int MPIR_Err_return_comm( MPID_Comm *comm_ptr, const char fcname[], if (errhandler == NULL || errhandler->handle == MPI_ERRORS_ARE_FATAL) { MPIU_THREAD_CS_EXIT(MPI_OBJ, comm_ptr); /* Calls MPID_Abort */ - handleFatalError( comm_ptr, fcname, errcode ); + MPIR_Handle_fatal_error( comm_ptr, fcname, errcode ); /* never get here */ } /* --END ERROR HANDLING-- */ @@ -365,7 +364,7 @@ int MPIR_Err_return_win( MPID_Win *win_ptr, const char fcname[], int errcode ) win_ptr == NULL || win_ptr->errhandler == NULL || win_ptr->errhandler->handle == MPI_ERRORS_ARE_FATAL) { /* Calls MPID_Abort */ - handleFatalError( NULL, fcname, errcode ); + MPIR_Handle_fatal_error( NULL, fcname, errcode ); } /* --END ERROR HANDLING-- */ @@ -432,8 +431,7 @@ static void CombineSpecificCodes( int, int, int ); static const char *get_class_msg( int ); /* --BEGIN ERROR HANDLING-- */ -/* This routine is called when there is a fatal error */ -static void handleFatalError( MPID_Comm *comm_ptr, +void MPIR_Handle_fatal_error( MPID_Comm *comm_ptr, const char fcname[], int errcode ) { /* Define length of the the maximum error message line (or string with diff --git a/src/mpi/errhan/file_call_errhandler.c b/src/mpi/errhan/file_call_errhandler.c index 7856e0c..4ef7671 100644 --- a/src/mpi/errhan/file_call_errhandler.c +++ b/src/mpi/errhan/file_call_errhandler.c @@ -98,6 +98,9 @@ int MPI_File_call_errhandler(MPI_File fh, int errorcode) goto fn_exit; } + if (e->handle == MPI_ERRORS_ARE_FATAL) { + MPIR_Handle_fatal_error(NULL, "MPI_File_call_errhandler", errorcode); + } switch (e->language) { case MPID_LANG_C: http://git.mpich.org/mpich.git/commitdiff/b3edcb539cee5705b14e59b48bd6e7462… commit b3edcb539cee5705b14e59b48bd6e74623fdbd1f Author: Rob Latham <robl(a)mcs.anl.gov> Date: Thu May 7 13:28:57 2015 -0500 proper return value for MPI_File_call_errhandler Lisandro Dalcin found we returned the wrong value from MPI_File_call_errhandler. it should return MPI_SUCCESS unless something goes wrong with MPI_File_call_errhandler itself. Closes #2259 Signed-off-by: Junchao Zhang <jczhang(a)mcs.anl.gov> diff --git a/src/mpi/errhan/file_call_errhandler.c b/src/mpi/errhan/file_call_errhandler.c index e132b24..7856e0c 100644 --- a/src/mpi/errhan/file_call_errhandler.c +++ b/src/mpi/errhan/file_call_errhandler.c @@ -95,7 +95,6 @@ int MPI_File_call_errhandler(MPI_File fh, int errorcode) not integers. */ if (e->handle == MPI_ERRORS_RETURN) { - mpi_errno = errorcode; goto fn_exit; } diff --git a/test/mpi/errhan/Makefile.am b/test/mpi/errhan/Makefile.am index 2098592..866112f 100644 --- a/test/mpi/errhan/Makefile.am +++ b/test/mpi/errhan/Makefile.am @@ -20,6 +20,7 @@ noinst_PROGRAMS = \ errfatal \ predef_eh \ errstring2 \ + file_errhdl \ dynamic_errcode_predefined_errclass EXTRA_PROGRAMS = errcode errring errstring diff --git a/test/mpi/errhan/file_errhdl.c b/test/mpi/errhan/file_errhdl.c new file mode 100644 index 0000000..7b58c2d --- /dev/null +++ b/test/mpi/errhan/file_errhdl.c @@ -0,0 +1,26 @@ +/* test case contributed by Lisandro Dalcin <dalcinl(a)gmail.com> */ +#include <mpi.h> +#include <stdio.h> + +int main(int argc, char *argv[]) +{ + int ierr; + MPI_File fh; + MPI_Init(&argc, &argv); + + MPI_File_open(MPI_COMM_WORLD,"/tmp/datafile", + MPI_MODE_CREATE|MPI_MODE_RDWR|MPI_MODE_DELETE_ON_CLOSE, + MPI_INFO_NULL,&fh); + + MPI_File_set_errhandler(fh,MPI_ERRORS_RETURN); + ierr = MPI_File_call_errhandler(fh,MPI_ERR_FILE); + if (ierr != MPI_SUCCESS) { + fprintf(stderr, "ierr: %d, expected: %d\n", ierr, (int)MPI_SUCCESS); + } else { + printf(" No Errors\n"); + } + MPI_File_close(&fh); + + MPI_Finalize(); + return 0; +} diff --git a/test/mpi/errhan/testlist b/test/mpi/errhan/testlist index ebd60ff..47e3381 100644 --- a/test/mpi/errhan/testlist +++ b/test/mpi/errhan/testlist @@ -5,3 +5,4 @@ predef_eh 1 predef_eh 2 errstring2 1 dynamic_errcode_predefined_errclass 1 +file_errhdl 1 ----------------------------------------------------------------------- Summary of changes: src/include/mpierror.h | 6 ++++++ src/mpi/errhan/errutil.c | 12 +++++------- src/mpi/errhan/file_call_errhandler.c | 4 +++- test/mpi/errhan/Makefile.am | 1 + test/mpi/errhan/file_errhdl.c | 26 ++++++++++++++++++++++++++ test/mpi/errhan/testlist | 1 + 6 files changed, 42 insertions(+), 8 deletions(-) create mode 100644 test/mpi/errhan/file_errhdl.c hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b2-30-g64b7756
by noreply＠mpich.org 10 May '15

10 May '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 64b775637d138f297c87306901c0978b5e671b37 (commit) via 43da8bb0113125a4a4415ebb60cc33d38c8c0f62 (commit) from 60482cc2ca9a740d049444a9261a04a3007bc306 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/64b775637d138f297c87306901c0978b5… commit 64b775637d138f297c87306901c0978b5e671b37 Author: Jithin Jose <jithin.jose(a)intel.com> Date: Wed May 6 15:53:24 2015 -0700 OFI: Set address length correctly while adding to business card Signed-off-by: Jithin Jose <jithin.jose(a)intel.com> Signed-off-by: Charles J Archer <charles.j.archer(a)intel.com> diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_cm.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_cm.c index a1e2538..dce84d0 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_cm.c +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_cm.c @@ -595,7 +595,7 @@ int MPID_nem_ofi_get_business_card(int my_rank ATTRIBUTE((unused)), str_errno = MPIU_Str_add_binary_arg(bc_val_p, val_max_sz_p, "OFI", - (char *) &gl_data.bound_addr, sizeof(gl_data.bound_addr)); + (char *) &gl_data.bound_addr, gl_data.bound_addrlen); if (str_errno) { MPIU_ERR_CHKANDJUMP(str_errno == MPIU_STR_NOMEM, mpi_errno, MPI_ERR_OTHER, "**buscard_len"); MPIU_ERR_SETANDJUMP(mpi_errno, MPI_ERR_OTHER, "**buscard"); http://git.mpich.org/mpich.git/commitdiff/43da8bb0113125a4a4415ebb60cc33d38… commit 43da8bb0113125a4a4415ebb60cc33d38c8c0f62 Author: Jithin Jose <jithin.jose(a)intel.com> Date: Wed May 6 15:58:22 2015 -0700 OFI: Added FI_RC_RETRY and use it for all TX/RX operations. Signed-off-by: Jithin Jose <jithin.jose(a)intel.com> Signed-off-by: Charles J Archer <charles.j.archer(a)intel.com> diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_cm.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_cm.c index e3302ae..a1e2538 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_cm.c +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_cm.c @@ -108,7 +108,7 @@ static inline int MPID_nem_ofi_conn_req_callback(cq_tagged_entry_t * wc, MPID_Re MPIU_Memcpy(bc, rreq->dev.user_buf, wc->len); bc[wc->len] = '\0'; MPIU_Assert(gl_data.conn_req == rreq); - FI_RC(fi_trecv(gl_data.endpoint, + FI_RC_RETRY(fi_trecv(gl_data.endpoint, gl_data.conn_req->dev.user_buf, OFI_KVSAPPSTRLEN, gl_data.mr, @@ -228,7 +228,7 @@ static inline int MPID_nem_ofi_preposted_callback(cq_tagged_entry_t * wc, MPID_R REQ_OFI(new_rreq)->vc = vc; REQ_OFI(new_rreq)->pack_buffer = pack_buffer; REQ_OFI(new_rreq)->pack_buffer_size = pkt_len; - FI_RC(fi_trecv(gl_data.endpoint, + FI_RC_RETRY(fi_trecv(gl_data.endpoint, REQ_OFI(new_rreq)->pack_buffer, REQ_OFI(new_rreq)->pack_buffer_size, gl_data.mr, @@ -240,7 +240,7 @@ static inline int MPID_nem_ofi_preposted_callback(cq_tagged_entry_t * wc, MPID_R sreq->dev.next = NULL; REQ_OFI(sreq)->event_callback = MPID_nem_ofi_cts_send_callback; REQ_OFI(sreq)->parent = new_rreq; - FI_RC(fi_tsend(gl_data.endpoint, + FI_RC_RETRY(fi_tsend(gl_data.endpoint, NULL, 0, gl_data.mr, @@ -249,7 +249,7 @@ static inline int MPID_nem_ofi_preposted_callback(cq_tagged_entry_t * wc, MPID_R MPIU_Assert(gl_data.persistent_req == rreq); rreq->dev.user_count = 0; - FI_RC(fi_trecv(gl_data.endpoint, + FI_RC_RETRY(fi_trecv(gl_data.endpoint, &rreq->dev.user_count, sizeof rreq->dev.user_count, gl_data.mr, @@ -318,7 +318,7 @@ int MPID_nem_ofi_cm_init(MPIDI_PG_t * pg_p, int pg_rank ATTRIBUTE((unused))) persistent_req->dev.next = NULL; REQ_OFI(persistent_req)->vc = NULL; REQ_OFI(persistent_req)->event_callback = MPID_nem_ofi_preposted_callback; - FI_RC(fi_trecv(gl_data.endpoint, + FI_RC_RETRY(fi_trecv(gl_data.endpoint, &persistent_req->dev.user_count, sizeof persistent_req->dev.user_count, gl_data.mr, @@ -337,7 +337,7 @@ int MPID_nem_ofi_cm_init(MPIDI_PG_t * pg_p, int pg_rank ATTRIBUTE((unused))) conn_req->dev.next = NULL; REQ_OFI(conn_req)->vc = NULL; /* We don't know the source yet */ REQ_OFI(conn_req)->event_callback = MPID_nem_ofi_conn_req_callback; - FI_RC(fi_trecv(gl_data.endpoint, + FI_RC_RETRY(fi_trecv(gl_data.endpoint, conn_req->dev.user_buf, OFI_KVSAPPSTRLEN, gl_data.mr, @@ -554,7 +554,7 @@ int MPID_nem_ofi_connect_to_root(const char *business_card, MPIDI_VC_t * new_vc) REQ_OFI(sreq)->pack_buffer = my_bc; if (gl_data.api_set == API_SET_1) { conn_req_send_bits = init_sendtag(0, MPIR_Process.comm_world->rank, 0, MPID_CONN_REQ); - FI_RC(fi_tsend(gl_data.endpoint, + FI_RC_RETRY(fi_tsend(gl_data.endpoint, REQ_OFI(sreq)->pack_buffer, my_bc_len, gl_data.mr, @@ -562,7 +562,7 @@ int MPID_nem_ofi_connect_to_root(const char *business_card, MPIDI_VC_t * new_vc) conn_req_send_bits, &(REQ_OFI(sreq)->ofi_context)), tsend); } else { conn_req_send_bits = init_sendtag_2(0, 0, MPID_CONN_REQ); - FI_RC(fi_tsenddata(gl_data.endpoint, + FI_RC_RETRY(fi_tsenddata(gl_data.endpoint, REQ_OFI(sreq)->pack_buffer, my_bc_len, gl_data.mr, diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_impl.h b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_impl.h index 5cab470..0aa9c4e 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_impl.h +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_impl.h @@ -132,6 +132,28 @@ fn_fail: \ fi_strerror(-_ret)); \ } while (0) +#define FI_RC_RETRY(FUNC,STR) \ + do { \ + ssize_t _ret; \ + do { \ + _ret = FUNC; \ + if(likely(_ret==0)) break; \ + MPIU_ERR_##CHKANDJUMP4(_ret != -FI_EAGAIN, \ + mpi_errno, \ + MPI_ERR_OTHER, \ + "**ofi_"#STR, \ + "**ofi_"#STR" %s %d %s %s", \ + __SHORT_FILE__, \ + __LINE__, \ + FCNAME, \ + fi_strerror(-_ret)); \ + mpi_errno = MPID_nem_ofi_poll(0); \ + if(mpi_errno != MPI_SUCCESS) \ + MPIU_ERR_POP(mpi_errno); \ + } while (_ret == -FI_EAGAIN); \ + } while (0) + + #define PMI_RC(FUNC,STR) \ do \ { \ diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_msg.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_msg.c index 9352936..dcad1d4 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_msg.c +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_msg.c @@ -79,7 +79,7 @@ REQ_OFI(cts_req)->event_callback = MPID_nem_ofi_cts_recv_callback; \ REQ_OFI(cts_req)->parent = sreq; \ \ - FI_RC(fi_trecv(gl_data.endpoint, \ + FI_RC_RETRY(fi_trecv(gl_data.endpoint, \ NULL, \ 0, \ gl_data.mr, \ @@ -88,7 +88,7 @@ 0, /* Exact tag match, no ignore bits */ \ &(REQ_OFI(cts_req)->ofi_context)),trecv); \ if (gl_data.api_set == API_SET_1){ \ - FI_RC(fi_tsend(gl_data.endpoint, \ + FI_RC_RETRY(fi_tsend(gl_data.endpoint, \ &REQ_OFI(sreq)->pack_buffer_size, \ sizeof(REQ_OFI(sreq)->pack_buffer_size), \ gl_data.mr, \ @@ -96,7 +96,7 @@ match_bits, \ &(REQ_OFI(sreq)->ofi_context)),tsend); \ }else{ \ - FI_RC(fi_tsenddata(gl_data.endpoint, \ + FI_RC_RETRY(fi_tsenddata(gl_data.endpoint, \ &REQ_OFI(sreq)->pack_buffer_size, \ sizeof(REQ_OFI(sreq)->pack_buffer_size), \ gl_data.mr, \ @@ -125,7 +125,7 @@ static int MPID_nem_ofi_data_callback(cq_tagged_entry_t * wc, MPID_Request * sre switch (wc->tag & MPID_PROTOCOL_MASK) { case MPID_MSG_CTS: vc = REQ_OFI(sreq)->vc; - FI_RC(fi_tsend(gl_data.endpoint, + FI_RC_RETRY(fi_tsend(gl_data.endpoint, REQ_OFI(sreq)->pack_buffer, REQ_OFI(sreq)->pack_buffer_size, gl_data.mr, diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_tagged_template.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_tagged_template.c index 3c6140b..28d33c7 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_tagged_template.c +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_tagged_template.c @@ -70,9 +70,9 @@ int ADD_SUFFIX(MPID_nem_ofi_recv_callback)(cq_tagged_entry_t * wc, MPID_Request REQ_OFI(sync_req)->event_callback = MPID_nem_ofi_sync_recv_callback; REQ_OFI(sync_req)->parent = rreq; #if API_SET == API_SET_1 - FI_RC(fi_tsend(gl_data.endpoint, + FI_RC_RETRY(fi_tsend(gl_data.endpoint, #elif API_SET == API_SET_2 - FI_RC(fi_tsenddata(gl_data.endpoint, + FI_RC_RETRY(fi_tsenddata(gl_data.endpoint, #endif NULL, 0, @@ -168,7 +168,7 @@ ADD_SUFFIX(do_isend)(struct MPIDI_VC *vc, ssend_match = init_recvtag_2(&ssend_mask, comm->context_id + context_offset, tag); #endif ssend_match |= MPID_SYNC_SEND_ACK; - FI_RC(fi_trecv(gl_data.endpoint, /* endpoint */ + FI_RC_RETRY(fi_trecv(gl_data.endpoint, /* endpoint */ NULL, /* recvbuf */ 0, /* data sz */ gl_data.mr, /* dynamic mr */ @@ -178,9 +178,9 @@ ADD_SUFFIX(do_isend)(struct MPIDI_VC *vc, &(REQ_OFI(sync_req)->ofi_context)), trecv); } #if API_SET == API_SET_1 - FI_RC(fi_tsend(gl_data.endpoint, /* Endpoint */ + FI_RC_RETRY(fi_tsend(gl_data.endpoint, /* Endpoint */ #elif API_SET == API_SET_2 - FI_RC(fi_tsenddata(gl_data.endpoint, /* Endpoint */ + FI_RC_RETRY(fi_tsenddata(gl_data.endpoint, /* Endpoint */ #endif send_buffer, /* Send buffer(packed or user) */ data_sz, /* Size of the send */ @@ -333,7 +333,7 @@ int ADD_SUFFIX(MPID_nem_ofi_recv_posted)(struct MPIDI_VC *vc, struct MPID_Reques msg.ignore = mask_bits; msg.context = (void *) &(REQ_OFI(rreq)->ofi_context); msg.data = 0; - FI_RC(fi_trecvmsg(gl_data.endpoint,&msg,msgflags), trecv); + FI_RC_RETRY(fi_trecvmsg(gl_data.endpoint,&msg,msgflags), trecv); END_FUNC_RC(FCNAME); } ----------------------------------------------------------------------- Summary of changes: src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_cm.c | 18 ++++++++-------- .../ch3/channels/nemesis/netmod/ofi/ofi_impl.h | 22 ++++++++++++++++++++ src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_msg.c | 8 +++--- .../nemesis/netmod/ofi/ofi_tagged_template.c | 12 +++++----- 4 files changed, 41 insertions(+), 19 deletions(-) hooks/post-receive -- MPICH primary repository

1 0

[mpich] MPICH primary repository branch, master, updated. v3.2b2-28-g60482cc
by noreply＠mpich.org 05 May '15

05 May '15

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "MPICH primary repository". The branch, master has been updated via 60482cc2ca9a740d049444a9261a04a3007bc306 (commit) via 46d7e443912e210018ccae74bd82543b47fa940a (commit) from b1820e5527045e5ea9c2b11d088615bdbf87dcfa (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- http://git.mpich.org/mpich.git/commitdiff/60482cc2ca9a740d049444a9261a04a30… commit 60482cc2ca9a740d049444a9261a04a3007bc306 Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> Date: Mon May 4 00:24:02 2015 -0500 tsuite: explicitly add -lpthread when used in tests For tests that call pthread routines, add an explicit dependency. This was not an issue previously, because either the compile wrappers always added -lpthread, or the linker discovered the pthread symbols via an interlibrary dependency on libmpi. However on FreeBSD 10, the interlibrary dependency resolution appears more strict. Applications must specify any libraries directly used, thus this fix. Signed-off-by: Halim Amer <aamer(a)anl.gov> diff --git a/test/mpi/configure.ac b/test/mpi/configure.ac index 4e9f418..7715ab9 100644 --- a/test/mpi/configure.ac +++ b/test/mpi/configure.ac @@ -695,6 +695,7 @@ if test "$spawndir" = "spawn" ; then AC_SUBST(nslib) fi +threadlib="" # Headers needed for threads tests if test "$threadsdir" = "threads" ; then # Check for needed threads headers and needed and optional routines @@ -703,7 +704,7 @@ if test "$threadsdir" = "threads" ; then # we shouldn't actually need to do this, but we are dodging the chicken/egg # problem b/c we don't have a working mpicc yet and we temporarily discarded # the MPICH-set LIBS (probably the wrong thing to do) - AC_SEARCH_LIBS([pthread_create],[pthread]) + AC_SEARCH_LIBS([pthread_create],[pthread],[threadlib="-lpthread"]) AC_SEARCH_LIBS([pthread_barrier_init],[pthread]) AC_CHECK_FUNCS(pthread_create pthread_yield pthread_barrier_init pthread_barrier_wait) @@ -711,6 +712,7 @@ if test "$threadsdir" = "threads" ; then AC_CHECK_FUNCS(MPI_Init_thread) fi fi +AC_SUBST(threadlib) # Check for h_addr or h_addr_list. This is needed for the singjoin test # in manual/singjoin.c diff --git a/test/mpi/manual/Makefile.am b/test/mpi/manual/Makefile.am index 040a021..94f2687 100644 --- a/test/mpi/manual/Makefile.am +++ b/test/mpi/manual/Makefile.am @@ -16,7 +16,7 @@ noinst_PROGRAMS = singjoin testconnect testconnectserial dimsbalanced \ # directory of the MPI library. This definition is not required. noinst_HEADERS = connectstuff.h testconnectserial_SOURCES = testconnectserial.c tchandlers.c tcutil.c -testconnectserial_LDADD = $(LDADD) -lm +testconnectserial_LDADD = $(LDADD) -lm -lpthread singjoin_LDADD = $(LDADD) @socklib@ @nslib@ diff --git a/test/mpi/threads/Makefile_threads.mtest b/test/mpi/threads/Makefile_threads.mtest index 00675eb..042b91c 100644 --- a/test/mpi/threads/Makefile_threads.mtest +++ b/test/mpi/threads/Makefile_threads.mtest @@ -13,7 +13,7 @@ include $(top_srcdir)/Makefile.mtest ## then add rules/vars for mtestthread.o -LDADD += $(top_builddir)/threads/util/mtestthread.$(OBJEXT) +LDADD += $(top_builddir)/threads/util/mtestthread.$(OBJEXT) @threadlib@ $(top_builddir)/threads/util/mtestthread.$(OBJEXT): $(top_srcdir)/threads/util/mtestthread.c (cd $(top_builddir)/threads/util && $(MAKE) mtestthread.$(OBJEXT)) http://git.mpich.org/mpich.git/commitdiff/46d7e443912e210018ccae74bd82543b4… commit 46d7e443912e210018ccae74bd82543b47fa940a Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> Date: Sun May 3 22:48:03 2015 -0500 Revert "Patched libtool for inter-library dependency on FreeBSD." This reverts commit 7c0f2967f758e25000bbaba9181726cbf8864357. Signed-off-by: Halim Amer <aamer(a)anl.gov> diff --git a/autogen.sh b/autogen.sh index 5bdf317..5370dbf 100755 --- a/autogen.sh +++ b/autogen.sh @@ -925,12 +925,12 @@ if [ "$do_build_configure" = "yes" ] ; then # Newer versions should have this patch already included. if [ -f $amdir/confdb/libtool.m4 ] ; then # There is no need to patch if we're not going to use Fortran. - libtool_patch_requires_rebuild=no + ifort_patch_requires_rebuild=no if [ $do_bindings = "yes" ] ; then echo_n "Patching libtool.m4 for compatibility with ifort on OSX... " patch -N -s -l $amdir/confdb/libtool.m4 maint/darwin-ifort.patch if [ $? -eq 0 ] ; then - libtool_patch_requires_rebuild=yes + ifort_patch_requires_rebuild=yes # Remove possible leftovers, which don't imply a failure rm -f $amdir/confdb/libtool.m4.orig echo "done" @@ -939,22 +939,7 @@ if [ "$do_build_configure" = "yes" ] ; then fi fi - # Set deplibs_check_method to unknown instead of pass_all on FreeBSD. - # Because the inter-library dependency is broken on the latest - # FreeBSD production release (10.1-RELEASE), but libtool cannot detect - # it correctly. - echo_n "Patching libtool.m4 for compatibility with inter-library dependency on FreeBSD... " - patch -N -s -l $amdir/confdb/libtool.m4 maint/freebsd.patch - if [ $? -eq 0 ] ; then - libtool_patch_requires_rebuild=yes - # Remove possible leftovers, which don't imply a failure - rm -f $amdir/confdb/libtool.m4.orig - echo "done" - else - echo "failed" - fi - - if [ $libtool_patch_requires_rebuild = "yes" ] ; then + if [ $ifort_patch_requires_rebuild = "yes" ] ; then # Rebuild configure (cd $amdir && $autoconf -f) || exit 1 # Reset libtool.m4 timestamps to avoid confusing make diff --git a/maint/freebsd.patch b/maint/freebsd.patch deleted file mode 100644 index 45afb60..0000000 --- a/maint/freebsd.patch +++ /dev/null @@ -1,11 +0,0 @@ ---- ../confdb/libtool.m4~ 2015-04-25 09:07:47.000000000 -0500 -+++ ../confdb/libtool.m4 2015-04-25 09:08:17.000000000 -0500 -@@ -3503,7 +3503,7 @@ - ;; - esac - else -- lt_cv_deplibs_check_method=pass_all -+ lt_cv_deplibs_check_method=unknown - fi - ;; - ----------------------------------------------------------------------- Summary of changes: autogen.sh | 21 +++------------------ maint/freebsd.patch | 11 ----------- test/mpi/configure.ac | 4 +++- test/mpi/manual/Makefile.am | 2 +- test/mpi/threads/Makefile_threads.mtest | 2 +- 5 files changed, 8 insertions(+), 32 deletions(-) delete mode 100644 maint/freebsd.patch hooks/post-receive -- MPICH primary repository

1 0